Common Core groups should be asked plenty of questionsState Board should ask about cost, security, timeline
On Wednesday, the California State Board of Education will hear presentations from representatives of two assessment consortia that have been awarded federal grants to develop “next generation” assessments to measure the Common Core content standards adopted by states last year. The assessments developed by these two consortia will be candidates to potentially replace California’s Standardized Testing and Reporting (STAR) statewide assessment system. The reason for the presentations, instigated by Board President Michael Kirst, is to provide information leading to a decision as to which consortium, if either, California will collaborate with to provide assessment system design and test development work leading to replacement tests for STAR.
The two assessment consortia are the Partnership for the Assessment of Readiness for College and Careers (PARCC), which has roughly 25 states as current members, and the SMARTER Balanced Assessment Consortium (SMARTER Balanced), which has roughly 30 states as current members. States may join either or both consortia as “participating” members, but to be a “governing” member with greater leadership obligations and formal voting status, they can only be a member of one consortium. About 15 states are currently governing members for each consortium, while other states belong to either both consortia or only one as “participating” members; a few states belong to neither. To belong to one of the assessment consortia, states must have formally adopted the Common Core content standards developed last year by the National Governors Association and Council of Chief State School Officers. Adopting the Common Core standards was necessary to be competitive for the Race to the Top competition. California adopted the content standards on August 2 last year, the deadline under Race to the Top.
PARCC and SMARTER Balanced were awarded funds last fall, and have been organizing their efforts since then. The grants are between $150 million and $200 million for each consortium and must be spent by the 2014-15 school year, when the states implement the assessments. The consortia’s applications have a lot in common, but a close reading of the design and development details reveals distinct philosophical and operational differences.
In considering what to do about consortium membership, State Board members should ask questions that flesh out how details in each consortium’s plan may affect California’s plans to replace the STAR assessment system. Here are questions I would ask and the context behind them.
- California’s Role in Consortia Efforts: California policymakers need to decide whether they wish to be a “governing” member of only one consortium, or a “participating” member of both consortia, or join neither at this time. California is currently a participating member of only the PARCC consortium, a decision made by the previous governor, State Board, and Superintendent of Public Instruction. Questions: What are the advantages/disadvantages of being a governing vs a participating member of PARCC and/or SMARTER Balanced? Will a participating state be able to influence PARCC/SMARTER Balanced strategic directions despite its non-voting status?
- Impact on Curriculum and Instruction: Both consortia plan to include Through Course assessments in their programs. Through Course assessments were required by the federal request for proposals and were defined as “components that are administered periodically during the academic year” yet contribute to a student’s final score. Question: What will PARCC/SMARTER Balanced do to address the concern that Through Course assessments will lead to a uniform “pacing guide” program for curriculum and instruction for all California schools, based on the timing of Through Course test administration schedules?
- Use for Teacher Evaluation: Much attention has been paid to potential use of statewide test results for evaluation of individual teachers in recent months, as well as possible use for layoff and assignment and compensation decisions. Question: Will PARCC/SMARTER Balanced assessments be developed to support this degree of high-stakes accountability?
- Computerized Testing: Both consortia include computerized testing in their plans, with SMARTER Balanced anticipating computer-adaptive tests (i.e., test question sequence depends on student response pattern, thus reducing the number of test questions that need to be administered) while PARCC anticipates computer-administered tests (i.e., fixed-length forms of tests administered via computer). These plans will depend on availability of computer hardware for test administration at all schools in the state. Question: Do PARCC/SMARTER Balanced have specifications for the computer hardware needed to administer their anticipated assessments so that California may estimate the cost for computer infrastructure necessary for administration of PARCC/SMARTER Balanced tests at each school site?
- Open-Ended Item Formats: In response to federal specifications, both PARCC and SMARTER Balanced anticipate extended use of open-ended test item formats rather than multiple-choice formats. Such formats provide increased risk for breaches of test security. Question: How will PARCC/SMARTER Balanced address the concern that extended use of open-ended item formats (i.e, constructed response, essays, projects) will increase opportunities to compromise test security and promote teaching-to-the-test behavior, thus compromising use of test results for accountability purposes?
- Augmentation: California added to the Common Core content standards when adopting them last August, in particular for Algebra I standards in grade 8 and additions in earlier grades to get students ready for Algebra I in 8th grade. We will have to add test questions to the base Common Core assessments in order to measure the full range of California’s adopted content standards. Question: Will PARCC/SMARTER Balanced assist California with California’s augmentation requirements, in terms of both test development projects and costs?
- Ongoing Test Development: For test security reasons, each state will need ongoing test development efforts to generate replacement test questions required to keep annual test forms “fresh” and mitigate efforts to narrowly “teach to the test.” Question: How will PARCC/SMARTER Balanced be able to assist California with these ongoing test development efforts in terms of (a) coordination with base PARCC/SMARTER Balanced tests, and (b) minimizing ongoing test development costs?
- Costs: We understand PARCC/SMARTER Balanced will cover at a minimum the costs for developing assessments for the base Common Core content standards, but not costs for administering PARCC/SMARTER Balanced tests annually. Questions: Do PARCC/SMARTER Balanced have information for estimating annual per-pupil costs for administering their base assessments? How will PARCC/SMARTER Balanced operational test administration costs compare to California’s current annual per-pupil cost of roughly $13/student [i.e. $60 million for 4.8 million students] for our current STAR assessment program?
- Unrealistic Design Features and Timelines: Questions: What are the chances PARCC/SMARTER Balanced may have bitten off more than can be chewed by the 2014-15 target date? What are the chances that political changes in Congress may kill or significantly alter the PARCC/SMARTER Balanced initiatives before completion?
- History: In the early 1990s, California undertook an ambitious “beyond-the-bubble” constructed response statewide assessment program labeled the California Learning Assessment System (CLAS). After roughly four years of development and initial implementation, it crashed and burned in 1994 via a veto from then Gov. Pete Wilson, after a review led by distinguished educational measurement expert Lee Cronbach from Stanford University found that CLAS did not generate individual-student-reliable scores as required by authorizing legislation. CLAS cost California taxpayers more than $100 million in early 1990 dollars. Question: How can California be confident that the PARCC/SMARTER Balanced assessment systems will not suffer a similar fate?
The questions do not cover all pertinent topics for the PARCC and SMARTER Balanced representatives. I encourage readers to enter their own comments and questions in the space below.
Doug McRae is a retired educational measurement specialist living in Monterey. In his 40 years in the K-12 testing business, he has served as an educational testing company executive in charge of design and development of K-12 tests widely used across the US, as well as an adviser on the initial design and development of California’s STAR assessment system. He has a Ph.D. in Quantitative Psychology from the University of North Carolina, Chapel Hill.
- “Taxpayers, Grab Your Wallets” | Inside Scoop with Alice Linahan
- Pastor Mikes Report | Kentucky Withdraws from Common Core Assessment Consortium
- Kentucky Withdraws from Common Core Assessment Consortium
- Kentucky Withdraws from Common Core Assessment Consortium | Political Thrill