EQI may replace API in rankings
Steinberg bill to use multiple measures
California’s Academic Performance Index (API) was never supposed to be based on a single test. When it was created as part of the Public Schools Accountability Act of 1999, the legislation was clear on that point.
“This bill would require the Superintendent of Public Instruction, with approval of the State Board of Education, by July 1, 1999, to develop the Academic Performance Index (API), consisting of a variety of indicators, to be used to measure performance of schools, especially the academic performance of pupils.”
In the dozen years since, the only variety has been in the changing high-stakes tests used to determine the school rankings. That would change under a set of bills approved yesterday by the Assembly education committee. SB 547, 611 and 612, by Senate President pro Tem Darrell Steinberg (D-Sacramento), would reduce the emphasis on the California Standards Test by limiting the exams to no more than 40 percent of a high school’s overall ranking, and a minimum of 40 percent for middle and elementary schools.
The new measurement system would also replace the API with a new system known as the Education Quality Index, or EQI, which would be based on multiple measures developed by a committee headed by State Superintendent Tom Torlakson. For starters, however, SB 547 calls for including graduation rates and how well schools prepare students for college and career success. And the bill allows more measures to be added down the road.
Free throws aren’t everything
Years ago, when I wrote about critics who said the API measure was too narrow, Jay Rosner, executive director of the Princeton Review Foundation, used a sports metaphor to illustrate. If Shaquille O’Neal were judged only on his free throws, he’d never have made it to the pros, said Rosner; that wasn’t his strength. Steinberg and California school superintendent Tom Torlakson reached into the wide world of sports to make a similar point in an OpEd that appeared in Wednesday’s Sacramento Bee.
“No one would judge Giants pitcher Tim Lincecum’s performance based on one inning,” they wrote, “why should parents and the public judge a school based on one set of tests?”
They warn that too much emphasis on standardized tests narrows the curriculum to the subjects being tested and ignores other important aspects of education.
“Are students staying in school or dropping out? Are they ready to continue their education? Do they have the training and skills to start a career? A test score alone won’t answer those questions.”
Ready for change
Just a year after California enacted the Public Schools Accountability Act, Congress approved No Child Left Behind, which added even more significance to high-stakes tests. Now it looks like history may repeat itself in reverse. U.S. Education Secretary Arne Duncan has already announced (and we reported here) that unless Congress takes up reauthorization in the next few weeks, the Department of Education will start granting waivers to give states some flexibility from the severe consequences if every student doesn’t score at the proficient level or better on their state exams.
Duncan warned that without a relaxation of those provisions, more than 80 percent of the nation’s schools may be mislabeled as failing.
Susanna Cooper, a consultant to Sen. Steinberg, says the frustration that’s been building among educators and state policy makers over the testing may account for the lack of opposition at yesterday’s committee hearing. In the long queue of people waiting to comment on the bills, not one spoke against them. Some said their organizations hadn’t taken a stand yet, but added that they generally support the measure.
“I think the amount of support for the bill is striking because it (the bill) represents significant change, but I think that’s a reflection of the current limitations of the accountability system,” said Cooper. “I think people are just ready to do something else. We’re at a point where the system is ready to embrace change.”
Is EQI planned to be a high stakes measurement?
Report this comment for abusive language, hate speech and profanity
Hi Paul,
I put your question to Susanna Cooper with Senate President pro Tem Darrell Steinberg’s office. Here’s what she wrote:
You are right that for now the EQI would essentially replace the API (there would be some “cleanup” of the code needed to address all the places where API appears – that would be done in follow up legislation). Right now, API doesn’t have a lot of stakes associated with it, other than a public recognition of how a school performs on the index. Some of the funding streams, like HP/IIUSP, that were once tied to API scores are no longer funded. We do use API to help determine eligibility for certain programs (open enrollment act, parent “trigger”, the list of chronically low performing schools eligible for SIG grants). So we would have some work to do moving forward to switch over to the EQI as a determiner of such things. Once the EQI is established, the Legislature could consider how to use it. We are interested in incentives/resources to help schools improve, and not in sanctions or punishments.
The stakes now for schools are much higher in the federal accountability world – did they make AYP or not?
Everyone eagerly awaits reauthorization of ESEA to see how that will alter federal accountability. Our desire to move now on a new state level accountability system is not just an effort to create a more balanced an rational system here in CA, but also to influence the conversation in DC. We want true multiple measures accountability at both state and national levels.
Report this comment for abusive language, hate speech and profanity
And to further clarify (and hopefully not contradict Susanna), the EQI would include standardized test scores, just as API does now (except it will call it the SAI, or State Assessment Index). So for all of you who place a premium on objective, standardized tests, the new EQI will include those (so nothing new on that particular front).
However, a school’s EQI score will include more than just their SAI score; it will have other indices of a school’s performance, including dropout rates, college preparation, and career readiness.
This is a very exciting opportunity to finally provide the media, parents and students a much clearer picture of how successful (or not) their community school truly is, going beyond just the memorization/regurgitation of its students in “bubble tests” of narrowly confined English/Math questions. I believe this represents true accountability, with the ability of the public to drill down into specific indices of performance that are relevant and meaningful. Thank you Senator Steinberg and SPI Torlakson!
Report this comment for abusive language, hate speech and profanity
Just dump the tests in favor of portfolio based assessment using a ready made rubric for each grade. Administrators teachers and maybe PTA can b involved… Peer review of open ended real life approach.
We are building mountains of regulations over a failed premise. Multiple choice IS the problem and no solutions can be based upon these narrow measure.
http://specialedvice.blogspot.com/2011/06/charter-schools-and-credit-default.html
Report this comment for abusive language, hate speech and profanity
Is there a way to incorporate on-site evaluations, such as those done by WASC?
Report this comment for abusive language, hate speech and profanity
This presents a new opportunity for students across California do get the quality education delivered in 49 other sates. If one of the new EQI requirements were to be that each school should have a quality school library, staffed with a credentialed librarian and budgeted to provide fresh materials, CA could one day hope to overcome the worst level of school library service in the nation.
Report this comment for abusive language, hate speech and profanity
Kudos to Steinberg and Torlakson for taking on this critical issue. Judging a school exclusively on one measure makes no sense, especially when other measures are readily available. Standardized tests can be good proxies for the quality of education that students at a school receive. But, when East tests become high-stakes, they can create unintended consequences including unerring of the curriculum and teaching to the test. Including measures like how well students actually do is they transition into college is a much better indicator of whether a high school is successful or not. In addition creating incentives for more kids to take and pass advanced placement courses just makes sense. With the rising costs of higher education, helping students get a leg up in college by taking AP courses is something that an accountability system should reward. Other states like Florida and Oklahoma have already started to incorporate these types of measures into their accountability frameworks. This proposal would go to even further by also incorporating career readiness indicators.
The one area where this bill could make further improvements is by adding in measures of student growth. We know that the existing accountability measures, as well as many of the measures proposed in this bill, are highly correlated with student demographics. Measuring the year-to-year progress of students will better account for the quality of education that a school is providing and are not dependent upon the school’s underlying demographics. Here’s a link to a recent paper that I wrote while still at Education Sector on implementing student growth models (http://www.educationsector.org/publications/growth-models-and-accountability-recipe-remaking-esea). Whether through reauthorization of ESEA or through federal Department of Education waiver process, there is likely to be opportunities for California to transform how it implements federal accountability for schools. But, any federal process will certainly require that student growth be a component in a state’s accountability system. As SB 547 moves forward Steinberg and Torlakson should figure out how student growth factors into an overall accountability system.
Report this comment for abusive language, hate speech and profanity
Yes. There is, at this time, no limit to the number of measurements or indices that the advisory committee can recommend. Senator Steinberg also said that he wants input from teachers and the public.
Report this comment for abusive language, hate speech and profanity
That above comment is in response to el’s question.
Report this comment for abusive language, hate speech and profanity
If I may, I’d like to build upon the points made by Rob by directing readers to a recent PPIC report (of which I was a co-author): http://www.ppic.org/main/publication.asp?i=909
LAUSD began reporting value-added measures of school effectiveness this year, and similar measures could–and perhaps should–be implemented statewide.
Factors over which a high school has little control–the student’s home, the student’s neighborhood, and elementary and middle schools attended by the student–have a significant effect on the likelihood that a student drops out or graduates from high school college- and/or career-ready. It is not fair, and perhaps counter-productive, to hold schools accountable for outcomes over which they may have very little control.
Julian Betts and Andrew Rice of UCSD and PPIC have demonstrated that it is possible to use student information from as early as 4th grade to predict with a fair degree of accuracy whether a student passes the CAHSEE. The same concept could be applied to many of the student outcomes Steinberg has proposed for the EQI. Using statistical methods similar to those used in value-added modeling, the state should develop a model that uses a student’s pre-high school test scores, attendance, and other factors–call them prexisting conditons–to predict the likelihood that a student drops out or graduates from high school having met certain requirements. Actual outcomes at each school could then be compared to predicted outcomes to determine whether each high school is doing better or worse than one would expect, given these prexisting condtions.
As Rob points out above, API may not be a valid measure of school effectiveness. Unless changes are made to SB 547 to account for students’ “prexisting conditions,” the bill will “improve” API by holding schools accountable for a number of other factors that are not valid measures of school effectiveness.
Report this comment for abusive language, hate speech and profanity
Another question, the additional measurements in the EQI seem to fit high schools much more easily. There are career oriented options and A-G requirements for college preparation. But it doesn’t seem possible to apply dropout rates or career and college readiness to elementary schools. Are there any measurements on the table for elementary schools?
Report this comment for abusive language, hate speech and profanity
The status quo would be only too happy to engage the debate about accounting for exogenous factors in student performance, as it would kick the can down the road and postpone reckoning, ideally until the ‘latest and greatest’ mandated change loses favor with the reliable cowardly political class. Our problem is more mundane; until you drill down to the statistical level of individual teachers working in the classroom, until you understand what an individual teaching practitioner does well and does not, until you have a professionally supported judgement about the quality of transactions that a teacher controls, any focus on school-level effects is secondary or worse, diversionary. School effectiveness is the sum of the effectiveness of individual teacher’s transactions, whatever the disparity and diversity of inputs. And you cannot mandate effectiveness by policy fiat or remote pontiffication; you have to build teachers up one at a time, and this can only be done by other teachers who share their work and their students. It’s hands-on work, slow and inglorious. Everyone not in the classroom seems delighted to tell teachers what to do, but unless you are authentically working to improve students hands on, unless your credibility flows from today’s classroom, you will not prevail, despite every effort. It’s simply too intractable. Teachers must salvage themselves. Thank you.
Report this comment for abusive language, hate speech and profanity
The move away from a focus on summative assessment is the long-term goal of those who have opposed accountability from the start. The EQI will diminish a focus on academic accounability in favor of broader measurements of school “health”, furthing the demise academic accountability. Just wait until a dozen indices are added to the EQI, covering everything from the “campus cleanliness index” to the “social harmony index” (one can only imagine how far this will go). In the end, we’ll return to the days of graduating students who read and write at elementary levels, but it will be OK because they will feel really good about themselves. Good grief.
By the way, somebody needs to inform Fred that California’s state assessment system includes more than English and math tests.
Report this comment for abusive language, hate speech and profanity
Paul,
The bill does seem to be more focused on multiple measures for high school where it lists some specific indices. Here’s what it says about elementary school (the SAI is the new acronym for test scores):
(3) (A) For schools and school districts maintaining kindergarten
or any of grades 1 to 7, inclusive, the EQI shall include, but shall
not necessarily be limited to, the SAI, as described in Section
52052.81.
(B) It is the intent of the Legislature that the EQI for schools
and school districts maintaining kindergarten or any of grades 1 to
7, inclusive, not be limited to the SAI, as described in Section
52052.81.
Report this comment for abusive language, hate speech and profanity
Has anyone examined students’ and parents’ decision-making processes during the school year? The “system” in our public schools is comprised of teachers, parents, students, administration, counselors, special education departments, and hopefully other support services like libraries, nurses, technology departments, fine arts, etc. To penalize teachers alone for decisions made by these other players, especially students and parents, doesn’t make sense.
Has anyone considered the possibility that some communities’ parents and students have decided not to try to master given subject matter? Not all communities consider state-mandated curricula relevant, no matter how appealing and comprehensible teachers make it. Some communities are very entrenched in their determination to value endeavors unrelated to knowledge and skills mastery deemed critical to a good education by the state (legislature, experts in the field, the voting public, etc.). Many parents are completely illiterate or are otherwise unable to support their child’s progress through school starting at the very earliest ages. They often base their judgment of teacher effectiveness on factors unrelated to best professional practices. Administration seems biased in favor of parents no matter how misinformed they may be. It is impossible to keep the best and brightest teachers under these circumstances.
Report this comment for abusive language, hate speech and profanity