The five unidentified reviewers of California’s Race to the Top application generally praised the district-led approach that California took and expressed optimism that ceding control to districts committed to reform could succeed.

But what stopped them, with one exception, from giving the state winning marks were three areas that participating superintendents and state officials figured would be problematic: lack of union support for the application; a troubled statewide data system that lags behind other states; and uncertainty whether the state could deliver commitments to create more effective teachers and principals.

The U.S. Department of Education released the scores and evaluations of states in the second round of Race to the Top on Wednesday. It showed that especially in California, the numbers were all over the place. You’d have thought that two of the reviewers had read different applications.

————————————————————————–

Average scores of California’s judges in key categories:

Articulating reforms, district participation: 51 of 65 pts.

Building capacity to implement plan: 25 of 30 pts.

Progress in closing achievement gap: 24 of 30 pts.

Standards and assessments: 69.2 of 70 pts.

Data systems to support instruction: 30.8 of 47 pts.

Producing great teachers and leaders: 107.8 of 138 pts.

Turning around lowest-achieving schools: 45.8 of 50 pts.

Making education funding a priority: 9  of 10 pts.

Creating environment for charter schools: 38.6 of 40 pts.

Demonstrating reform conditions: 4.8 of 5 pts.

Making STEM a priority: 15 of 15 pts.

Total: 423.6 of 500 pts.

——————————————————————–

Reviewer 2 gave California an enviable 466 points out of 500. Reviewer 4 awarded only 385: 81 points below Reviewer 2 and 38 points below the state’s average of 423. The other three judges (403, 424, 440 points) were closer to agreement.

The variation between California’s high and low scores will feed the arguments of critics, who have charged that Race to the Top’s complex scoring metric is subjective and unreliable.  Certainly in the case of Colorado, which most observers expected to among the winners, an outlying judge’s inexplicably low score of 339 probably put Colorado out of the running. And in nine of the 10 winning states in the second round, there was less variation in scores (only 19 points separating top and bottom scores in Rhode Island, 44 point spread in Massachusetts). (update: Rob Manwaring of the Quick and the Ed calculated that throwing out the high and low judge scores for all of the states wouldn’t have changed who got the money. Colorado would have moved up, but not far enough. )

But weaknesses in the application, not fickle judges, were to blame for California’s 16^th place. Throw out the judges’ high and low scores, as some advocate, and the results wouldn’t have changed.

Because it has strong charter school laws, has adopted the Obama administration’s method for turning bad schools around, passed common-core standards and will participate in creating national assessments, California got nearly the maximum points in several key areas.

But judges differed on overall prospects of success. Only one third of teachers unions in the participating districts – and only two in seven lead districts – signed on. Both Reviewers 2 and 4 agreed that would create doubts. But when it came to scoring, one was Pollyanish, the other pessimistic. Reviewer 2 gave the state 55 of 60 points, while Reviewer 4 knocked it down to 36, noting that the state’s premise – that districts would lead by example – assumed teachers unions would agree to new rules on tenure, school assignments and performance evaluations, the key criteria under Race to the Top for producing “great teachers and leaders.”

While Reviewer 2 bought the state’s plans to equitably distribute teachers and principals, Reviewer 4 was skeptical and awarded only 13 of 25 points, because unions in each district might alter the terms or scotch the deal. “The application seems to spin out ideas and examples of what could be done rather than offer concrete plans to be implemented,” Reviewer 4 wrote.

While two judges gave nearly the maximum 30 points for showing progress in closing the achievement gap, Reviewer 4 observed that the state’s graduation rate has barely budged and its scores on NAEP, the national standardized test, have shown little progress.

In one area, even Reviewer 4 was fooled: The state and the seven lead districts convinced the judges that California makes “education funding a priority”  and equitably funds low-income schools – an area worth 10 points. (The argument has to do with the percentage of spending on education in relation to the whole state budget. Education groups have sued over this issue.)

Promises, promises

In some of the dozen winning states,  legislatures  passed laws (not all of them sensible)  dealing with teacher performance and tenure.  Others have reached agreements with unions on assigning effective principals and teachers to failing schools. But for the most part, Race to the Top is a collection of promises, backed by MOUs. Now winning states must deliver.

As American Enterprise Institute researcher and blogger Rick Hess noted this week, “This emphasis on promises and practices means that fidelity of implementation becomes nearly the whole ball game.”

Hess pointed out that 80 percent of the dozen winning states  will or may have new governors come November, raising questions whether the next administrations will follow through with the plans.

California’s superintendent-initiated approach, with an independent organization at the helm, would shield it from changes in administration and some of the vagaries of Sacramento politics. It’s too bad California won’t be able to prove  its model would work.