Judge to rule on teacher evals
Landmark decision on using test scores
A Los Angeles County Superior Court judge has promised to finish up this weekend his decision on whether state law requires school districts to consider student test scores in evaluating teachers.
A landmark ruling on behalf of students and parents suing Los Angeles Unified and the district’s teachers union would give Superintendent John Deasy the muscle of the law to press ahead with adopting growth in student performance as one of several metrics for teacher evaluations. It would signal to other school districts that they must also do so in some fashion. A ruling against the plaintiffs might not change the status quo, since few school districts currently use student test scores in formal evaluations.
Oral arguments in the case had been scheduled for Tuesday, and Deasy, who’d been subpoenaed to testify, was in court. But after Judge James Chalfant indicated he had nearly completed his tentative ruling, attorneys for both sides agreed to wait until the next hearing, on June 12, to respond to what he has written. Chalfant didn’t preview his position, although he did dispute a characterization by the attorney for the district of the broad issue in the case – possibly a sign which way he’s leaning. He did make clear several times that he was thoroughly versed in the case, had read the record and recognized the case’s importance.
What does Stull Act require?
Sacramento-based nonprofit EdVoice, filing suit for the parents, has forced the issue in Jane Doe et al vs John Deasy. It argues that the four-decade-old Stull Act, the state law laying out procedures for teacher and principal evaluations, requires that evaluations consider “pupil progress as it reasonably relates” to district standards and to state academic standards as measured by the California Standards Tests. Among those agreeing with that position is Los Angeles Mayor Antonio Villaraigosa, who, when he was speaker of the State Assembly, sponsored a bill that updated the Stull Act.
Los Angeles Unified has extensive data tying student test scores to teachers. It created a system, called Academic Growth over Time (AGT), that uses demographic data on individual students and their results on past years’ tests to project future results. Teachers are scored on whether students in their classes exceed or fail to meet those expectations.
Deasy has shared AGT scores with individual teachers, though they haven’t counted in evaluations yet. He hasn’t said what percentage of an evaluation AGT would comprise, although he said it would be a minor piece.
The plaintiffs argue that, in failing to incorporate test scores, the district “has relinquished its obligations to the students in order to placate more powerful interests” – in other words, caved to union pressure. As evidence, they point to 600 randomly selected teacher evaluations that they reviewed. While 98 percent of teachers had satisfactory reviews, less than 3 percent made any reference to standards or tests, attorneys said, and the evaluation forms contained no questions requesting information on students’ learning relative to standards. The plaintiffs argue that the failure to evaluate teachers effectively violates students’ civil rights, denying them their constitutional right to an opportunity for a quality education.
Deasy has proven to be the plaintiffs’ star witness. Their brief begins by quoting from his deposition: “We do not currently construct evaluations of teachers by using how students do over time in terms of their academic outcomes.” On Tuesday, after a brief hearing, Deasy told reporters that he looked forward to a ruling that would help clarify the state law and support his effort to use AGT districtwide. He acknowledged that the district and UTA had discussed trying to settle the lawsuit, without success.
Deasy’s statements under oath notwithstanding, the district’s formal position is that it is complying with the law. The AGT model that the district is piloting is proof that the district is linking test scores to teacher evaluations. (This argument glosses over that actual implementation of AGT is at least a year away and will be contested in court by United Teachers Los Angeles.) The district’s teacher evaluation form also contains many questions that relate to student performance, including whether a teacher:
- Demonstrates knowledge of state standards and student development;
- Plans and implements classroom procedures that support student learning;
- Uses the results of multiple assessments to guide instruction.
Lawyers for the parents, however, argue that these areas may examine and support effective teaching, but don’t tell you whether student learning has actually increased.
United Teachers Los Angeles makes a very different argument. Contrary to what EdVoice, Villaraigosa, and the parents claim, UTLA says point blank, “The Stull Act does not prescribe how local school districts must conduct employee performance evaluations.” The criteria for evaluations are left to the school district, and any changes must be collectively bargained. And the district has wide discretion in determining how pupil growth “reasonably relates” to standards of student achievement, the UTLA argues in its brief.
It’s so stupid that this would be decided by a judge. The criterion should be: Is this a valid measure of the effectiveness of a teacher? All the science shows that it’s not, so it’s insane that it would be pushed forward. The Los Angeles Times bears a huge share of the blame.
Report this comment for abusive language, hate speech and profanity
Caroline – This really isnt a matter of whether you like it or not. It’s simply a matter of what the law prescribes and whether school districts are following the law. Arbitrating such matters is completely within the purview of a judge. If it’s not a “valid measure of the effectiveness of a teacher” then it’s the job of the legislature to change that.
But in looking at the Stull Act it appears the legislature already determined that test scores are a pretty good measure of effectiveness.
Report this comment for abusive language, hate speech and profanity
Understood; I was just voicing my dismay.
The fact that legislators with no understanding of education set policy, while being heavily influenced by those who profit from our kids’ public education funding (in this case the mighty testing industry), is a deeply problematic situation. Legislators make their decisions to benefit the private for-profit sector, and children and education are collateral damage.
Report this comment for abusive language, hate speech and profanity
Capitol, I dont believe the stull act does state that test results are a good, direct measure of effectiveness. Not only does it not specify how the results are to be used, but in a number of places the stull act clearly states that the impact needs to ‘reasonably relate’ to the measure in question. That language seems to clearly be leaving that decision up to the evaluators/-ions in question. There is evidence that test results relate much more strongly to things other than teacher effectiveness.
http://tinyurl.com/7hsb2rz
Report this comment for abusive language, hate speech and profanity
It’s all in how you use them. Test scores are *interesting* insights into a teacher. They give you a set of questions to start with.
I wouldn’t want a teacher to be retained over the objections of teachers, students, and administration just because his students had high test scores, just as I wouldn’t want a teacher to be fired over the combined objections of teachers, students, and administrators based on low test scores. You have to ask why. You have to look for yourself to see what might be going on.
Take the common example of kids filling out the test answer sheets in patterns instead of attempting to take the test in good faith. Obviously, the test in that case is not a valid measure of the student’s knowledge or mastery of the subject. It is a measure of his respect for himself, his teachers, his parents, his school, and his attitude towards academics. It doesn’t necessarily mean that the teacher is at fault, but somewhere we’ve failed this child. It also doesn’t mean the teacher is blameless. The actual answer will be different in different cases.
Report this comment for abusive language, hate speech and profanity
Agreed El, but I would just point out that we cannot even currently properly disaggregate the data we do have, let alone add new metrics to tease out new nuances.
I am definitely a fan of the concept of accountability and especially transparency, but our implementation of the former seems to be when we do something wrong or improperly, and it does not work, that our ’solution’ is to do the same thing even more wrongly or more improperly. Classic American approach to politics, imho. If it werent our kids who were paying the price, it would be a joke..
Report this comment for abusive language, hate speech and profanity
Reacting to this:
As evidence, they point to 600 randomly selected teacher evaluations that they reviewed. While 98 percent of teachers had satisfactory reviews, less than 3 percent made any reference to standards or tests, attorneys said, and the evaluation forms contained no questions requesting information on students’ learning relative to standards.
It seems to me that you could meet the Stull Act requirements with a sentence or two mentioning test scores. IE, “Test scores for Ms. M’s classes were evaluated and found to be [ 'above average' | 'average' | 'below expectations' | 'in line with other teachers' | 'somnolent' | 'dismaying' | 'green' | 'extraordinary' | 'difficult to understand' ].
Report this comment for abusive language, hate speech and profanity
I don’t think it could be any clearer. There is a requrement to create a direct connection between the evaluation of the teacher and the progress of student test scores. I will just paste the language here and appeal to the judgment of the readers:
(a) The governing board of each school district shall
establish standards of expected pupil achievement at each grade level
in each area of study.
(b) The governing board of each school district shall evaluate and
assess certificated employee performance as it reasonably relates
to:
(1) The progress of pupils toward the standards established
pursuant to subdivision (a) and, if applicable, the state adopted
academic content standards as measured by state adopted criterion
referenced assessments.
…
Report this comment for abusive language, hate speech and profanity
I expect that Judge Chalfant will spend considerable attention interpreting what “reasonably relates” means. It should be an interesting decision.
Report this comment for abusive language, hate speech and profanity
I realize I am probably an outlier, but I assessed myself as a teacher using multiple measures (valid national assessments, portfolio review, student growth in writing, etc.), and I never had a problem with being evaluated by administrators. I think that coming from a system where you could be hauled over the coals for poor student examination scores (published in the newspapers) made a difference!
I started every school year with a specific evaluation of students, then looked also at their incoming portfolio assessments if they had them, and eventually at CA testing results. I discussed where I found each student to be (yes, it took time), and growth then becomes a team effort – and parents were in the loop as well. Maybe setting off with a team approach would ease the resistant into the system!
It will be interesting to see what the Judge says. I’d at least like agreement that teachers who consistently lead students downwards are moved to another career!
Report this comment for abusive language, hate speech and profanity
When the quality of a student’s education depends on the evaluation by a judicial representative that is bound by word games; we are in trouble. Each and every student should have access to meaningful educational experiences. The evaluation of that experience has to be reflected in some measure of assessment. The real question is the validity of that assessment. Accountability has to be measured in the reality of the teachers’ controlling interest. Administrative oversight can provide the buffer to naked statistics. After 28 years in administrative positions, I know that teacher evaluations are a compilation of observation and data. Trust me, human assessment is as important as data.
Report this comment for abusive language, hate speech and profanity
Looks as if the words “if applicable” under (b) (1) (a) might be open to interpretation.
So how do we evaluate teachers of subjects and grades that aren’t tested? Do we test kindergartners, preschoolers, art students? Good news for the testing companies, certainly. For the rest of us, not so much.
Report this comment for abusive language, hate speech and profanity
Any teacher who “consistently lead(s) students downward” is already known to be a problem to administrators, and if they are competent they will bite the bullet and deal with that. (No, it’s not “impossible.”)
I asked this question of the deity of teacher-bashing, Eric Hanushek, at a UC-Berkeley panel a few years ago on the LA Times teacher evaluation project, after Hanushek admitted that VAM was only valid in revealing the very top and very bottom teachers: Doesn’t a competent administrator already know who the very top and very bottom teachers are? He admitted that yes they did. (To clarify, I submitted the question in writing; it was read to Hanushek by the moderator, Louis Freedberg.)
Report this comment for abusive language, hate speech and profanity
The thing that I find fascinating here is that anyone would object to utilizing pupil progress as it “reasonably relates” to district standards when evaluating teachers. The language allows flexibility in the use of this data, while use of the AGT data offers an objective measure that would lend credibility to teacher evaluations. Parsing words and arguing the letter of the law misses the point: in all discussions of public education it is vital to remember that the students are our first concern, not the teachers, not the administrators.
Report this comment for abusive language, hate speech and profanity
I think people are getting lost in tangents. This story is about a judge who is going to interpret a law. That, by definition, means parsing words and extracting nuance. This does not have to mean anything about common sense nor what ’should’ be happening. Obviously, it ideally would, but for that you need to have a legislature and voting populace that have at least some minutiae of real understanding of our education system. Something we don’t have.
In addition, the question about ‘reasonably relates’ is not whether student progress reasonably relates to teacher ability, rather it is about whether standardized assessments reasonably relate to teacher ability and student progress. The last california teacher survey I saw said that 55% of teachers believe their students do not take standardized tests seriously. That would be the first knock. Student tests scores are highly correlated to things like ethnicity, race, socioeconomic status, special ed status, native language, and yes, parent education level. There is a reason we disaggregate all those things. All of the rhetoric surrounding teacher evaluations is very careful to say that teachers are the most important ‘in school’ factor related to student success. This is because it is painfully clear that there are other, much larger factor’s that are either out of school or not related to the teacher. The concern with using tests is not whether they tell us something, its whether they tell us something about the teacher. In the graph I linked to in an earlier post, the difference in proficiency rates in the state of california last year between children from parents who went to grad school vs those who did not have high school diploma is almost 50 percentage points. No, they dont differ by 50%, they differ by almost 50 percentage points!! For 4th grade, for example, which is the high point for all groups, the students in the grad school group were at 89% proficiency. The no high school group, 45% proficiency. We are not talking about nuances here. That is for the entire state of california last year.
In addition, I encourage people to ask their own districts for their API calculation numbers, which will include disaggregation of test and API numbers for each ethnicity, and separated by special education status and grade level. It is not uncommon to see differences for the same ethnicity in middle and high school to be on the order of 300 to 400 points in API between special ed and non special ed. Thats for the same ethnicity, within the same district! Again, we are not talking nuances. This is all the more startling when one puts that together with the fact that special education rates differ by ethnicity in our state. And for some districts this difference is extreme. I have seen districts with 30% special ed rates in middle schools for african americans, but near or below 10% for whites and asians. Unfortunately, our state does not seem to care enough to require that that data is actually made public. Instead, we’ll pretend that teachers are really the cause of all this and that we can ‘fix’ it by removing tenure, seniority and experience, and giving administrators who currently refuse to do their jobs even more power to capriciously wield that power. I say capriciously because for most of them, API is the holy grail, just as low performance metrics must equate to horrible teacher.
I am glad a judge is deciding this since they generally actually have to put some though process into the exercise, and even explain their decision, unlike any of our other ‘policy makers’.
Report this comment for abusive language, hate speech and profanity
A quote from a recent Diane Ravitch blog post:
“If the public comes to understand how flawed the tests are, how marred by random error, statistical error, measurement error, and human error, they would lose their luster. They are not scientific instruments. They are social constructions. And these imperfect instruments are being misused and abused to advance a wrong-headed political agenda. ”
People should recall that Ravitch was there for the “big bang” of the present test based accountability fiasco. She finally figured out that the whole NCLB/RTTT style reform was an absolute failure. That style has now been in place for over a decade. It is the “status quo.”
Perhaps Deasy should have taken a few more classes for his Phd. and he would then have discovered that there is no existing system (AGT or VAM) of using student test data to link to teacher evaluation that is not wildly unreliable and invalid. Just ask the National Research Council.
Not that we don’t need accountability though, we do. An accountability system that takes all the players into consideration: teachers, school administrators, district administrators, parents, community leaders, state and federal officials, et al. Just think of the conclusions we might draw if all of the people who really contribute to student achievement were held accountable in a true 360 degree evaluation system.
Report this comment for abusive language, hate speech and profanity