LA groups want test scores part of evaluations
Between a quarter and a third of evaluation score
Two Los Angeles education groups have offered separate teacher evaluation frameworks that they hope will help break the impasse between Los Angeles Unified and its teachers union, United Teachers Los Angeles.
“There is frustration that, even after years of discussion, there still is no new system in Los Angeles,” Mike Stryer, a former Los Angeles Unified teacher who helped create the plan for Our Schools, Our Voice Coalition, said at a news briefing Thursday.
Our Schools, Our Voice Coalition wants teacher observations to comprise 60 percent of a teacher's evaluation score, followed by student test scores at 25 percent. Source: Our Schools, Our Voice. (Click to enlarge)
The biggest barrier – at this point seemingly uncrossable – is disagreement over the inclusion of student standardized test scores in the evaluation. The district uses a method, Academic Growth over Time, that measures a teacher’s impact on student test results. Superintendent John Deasy wants to include the AGT score in the evaluation, although he has not said how much weight it and other factors would have. UTLA remains adamantly opposed, and devoted considerable space in a 53-page evaluation proposal released in March to argue why, as unsuitable and inaccurate measures, “standardized test scores should play no part in high stakes decisions leading to dismissal.”
Both Our Schools, Our Voice Coalition – with parents, education advocates, and some Los Angeles teachers – and Teach Plus, a national network of teachers with a chapter in Los Angeles, support phasing in AGT, but with conditions. Among requirements under the Our Schools, Our Voice Coalition plan, AGT wouldn’t count unless a course’s curriculum matched the standardized tests and there was a statistically significant sample size. AGT wouldn’t count for probationary teachers. And all test results would remain confidential, inaccessible to the public and the press (no more providing data for publishing in the Los Angeles Times). Use of test scores would be phased in, counting 10 percent the first year, reaching a maximum 25 percent after three years. Teach Plus also advocates starting at 10 percent, working up to a third of a teacher’s evaluation, if benchmarks for test integrity and reliability are met, said John Lee, executive director of Teach Plus Los Angeles.
What the union, the district, and the two outside groups all agree on is that classroom observations should constitute the biggest piece of an evaluation: 60 percent under Our Schools, Our Voice’s plan and at least half, Deasy has indicated, under the district’s. The district is currently training principals in uniform observation rubrics and piloting observations in 100 schools involving 700 teachers. Teach Plus wants teachers to help evaluate their peers in areas requiring content expertise but in a capacity of providing classroom guidance, separate from a formal evaluation with consequences. UTLA favors an expanded use of Peer Assistance and Review, a panel of teachers who counsel teachers needing improvement and recommend dismissal for those who “have been given a real chance to improve but are unable to meet clearly defined standards.” Under the Our Schools, Our Voice recommendations, a mentor will be assigned to a teacher identified as needing intensive support for at least a full year.
Like the district’s eventual plan, Our Schools, Our Voice proposes student surveys (beginning in the third grade), parent surveys, and a measure of contributions to the community – each counting 5 percent. And Our Schools, Our Voice includes a new, intriguing element: a way to identify and reward, with up to a bonus 10 percent score, those teachers who help close the achievement gap for Hispanic students, African American students, and English learners in the bottom quarter who make marked progress.
The release of both organizations’ recommendations is intended to prod UTLA and the district to start talking. But at this point, leverage is more likely to come from the courts or the Legislature.
On Tuesday, in Los Angeles County Superior Court, there will be arguments in a suit brought by the nonprofit EdVoice on behalf of Los Angeles Unified and UTLA over the failure to include standardized tests in evaluations. EdVoice makes a good case that the Stull Act, the 40-year-old state law on teacher evaluations, requires test-score use, but districts like Los Angeles Unified have ignored the provision. A victory by EdVoice – and indirectly for Los Angeles Unified, though named as a defendant – might force UTLA to back off its unqualified opposition to the use of test scores.
Until now, Los Angeles Unified has argued that it has the exclusive right to determine the requirements for an evaluation. It exercised that right in setting up the pilot evaluations, despite the opposition of UTLA. But later this summer, the Senate will likely take up AB 5, sponsored by Democratic San Fernando Valley Assemblymember Felipe Fuentes, which would replace the Stull Act. As currently written, most aspects of an evaluation process would have to be negotiated with unions, which could stretch out adoption of a new system for months, if not years.
24 Comments
Trackbacks
- The Educated Guess: LA groups want test scores part of evaluations : SCOE News Reader
- Ed News- June 1, 2012 « tigersteach
Given that big purple slice it seems that principal evaluations can’t be separated from teach evaluations. Will principals be evaluated on school test scores. If so, how much?
Report this comment for abusive language, hate speech and profanity
Does principal observation/evaluation include some kind of growth portfolio work from the teachers’ classrooms showing specific examples of student improvement? I can see a valuable component where integrated project work, mastery of concepts and knowledge exemplified through student presentations, etc., could provide a snapshot of the classroom in a workable and non-intrusive manner. A benefit would be increased collaboration between teachers and more agreement as to what constitutes student mastery.
I loved the collaborative portfolio assessment work we did in the UK, not least because it was professionally beneficial to collaborate with teachers from other schools to really work on fundamental issues concerning teaching and learning. Using micro teaching (so much easier now than in the 80s) the teacher can also receive feedback, as well as personally critique classroom performance, and use the materials as a personal growth tool – again over time.
Report this comment for abusive language, hate speech and profanity
I haven’t done this legwork myself, but I’ll just raise this question:
Who are the funders of Teach Plus and Our School, Our Voice? Is their funding partly conditioned upon these organizations’ calling for including student test scores in teacher evaluations?
I find the coverage from all sides to overlook that enormous elephant in the room in discussions of this or that organization and its advocacy goals.
Report this comment for abusive language, hate speech and profanity
Caroline: From the website of Teach Plus: Funders include: The Bill and Melinda Gates Foundation, The Mind Trust, The Joyce Foundation, EdVestors, The Wasserman Foundation, Barr Foundation, Richard and Susan Smith Family Foundation, and The Boston Foundation.
Gates is pushy on this issue, as you know.
Report this comment for abusive language, hate speech and profanity
But what is the output of classroom observation? If, as many have complained about in the past, that is essentially a binary metric, wont that in essence make test scores the single differentiator?
And making the data confidential as a way to maintain integrity seems like an oxymoron.
Report this comment for abusive language, hate speech and profanity
Congratulations to these groups for working hard to find common ground amidst the posturing. They are trying to address the chief concerns of both sides of the test score issue: balancing the need to have an objective measure of teaching effectiveness with the equally important need to have that objective measure be reliable and statistically significant. It’s not an easy task but offering solutions is better than standing in one’s corner doing nothing but questioning the motives of the other side.
Compromise by people of good will is essential if we are going to design effective teaching systems that actually measure effectiveness as a way to improve teaching practice.
Report this comment for abusive language, hate speech and profanity
It’s a little antithetical to include statewide test scores as measures of teacher performance and strive for college and career readiness at the same time. Maybe the impending Common Core assessments will be better, but I don’t hold out too much hope. Most of the LAUSD high schools with high API scores have abysmally low EAP passage rates. Most of the higher performing charters (high API scores) have low SAT scores and EAP passage rates. The divide between what is measured (basic skills) and what is valued (higher order thinking skills) will become exacerbated (and it already is pretty large) if teacher performance and eventually compensation is linked at all to measurements of basic skills such as STAR and CAHSEE.
Report this comment for abusive language, hate speech and profanity
How will kindergarten, 1st grade, PE, and art teachers be evaluated under this scheme?
Report this comment for abusive language, hate speech and profanity
I would like to better understand what is meant by the % components and how one arrives at a final ranking, and what that ranking means for the teacher and the school.
How are the (laughably small) 5% contributions from student surveys, parent surveys, and community contributions calculated?
Let’s say the parents and the students rate the teacher above average (whatever that means) and the principal rates the teacher above average (whatever that means). If the test score metric (however it is calculated) puts that teacher in the bottom quartile, is that teacher in danger of being swept up in some automatic action?
If the teacher gets abominable results on the student and parent surveys, but meets the principal evaluation rubric and has average or better test scores, is that teacher protected from dismissal? Is it possible a rational principal would have an appropriate reason for finding a teacher detrimental to the program in a way that doesn’t come up in the rubric? IE, is it possible you could have a teacher that everyone agrees is not good for the kids but who is getting evaluation results that would impede dismissal actions?
I think the exercise of including all these elements makes perfect sense. Certainly if I were evaluating a teacher, I’d want to consider test score data and community input, and investigate carefully both high and low results. My concern is that when we try to attach a number to all these elements and combine them with an algorithm so that a computer or someone in Sacramento who has never met the teacher is making the final assignment of Goodness Level, that’s when it stops working. (And that’s not a model you see in successful private companies, either.)
Report this comment for abusive language, hate speech and profanity
I’ve posted many times the dozens of reasons that test scores do not belong in teacher evaluation – at all. But I would just like to add one new one: we don’t even know what the tests are going to look like by the time these policies can be adopted and phased in. Isn’t it silly to suggest using tests we haven’t yet developed, piloted, or administered to even one class of students?
Report this comment for abusive language, hate speech and profanity
Ooops – in a bit of a rush. Just to clarify, I’m referring to the new Common Core assessments that will be coming out in a couple years. Moving ahead with any policy that depends on the current and soon-to-be-discarded CSTs would be a waste of time.
Report this comment for abusive language, hate speech and profanity
As David has asserted, and that has been repeatedly stated by the National Research Council (NRC) and other researchers, there is no science to support using student test scores for teachers’ evaluations. Systems, VAM, AGT, etc. to evaluate teachers based on student scores are wildly unreliable and invalid for that purpose.
Testing experts agree that tests designed to assess one thing cannot (without considerable research) be used to assess something else. It’s like using your bathroom scale to take your temperature.
The NRC states not only don’t we have the research to support using the test scores for evaluation, but if we did we still shouldn’t do it because of the narrowing effect it has on curriculum. It has the potential to hurt learning. Get it?
And being “objective” is no reason to support it. Being objectively bad is not a good thing.
There are systems used by many university teacher preparation programs that do have scientifically supported predictive value for assessing teacher performance. In CA these are the PACT and TPA. These are performance assessments using multiple measure and multiple, highly trained and “calibrated,” evaluators. Of course, this costs. One of the real drivers of the whole “test score based evaluation” thing is that it is cheap, though that is rarely mentioned.
The cheap, unreliable, and invalid evaluation systems are supported by conservatives, neo-liberals, sensationalistic newspapers, and those seeking the classic simple solution to the complicated problem that is, invariably, wrong.
Report this comment for abusive language, hate speech and profanity
Would be even cheaper to roll dice for that part of the evaluation. Conveniently and quantitatively ranks the teacher 1-6.
People assume that the test score number will be “close enough” to some sort of “truth” but in fact you see that a teacher rated “highly effective” one year could frequently be rated “ineffective” a year or two later using some of the models in play.
Report this comment for abusive language, hate speech and profanity
Granted, teacher evaluation is a mess, and historically that’s management’s fault, not the union’s. (Management had a half-century to develop robust teacher evaluations, but instead it made them trivial and particularistic.)
There is almost universal agreement that teacher evaluation should include some measure of how well students are doing. The “I taught them, but they didn’t learn” excuse impresses no one, including progressive unionists.
The rub comes because LAUSD has signed onto value added assessment using standardized test scores as the measure of achievement, although it appears to be coming to the realization that value added measures make more sense as indicators of school performance than they do of individual teacher performance. The data availability and reliability problems with individual teacher rankings are severe and permanent.
The community groups suggest a compromise between the United Teachers Los Angeles opposition to any use of student test scores and the district’s idea by suggesting a gradual phase in and a lower weighting.
This is half a solution. A better solution may lie in the UTLA plan itself. The union has come a long way in its discussions, and a draft document, which is circulating around the Internet, advocates a well-developed peer review plan. Peer review offers the possibility of evaluations that consider student achievement on a much broader range of both teacher activities—not just teaching a single lesson—and for including a variety of student outcomes.
For example, peer reviewers could examine portfolios of student work, changes in student behavior and engagement over the year. Examine what the student actually produced as opposed to how well they did on the state’s test. These data won’t allow ranking of school gains, but they allow a trained observer to see whether all, most, or few of a teacher’s students are making good gains.
Still, there is the matter of test scores. If students are expected to take the tests, teachers ought to be preparing them to do well on them. If the tests are lousy, and I believe that many are, unions ought to be in the corner of creating better ones, not using deficiencies in the current system as a reason to stonewall accountability. So, given the current ability to measure student progress on standardized tests, it would make better sense to use value added measures at the school level, and use examinations of student work itself for the evaluation of teachers.
Report this comment for abusive language, hate speech and profanity
Gates Foundation and their ilk are like the man with the hammer- everything looks like a nail. They are far beyond just being “pushy” on this and other “solutions” to pre-defined “problems.”
I’ve seen the increasing de-skilling and de-professionalization of teachers and teaching, and the Taylorization of teaching and learning, and from my perspective, standardized testing is the epitome of that agenda. I disagree with all of it! and I am deeply concerned for the future of our country and our society and culture- when we have a generation of adults who can only do what they’re told, who have the “will it be on the test?” mentality.
And, who will want to be a teacher? The pay is bad, benefits and pensions are sure to be cut even further, you get little to no support from administrators, you pay a boatload for your own education to become qualified, you get no respect, you are blamed for every failed student, you have to deal with demanding, ignorant parents and community leaders, you suffer under egotistical boards of trustees, you become a pawn in the political struggles between unions, districts, voters, extremist ”reformers,” and right wingers who really just want to bust the union, any union, etc. Now add on top of all that a meticulous, bean-counting, super-rigorous performance evaluation that goes far beyond what any other job currently requires. Seriously, who wants that job?
I recently submitted evaluations for administrators at my school- via SurveyMonkey. The measures were broad and vague categories like Communication Skills, Organization and Planning, Professional and Community Service, and the scale of choices was Excellent, Good, Neutral/Not Applicable, Needs Improvement, Unsatisfactory. I’m asking, “What’s wrong with this picture?”
Lastly, I see no role for community advocacy groups, regardless of funding, to enter into labor negotiations. It might even be illegal, depending on the extent of their involvement. Negotiations are between the parties authorized.
Report this comment for abusive language, hate speech and profanity
I object to denigrating impassioned advocacy by volunteers as “posturing,” if that’s the implication. (It’s fine with me if you say it about the work of hired spokespeople.)
Report this comment for abusive language, hate speech and profanity
If this group is not just another Eli Broad, Bill Gates Scam designed to undermine public schools and teachers, I give them great respect for being proactive, industrious and socially responsible. Teachers have to be evaluated so they can improve and schools will improve as well. Unfortunately, what this organization has to work with is extremely skewed and ineffecient.
The truth is that most administrators and officials at LAUSD are failed teachers, who could not wait to flee from classrooms. Teaching is demanding work and it requires gifts not everyone has. Ironically, the only administrators I know of that are capable leaders were teachers for at least a decade before being promoted. In my mind one of the districts greatest flaws is that so many overpaid unnecessary suits know so little about pedagogy. Of course these parents can’t know this or that behind the smiling face in the respectable suit lurks a few too many sociopaths and other personality disorders drawn to education because they are bullies and understand this system, which has little oversight. So what we have now are administrators torpedoing older teachers with bad evaluations. They are following a directive from thier boss, Mt, Deasy who is agressively unloading expensive veterans any way he can to bring up the profits for his bosses.
Who are his bosses? The two men mentioned in the first paragraph. Vulture philanthropists are eager to keep labor costs down as we know thanks to outsourcing and wage slaves being exploited by employers like Walmart. Walton, who has also handed LAUSD a big check or two recently, may be looking for his cut too.
These men installed principals who often are not capable of evaluating teachers because they have two years experience in classrooms, no knowledge of instruction or the standards and they operate like vindictive despots. A teacher who defies a dictate by standing up in a staff meeting about a matter he or she disagrees with will be punished. Say she is angry that grant money is used to purchase iPads for administrators instead of books for students or feild trips. She will find her self on the wrong end of an evaluation she cant possibly get a decent shake on.
At my school 3 teachers on a committee had to vote to keep 1 test coordinator( the principal’s top crony) or 2 teachers: the only college counselour and a fabulous librarian. They were warned not to vote against the principal’s wishes as parents, union reps and students looked on in horror.
These teachers voted against the principal because there is a way we can get the test coordinator jb done without him, in fact, the district had just announced tbe position was no longer funded. The test coordinator who cost as much as two vital colleagues, sat around nmuch of ythe time shooting the breeze,something none of us could ever do. These teachers understood howvtgis grant, which Gates gave the school gor the success it had prior to this principak’s arrival. They did the right thing. This is best for students. Nevertheless, the pincipal threw a tantrum and vowed to ruin these teachers, some of the finest I know.
The next morning the local superintendent showed up. He went to each of these teachers’ classrooms. Every one was given a very bad evaluation. While school reps defended these teachers and demanded the union protect them, nothing was done.
The union has lost interest in menbers. Three teachers from that school, all popular, active, dedicated members of the school community were sent to the rubber room on phony charges because they spoke up for students and other teachers. Dozens have been driven into early retirement. It had been a successful school with progressive scores and innovative configurations before this man took over. He was not even properly credentialed at the time but because he had completed the courses at the Broad Academy he was given free reign. His abuse of power is so unconsionable even students refuse to take part in witch hunts that involve their least favorite teachers.
They do know right from wrong. This situation, btw, is not an anomoly. It is now the norm. The rubber rooms are full of innocent teachers, but none of them are in there for being ineffective or having low test scores. They are in there because they angered the principal, blew the whistle, haplessly witnessed some thing they shouldnt have or had a civil case against LAUSD. Yes, some are indeed guilty of inappropriate contact with students, unprofessional behavior or other transgressions—they are easy to distinguish becuause they are treated better than the innocent teachers who the suits watch closely, hoping to get a real excuse to fire them.
Due process is a pretense. So are evaluations unless the evaluator is objective. I suggest this plan be used but with a few adjustments. Instead of principals , have retiresd administrators and teachers from other areas meet with parents to evauate teachers and discuss their strangths and weaknesses with that teacher to help him or her improve. There are few teachers who will disagree with the fact that there is always room for improvement. Those who do,probably should not be teaching.
As for the tests, they too need significant revision before they can be use as an accurate tool evaluating teachers. I urge this group to deconstruct the value added model and to ask themselves why a teacher in Watts must compete with a teacher in San Marino. It is one thing to insist that students have a level playing feild but quite another to punish their teachers because they chose the challenges of working with students who don’t speak Rnglish well and struggle with issues like poverty, broken families, gangs and addiction where they come from.
When the teachers are evaluated by objective individuals who fully comprehend the nature of their work and there is a test in place that is not culturally biased, poorly written and erroneous that can be scored in a way that measures each students growth ( a formative asessment makes sense and guides teachers’leasson plans each year), this plan will be difficult to contest What this group is asking for is perfectly reasobnable, but in order for it to be fair, the members have to scrutinize district s and schools closely. They are speaking to administrators, school officials, & a select group of teachers. But it is necessary to survey , interview and meet with other teachers, particularly those who are currently recieving poor evaluations. The union is ecen less reliable than the district when it comes to these typs of issues, but it too deserves to be heard. My advice is to remember all of these groups have an agenda. Teachers are probably the ones most interested in what is best for students. I concede many of them have no clue what that is. However, by gathering many points of views and reading the copious studies on these tests and school leadership, this collection of human beings can and should make the difference our schools truly need to serve the needs of our children and community.
Suggessted Reading / Experts
Diane Ravitch
Lois Weiner
Karen Horwitz
Battling Coruption in America’s Public Schools
Breaking the Silence (Blase and Blase)
When Teachers Talk
Betrayed
Websites of Interest
The Broad Report
Perdaily.com
Common Dreams
NYC Rubber Room Reporter
Bartleby Project
Parents Across America
Save Our Schools
Modern Schools
Schools Matter
The Patch
The Frustrated Teacher
Learning is Messy
Big Education Ape
Thank you for reading this and good luck.
Report this comment for abusive language, hate speech and profanity
And here’s another thing to consider-
Years ago when standardized testing was touted as the “solution” to, I guess, poor test scores!! opponents argued that teaching would be lowered to “teaching to the test” and proponents claimed that would not happen. Well guess what: teachers more and more are teaching to the test. Teachers now have less control over what they teach, how they will teach it, when and for how long they will teach it, and are more restricted by mandates, rubrics, guidelines, acronyms-du-jour, standardized curriculum, standardized texts, standardized tests. Any creative or innovative teaching method is out. Any adaptation to the needs of specific individual students in the class is out, unless it’s for ADA accommodation. So, how unfair is it to evaluate an employee on something over which they have little or no control? Teachers are commanded to make bricks without straw.
If your job depends on making a target score, average for the class, and you can’t change your teaching methods, can’t adapt the content, can’t spend more time with struggling learners and less with confident ones, what are you going to do?
Report this comment for abusive language, hate speech and profanity
Caroline, this what I have found out so far: Teach Plus Inc is not registered with the State of California as a charity corporation. Nor is it listed in the Charity web page of the Attorney General of California. Therefore, Teach Plus Los Angeles is operating illegally if I understand California laws correctly (someone please correct me).
It is, however, incorporated in Boston, MA. Its 990s for 2009 and 2010 (available at guidestar.org) have the following for Part VI, Section A, Line 7B; “PRIOR TO APPROVING THE BUDGET OF TEACH PLUS INC., THE BOARD MUST RECEIVE BUDGETARY AUTHORIZATION FROM THE BILL AND MELINDA GATES FOUNDATION FOR ANY DISBURSEMENTS PLANNED USING FUNDS FROM THE GATES GRANT.”
From these same 990s, one finds that its president, identified in numerous press releases as being “an education-reform advocate specializing in charter schools” was paid $122.5k in 2010 and $99k in 2009, all from a combined budget of $2.3M for 2009-10. The 990s do not identify how much each of the funders listed by John contributed to this budget. The 990s for 2011 will be available sometime in October (it takes time for the IRS to post them).
Grass-roots or AstroTurf? You be the judge.
Report this comment for abusive language, hate speech and profanity
Thanks, Manuel. Activists around the country (actual grassroots activists) have been asking about that.
Report this comment for abusive language, hate speech and profanity
Also, can anyone describe how Teach Plus is “a national network of teachers”? Is it a membership organization?
Report this comment for abusive language, hate speech and profanity
As John indicated above Teach Plus is primarily a function of , and funded by, the Gates Foundation. They seem to work closely with Ed Trust, which should not be a surprise.
I am more than somewhat taken aback by Professor Kirchner’s comment: “If the tests are lousy, and I believe that many are, unions ought to be in the corner of creating better ones, not using deficiencies in the current system as a reason to stonewall accountability.”
1) The new assessments have in large part been developed by testing people, non-profits, and university academics. The unions and teachers have been something of an afterthought. Both of CA’s teacher unions (CFT & CTA) are participating, as requested, in item development/review.
2) Both organizations (CFT & CTA) are participating in the Educator Excellence Task Force process that will result in recommendations for a rigorous teacher evaluation (accountability) system.
Exactly how 1 & 2 above can be construed as “stonewalling” is beyond me.
Report this comment for abusive language, hate speech and profanity