Why I Feel We Are Being Sold Out

It is becoming common practice nationally to rank teachers for their effectiveness or value added, a measure that is defined as how much a teacher contributes to student progress on standardized tests. The latest fad formula for using student tests scores to rate teachers is called Value Added Methods or VAM. It’s intended to measure the growth of students in classrooms and to attribute that growth, or lack of, to the teacher.

Unfortunately, research is abundant which indicates the problems associated with this practice. Every respected independent testing expert in the country agrees that VAM is not a valid or reliable measure for making high stakes decisions about teacher effectiveness.

· A panel of America’s top researchers recently found that those scores vary wildly from year to year—hardly a reliable measure of teacher quality, and neither a fair nor accurate way to judge a teacher’s work (www.nea.org/value-added).

· In October, 2009 the Board of Testing Assessment wrote a letter to ED Secretary Duncan stating: “Too little research has been done on VAM’s validity to base high stakes decisions about teachers on them. A student’s scores may be affected by many factors other that a teacher.”

· In July, 2010 ED’s own Institute of Educational Sciences concluded in a 36 pg. analysis of VAM research data that “more than 90% of the variation in student gain scores is due to the variation in student – level factors that are not under the control of the teacher.

· In August, 2010 a panel of experts assembled by the Economic Policy Institute (EPI) warned in a report that it would be “unwise” to give substantial weight to VAM scores in measuring teacher effectiveness. The EPI concluded “…there are good reasons to be concerned about the claims that measuring teacher effectiveness largely by student test scores will lead to improved student achievement. There is also little or no evidence for the claim that teachers will be more motivated to improve student learning if teachers are evaluated or monetarily rewarded for student test score gains.”

· Researchers from Princeton identified sever hazards of merit pay systems: teachers may emphasize tested subjects over those that are not tested; may teach to the test; or may be unwilling to cooperate with colleagues with whom they are in competition.

· In a new study forthcoming in the Quarterly Journal of Economics, “Teacher Quality in Education Production: Tracking, Decay, and Student Achievement”, Jesse Rothstein, an Assistant Professor of Economics and Public Affairs at Princeton’s Woodrow Wilson School of Public and International Affairs suggests we do not know how to adequately measure a teacher’ value added contribution. He states “If accountability and merit pay policies are to produce improvements in teacher quality, it is essential to ensure that teachers who get the “right “ students who test well do not get unfair advantages, and that teachers who get the “wrong “ students do not get unfair disadvantages.

· Douglas N. Harris, an educational professor at the University of Wisconsin, Madison found that two analysts might rank teachers differently if one analyst took into account certain student characteristics, like which students were eligible for free lunch, and the other did not. He found that millions of students change classes or schools each year, so teachers can be evaluated on the performance of students they have taught only briefly, after students’ records were linked to them in the fall.

· In Houston, some teachers have reported being the victim of a problem known as the “ceiling effect.” Advanced students can score so highly one year that standardized state tests are not sensitive enough to measure their learning gains a year later. This has lead to smaller bonuses as compared to other teachers.

· Diane Ravitch is a popular and well respected educational researcher who wrote on her blog, The Answer Sheet, that she was in Los Angeles and spoke to teachers who were shamed by the Los Angeles Times disgraceful release of test score data and ratings of 6000 elementary teachers as more or less effective. One of the teachers killed himself. Last October, the NYC Department of Education’s teachers union filed an injunction to keep the district from releasing 12,000 value added ratings to the press. Ravitch concluded that VAM should not be used at all. Never. It has a wide margin of error; it is unstable. A teacher who is highly effective one year may get a different rating the next year depending on which students are assigned to his or her class. Ratings may differ if the tests differ. Teachers will be mislabeled and stigmatized. Many factors that influence student scores will not be counted at all.

· The latest review of VAM was written by New York University economist Sean Corcoran. He examined VAM in Houston and NYC. He describes a margin of error so large that a teacher at the 43rd percentile (average) might actually be at the 15th percentile (below average) or the 71st percentile (above average).He asks “What is the value of such a measure? Why should it be used at all?” He finds that VAM can be useful to schools, but not as part of a teacher’s formal evaluation, in part because of the yearly fluctuation. “At best, they can tell you who the consistently high and low performers are”, Corcoran says. “But for the vast majority of teachers, you learn almost nothing from them.” Corcoran found that certain student characteristics also influence teachers’ ratings. For example, teachers of gifted students tend to have lower ratings because high achieving kids test high on standardized tests and, thus, make smaller gains.

· Daniel McCaffrey of RAND Corp. and author of several studies on VAM advises caution. The ratings, which use a statistical method to judge teachers’ contributions to their students’, test scores. Can’t compensate for an otherwise weak teacher evaluation system, he says. Individual VAM ratings, while informative, can be skewed by random events, such as student illness or a neighborhood crisis, which are outside a teacher’s control.

· Linda Darling Hammond, education professor at Stanford University rejects VAM scores as too variable from year to year to provide useful feedback. In her October Report for the Center for American Progress, she advocates for a national evaluation system that would rate teachers on practices shown by research to improve student learning, such as collaborating to improve instruction.