Teacher turnover under IMPACT improved student performance, study says

Teacher turnover is often rightly perceived as a problem for schools. A growing body of evidence finds that teacher turnover reduces student achievement, either directly because replacement teachers are less effective than exiting teachers or indirectly through the disruptions caused by high turnover rates. This is especially true in schools whose students come from high-poverty households, where teacher turnover rates are especially high and where it is often very difficult to recruit effective new teachers who are as effective as those who left. 

However, results from a new study shows that teacher turnover under IMPACT, the teacher-evaluation system used in the District of Columbia Public Schools (DCPS), improved student performance on average. The study also shows exiting teachers who were identified by IMPACT as low-performing, resulted in substantial improvement in overall teacher quality and student achievement.

The study by researchers from Stanford Graduate School of Education and the University of Virginia Curry School of Education examines the effects of teacher turnover at DCPS from 2009-10 through 2011-12 and is available here.

IMPACT is a multi-measure comprehensive teacher assessment system that evaluates the performance of all teachers at DCPS. Teachers whose ratings are Ineffective are dismissed; those whose performance is rated as Minimally Effective have one year to improve; and teachers who are rated Highly Effective receive large bonuses and the potential for substantial increases in base pay. This system is intended to encourage high-performing teachers to remain at DCPS and to induce low-performing teachers to leave. 

In addition to the longstanding belief that replacing exiting teachers is difficult, additional concerns have been raised in DCPS that high-performing teachers may leave because of the stress of high-stakes evaluation.

“While these are reasonable concerns and in some situations this may occur, overall our analysis suggests they don’t hold true at DCPS” said James Wyckoff, professor at the University of Virginia and one of the study’s authors. “This likely reflects IMPACT’s design to retain more effective teachers and encourage low-performing teachers to leave.” Wyckoff added. 

Findings were especially positive when low-performing teachers turned over.

Teacher quality and student achievement in both math and reading increased substantially when low-performing teachers, those dismissed by IMPACT or who voluntarily left DCPS following their first ‘minimally effective’ rating, exited DCPS.

“We found that a disproportionate share of low-performing teacher exits are from high-poverty schools,” explained Thomas S. Dee, professor of education at Stanford and an author of the study. “Our results indicate that DCPS is able to accurately identify low-performing teachers and consistently replace them with teachers who are more effective in raising student achievement, particularly in high-poverty schools.”

On average the exit of a low-performing teachers resulted in an increase in student achievement equivalent to four months of learning in math and reading.

The findings were surprising even when focusing on the high-performing teachers who exited DCPS.

“In other school districts, we usually observe negative effects when high-performing teachers leave,” said Dee. “That doesn’t appear to be the case in DCPS. The turnover of high-performing teachers is a challenging problem but, in DCPS, we find that the exit of high performers generally has small and statistically insignificant effects on student achievement.”

These findings provide the first empirical evidence of the effects of a high-stakes teacher assessment system on the composition of teacher effectiveness and student achievement. As in all urban school districts, there is substantial turnover of teachers in DCPS and for a variety of reasons.

“We have no way of differentiating their motivations for leaving,” explained Wyckoff. “We suspect a good many leave irrespective of IMPACT for many of the same reasons that turnover is high in many urban schools. However, there is no doubt that when teachers identified as low-performers exit, both teacher quality and student achievement substantially improve.”

As encouraging as these gains may be, student achievement in several high-poverty schools in DCPS remains low. In 2012-13 IMPACT was redesigned so that more teachers were designated as low-performing and to target more narrowly the financial incentives for high-performing teachers on those teaching in high-poverty schools.

“Whether these recent changes provide additional gains to students in these schools remains to be seen,” added Dee.

It is uncertain whether other school districts can replicate these effects because it depends on factors which many have little control over, (e.g., the local supply of high-performing teachers) as well as those they do control (e.g., the introduction of meaningful teacher evaluation).

These findings build on prior work by the research team that found that IMPACT’s incentives improved teacher quality. 

Dee and Wycoff also were honored with the Vernon Memorial Award for their research on IMPACT.

___

Leslie Booren is the managing director of EdPolicyWorks at the Curry School of Education at the University of Virginia.