Education CommitteeWritten evidence submitted by Rebecca Allen (ioe) and Simon Burgess (CMPO)

Summary

We think of Initial teaching training (ITT) as encompassing both the initial training and the probationary year. We assume that the point of ITT is to produce effective teachers who will have the greatest possible impact on pupil progress. The two central facts are that variations in teacher effects on pupil progress are very substantial, and that the future effectiveness of a potential teacher is hard to judge from their own academic record.

As with any professional accreditation process, ITT has a training aspect and a selection aspect. Selection has been under-emphasised relative to training, and we argue that this needs to be rebalanced. We believe that the evidence suggests that the current operation of selection in ITT (tight at the beginning, negligible thereafter) is the wrong way round. Instead we should let a broader group try out to be teachers, but enforce a much stricter probation policy based around measures of teacher effectiveness in facilitating pupil progress. Full certification and an on-going first job would only be granted once performance data showed a teacher to be effective. The expectation would be that only the most effective teachers would make it through to full certification.

Introduction

1. We assume that the point of ITT is to produce effective teachers who will have the greatest possible impact on pupil progress. We think of Initial teaching training (ITT) as encompassing both the initial training and the probationary year.

2. An analysis of ITT has to be based on an understanding of the teacher labour market as a whole. Combining research in England with relevant work from elsewhere, we are starting to build up a picture of the key facts about how the teacher labour market operates.

3. Over the last decade in England, there has been a major increase in data on pupils available to researchers, leading to a better understanding of educational attainment. However, there is much more to learn about teacher careers, and new data now becoming available holds out the promise of making some real progress.

Teacher Performance

4. We start from a brief summary of what is known about teacher performance.

(a)Teacher effectiveness matters enormously. A pupil being taught for eight GCSEs by all effective teachers (those at the 75th percentile of the teacher effectiveness distribution) will achieve an overall GCSE score four grades higher than the same pupil being taught for eight GCSEs by all ineffective teachers (at the 25th percentile). A range of studies have consistently shown a very high impact of teacher effectiveness on pupil progress. While there are also papers contesting the validity of the assumptions required to identify true effectiveness, there is other research arguing that the results are secure.[1]

(b)Measures of teacher effectiveness are noisy. Numerous factors affect exam scores, from good or bad luck on exam day, through the pupil’s ability, motivation and background to a school’s resources. Research shows that it is possible to measure a teacher’s contribution to this, but it is an estimate with less-than-perfect precision. There is simple sampling variation, plus non-persistent variation arising from various classroom factors. For example, a teacher’s score is any one year may be affected by being assigned a particularly difficult (or motivated) class (in a way not accounted for in the analysis).[2]

(c)Experience doesn’t help beyond three years. Research shows that on average teachers do become more effective in their first two or three years. Thereafter, there is no evidence of systematic gains as their experience increases: a teacher is as effective after three years as s/he will be after 13 years and 30 years.[3]

(d)Good teachers are hard to spot ex ante. One of the more surprising findings to come out of the research on teacher effectiveness over the last decade has been that the characteristics that one might have thought would be associated with better teachers simply aren’t. Experience, a Masters degree, and a good academic record in general are not correlated with greater effectiveness in the classroom. These results have been found in both the US and England. We need to be careful what we are claiming here. The research shows that easily observable, objective characteristics such as those noted above, variables typically available to researchers, are no use in predicting teacher effectiveness. This is not to say that no-one can identify an effective teacher, nor that more detailed subjective data (for example, from watching a lesson) can be useful. No doubt many Headteachers are adept at spotting teaching talent. But there are enough who aren’t to mean that there are ineffective teachers working in classrooms.[4]

(e)Very few teachers are dismissed from the profession in England.

Two Roles Played by ITT: Training and Selection

5. ITT plays two roles for the profession—training and selection. The emphasis has typically been on the former. Both are important and neither should be neglected, but we argue that the facts set out above suggest that if anything selection is the more important.

6. This is not unusual. For all professions, certification and induction involves a process of selection, both by the profession itself winnowing out the less able, and by the trainee deciding that it is not for them.

Selection Aspect of ITT

7. There are three key points of selection: into ITT, graduating out of ITT and final accreditation as a qualified teacher. The first and last of these are the most important. We need to consider the whole process of training teachers as a whole, thinking of the PGCE and the probationary period as both being critical parts of ITT.

8. Selection into ITT is about gaining a place on a course. The facts set out above on the difficulty in identifying people likely to be good teachers are very relevant here. They suggest a high degree of agnosticism would be appropriate when faced with applicants: it is very hard to tell who will be a good teacher. This is certainly true for selection based on objective criteria from the applicants’ academic records. We know that these are unrelated to teaching ability, and so should be irrelevant in selection into ITT. Beyond that, even if selectors are highly skilled at spotting potential, and it is not clear that they are, it is impractical to ask each applicant to teach a practice lesson.

9. Therefore, we argue that selection into ITT should be very broad, with a relatively low academic entry requirement. This of course is not the situation now, nor the direction of travel of current policy. We argue that the tightening of academic entry requirements into teaching is not helpful: it will restrict the quantity of recruits and have no impact at all on average teaching effectiveness.

10. Graduation from ITT should be tough. Given that much of an ITT course is now school-based, time spent in the classroom will form an important part of the assessment. Arguably the classroom experience is the key part of the course. However, in such a short space of time it will not generate sufficient data for a robust and objective view of the trainee’s effectiveness. It will nevertheless allow the trainee to discover whether teaching is for them.

11. One innovative route into teaching is through Teach First. In some ways this is a positive development, as it allows a lot of people to try out teaching and also gives the schools which employ them an “out” at the end of the two years. On the other hand, by restricting entrants on academic background.

12. Once in a job in a school, the progression to being a qualified teacher should be very different to the typical experience now. The key decision on final certification should be made after a probation period of say three years. The period probably cannot be less, though the appropriate length of the probation would need to be analysed properly, depending on the statistical reliability of any pre-hire indicators, school-based performance data, and the cost of being wrong (Staiger and Rockoff (2010) discuss this issue in depth). This is the point when enough data is available to make a reliable judgement on the effectiveness of the teacher. There should be an expectation that not all probationers will make it through to final certification, and indeed only the most effective should be retained. The key judgement should be a minimum threshold of progress that the probationer’s pupils make. Obviously, the measurement of that progress and the parameters of the threshold require a great deal of careful work. Like any statistical data, estimates of teacher effectiveness will never be perfect, and a good deal of evidence over a number of years will be necessary to reach a decision, but this is clearly necessary to raise the average effectiveness of the teaching profession in England.

13. Ideally, the probation should involve classes of varying ability and year group.

14. In summary, our view is that the evidence shows that the selection aspect of ITT is completely the wrong way round. Selection is tight to get into ITT in the first place, but once in, progression to full certification is normal and expected. We argue it should be the other way around: we need to be more agnostic about likely teaching ability in the first place and allow a much broader group of people try out to be teachers, but have a much tougher probation regime before trainees be given final certification. It makes much more sense to make final decisions later once more evidence on effectiveness has accrued.

15. It is important to see the teacher labour market as a whole, and to see how the different stages of a teacher career fit together. This implies that the nature of the teacher labour market for mid-career teachers has implications back to ITT. It seems to be very hard to fire ineffective teachers. While the regulations on this have recently changed, generating a culture that encourages head-teachers to take a more proactive stance seems harder. While this may change, it may be that the best way to reduce the problem of low-performing teachers is to make it very difficult for ineffective teachers to get into the profession in the first place.

Training Aspect of ITT

16. We have less to say about the training aspect of ITT. Evidence suggesting that different routes into teaching do not seem to make much of a difference to effectiveness, and bolster the general view that people either have the ability to be a good teacher or not. However, while raw ability is very important, there is obviously much that a new recruit needs to learn about, and so the training side of ITT cannot be neglected.[5]

17. On the issue of what they should learn, two recent research papers combine traditional classroom observation of teaching practice with recent methods for estimating teacher effectiveness. Clearly, the design of a curriculum should depend on more than two studies, but this research points the way to establish a more systematic understanding of effective teaching practices.[6]

Wider implications of a tougher probation regime

18. These changes would make starting out on a teaching career much more risky financially. In order to maintain the same average lifetime expected income from the profession, the pay rate of those making it through to final full certification will need to be higher. And the lower is the chance of making it through, the higher is the full professional pay.

References

[1] Aaronson, D, Barrow, L and Sander, W (2007). “Teachers and Student Achievement in the Chicago Public High” Schools Journal of Labor Economics, Vol. 25, pp. 95–136.
Chetty, R, Friedman, J, and Rockoff, J (2011). The long-term impacts of teachers: teacher value-added and student outcomes in adulthood. NBER WP 17699. www.nber.org/papers/w17699
Hanushek, E (2011). The economic value of higher teacher quality. Economics of Education Review. Vol. 30 pp. 466–470
Hanushek, E A, and Rivkin, S G (2010). “Generalizations about Using Value-Added Measures of Teacher Quality.” American Economic Review Vol. 100, pp. 267–271.
Kane, T J, and Staiger, D O (2008). “Estimating teacher impacts on student achievement: An experimental evaluation.” NBER Working Paper 14607, NBER Cambridge
Kane, Thomas J, and Douglas O Staiger. 2008. “Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation.” NBER Working Paper 14607.
Rivkin, S G, Hanushek, E A, and Kain, J F (2005). “Teachers, schools, and academic achievement” Econometrica, Vol. 73, pp. 417–458
Rockoff, J E (2004). “The impact of individual teachers on student achievement: Evidence from panel data.” American Economic Review. Vol. 94, pp. 247–252.
Rothstein, J (2009). “Student sorting and bias in value-added estimation: Selection on observables and unobservables.” Education Finance and Policy, Vol. 4, pp. 537–571.
Rothstein, J (2010). “Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement*.” Quarterly Journal of Economics Vol. 125, pp. 175–214.
Slater, H, Davies, N and Burgess, S (2011). Do teachers matter? Measuring the variation in teacher effectiveness in England Forthcoming, Oxford Bulletin of Economics and Statistics.

[2] Staiger, D and Rockoff, J (2010). Searching for Effective Teachers with Imperfect Information. Journal of Economic Perspectives vol. 24 no. 3, pp. 97–118.

[3] Aaronson, D, Barrow, L and Sander, W (2007). “Teachers and Student Achievement in the Chicago Public High” Schools Journal of Labor Economics, Vol. 25, pp. 95–136.
Hanushek, E (2011). The economic value of higher teacher quality. Economics of Education Review. Vol. 30 pp. 466–470
Hanushek, E A, and Rivkin, S G (2010). “Generalizations about Using Value-Added Measures of Teacher Quality.” American Economic Review Vol. 100, pp. 267–271.
Rivkin, S G, Hanushek, E A, and Kain, J F (2005). “Teachers, schools, and academic achievement” Econometrica, Vol. 73, pp. 417–458
Rockoff, J E (2004). “The impact of individual teachers on student achievement: Evidence from panel data.” American Economic Review. Vol. 94, pp. 247–252.
Slater, H, Davies, N and Burgess, S (2011). Do teachers matter? Measuring the variation in teacher effectiveness in England Forthcoming, Oxford Bulletin of Economics and Statistics.

[4] Gladwell, M (2008). Most likely to succeed? How do we hire when we can’t tell who’s right for the job? New Yorker. http://www.newyorker.com/reporting/2008/12/15/081215fa_fact_gladwell
Goldhaber, D, and Hansen, M (2010). “Using Performance on the Job to Inform Teacher Tenure Decisions.” American Economic Review Vol. 100 pp. 250–255.
Jacob, Brian, and Lars Lefgren. 2008. “Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education.” Journal of Labor Economics, 26(1): 101–136.
Kane, Thomas J, Jonah E Rockoff, and Douglas O Staiger. 2008. “What Does Certification Tell Us about Teacher Effectiveness? Evidence from New York City.” Economics of Education Review, 27(6): 615–31.
Rockoff, J E, and Speroni, C (2010). “Subjective and Objective Evaluations of Teacher Effectiveness.” American Economic Review Vol. 100, pp. 261–266.
Rockoff, Jonah E, Brian Jacob, Thomas J Kane, and Douglas O. Staiger. Forthcoming. “Can You Recognize an Effective Teacher When You Recruit One?” Education Finance and Policy.
Slater, H, Davies, N and Burgess, S (2011). Do teachers matter? Measuring the variation in teacher effectiveness in England Forthcoming, Oxford Bulletin of Economics and Statistics.

[5] Clotfelter, C T, Ladd, H F and Vigdor, J L (2007b). “Teacher credentials and student achievement: Longitudinal analysis with student fixed effects.” Economics of Education Review Vol. 26, pp. 673–682.
Kane, T J, Rockoff, J E And Staiger, D O (2008). “What does certification tell us about teacher effectiveness? Evidence from New York City.” Economics of Education Review Vol. 27, pp. 615–631

[6] Kane T, Taylor, E, Tyler, J, and Wooten, A (2010). Identifying effective classroom practices using student achievement data. NBER WP 15803. www.nber.org/papers/w15803
Lavy, V (2011). What makes an effective teacher? Quasi-experimental evidence. NBER WP 16885. www.nber.org/papers/w16885
Metzler, J and Woessmann, L (2010). “The Impact of Teacher Subject Knowledge on Student Achievement: Evidence from Within-Teacher Within-Student Variation” CESifo WP 3111, Munich

March 2012

Prepared 2nd May 2012