Select Committee on Children, Schools and Families Minutes of Evidence


Annex A

Making Good Progress

RESPONSE OF THE ASSOCIATION OF SCHOOL AND COLLEGE LEADERS

A.  INTRODUCTION

  1.  The Association of School and College Leaders represents 13,000 members of the leadership teams of colleges, maintained and independent schools throughout the UK. This places the association in a unique position to see the progress measure from the viewpoint of the leaders of both secondary schools and colleges.

  2.  ASCL welcomes the opening of a debate by the Secretary of State on progress measures. Helping every student to make good progress is the function of the education system and so consideration of policies aimed specifically at supporting that is long overdue.

  3.  The use of pupil progress measures is in principle a move in the right direction of intelligent accountability for schools. Good teachers measure the performance of individual pupils on progress made and it is right that the same principle should be used to measure the performance of schools.

  4.  However, the proposals as set out will not have the desired effect. There are several specific aspects about which ASCL has major concerns. The association looks forward to a period of consultation and piloting in which the best features of the proposals can be developed and the worst amended or dropped.

  5.  To support such a process an alternative measure is proposed in section E of this document which the association believes would command much greater support, avoid the faults of the measure proposed in Making Good Progress, and would better lend itself to target-setting at all levels.

B.  ASSESSMENT AND TESTING

  6.  The document's clear statement of assessment for learning and endorsement of it is welcome. ASCL shares the belief that this approach, always used by good teachers to some extent and in some form, can be usefully extended; and has for some time championed it. However, it should be remembered that high-stakes, externally marked tests are antipathetic to assessment for learning. For testing to be supportive of learning it must be kept closer to home, with frequent assessments (of all kinds) devised or chosen by the teacher and marked by the teacher.

  7.  The association welcomes a move away from age-linked testing. The further idea of "testing when ready" is also welcome. Sadly, what the paper sets out is not that but rather a proliferation of the existing testing regime. Modern technology can surely lead us to aspire to forms of testing that enable students to be tested whenever they are ready, not on a given day in an examination hall in six months time. To propose tests on the current model, but more often, is to miss an opportunity to devise something better for our children, and potentially to exaggerate the faults and costs of the present system.

  8.  In both of these areas the paper is hidebound by the prevailing orthodoxy of testing, which has prevented any genuinely creative ideas.

C.  PERSONALISED LEARNING

  9.  ASCL also welcomes the renewed emphasis on personalised learning. It is closely bound to assessment for learning, and is again not new; good teachers, and good schools and colleges, have always tried to personalise their offering to students.

  10.  This is recognised in Making Good Progress and in the 2020 Vision report, one of the good features of which was its recognition of the good practice already in the current education system. The present document somewhat loses sight of that by extracting (on page 14) a list of approaches that schools "will need to adopt" as if they were not all in the usual repertoire of school behaviour. Some may need more emphasis in some schools.

  11.  The "personalised classroom" as set out in the first paragraph of page 16 is an attractive prospect, but for it to be realised it is imperative that the teacher not only has ready access to the necessary data but also can rely on it. The present high-stakes testing regime and the weakness of the national curriculum tests prevent any such reliance. Many schools make use of CAT, Midyis or other diagnostic tests for example because they do not feel able to rely on the National Curriculum tests, which were devised as summative tests, as a good baseline for predicting the future performance of each pupil.

  12.  A more rapid response to pupils who are falling behind is clearly welcome, provided that that does not translate into ever more frequent, stress-inducing, external tests. Our young people have become the most tested in the world, and their stress levels have risen markedly as that has happened. There is now a need to take greater care with their mental health and normal development.

  13.  The document does recognise at this point that teachers are already skilled at discovering the progress of their individual pupils and tailoring their courses to their needs.

  14.  ASCL welcomes the clear statement that personalisation does not mean devising a separate plan for each student, and the renewed promise of greater flexibility in the secondary curriculum to allow schools room to be more creative in devising programmes suited to their particular students.

  15.  There is a contradiction between the idea of personalised learning, which recognises the different needs and abilities of each person, and the setting of systemic targets, which presupposes that all young people should ideally travel the same path at the same rate.

  16.  The suggestion in the 2020 Vision report that students from disadvantaged backgrounds should receive additional support is welcome, and the document does no more than reiterate this. However, at one of the DfES presentations to stakeholders this was extended to an intention to provide 10 hours of individual tuition to students not "on trajectory", possibly at home, at weekends or in the school holidays—provided by local authorities. ASCL cannot welcome this interpretation of the 2020 suggestion. It would be very expensive, costing far too much to administer as well as overlooking the possibility of joint work with small groups of students in similar states of learning and with similar needs. It would be very unlikely to be good value for money.

  17.  If additional funds are available for this type of support they should be delegated to schools, which are closer to the individual students and will be better able to apply them than local authorities.

D.  MEASURES AND TARGETS

  18.ASCL strongly opposes the proposals in section five of the document. In this section the proposals go badly wrong in ways that would ensure that the good intentions of the earlier sections could not be realised.

  19.  First, measures framed as "the percentage of children who . . ." are bad measures of progress. They concentrate the attention of teachers, schools, partnerships, local authorities, inspectors, government and the media on those children on the borderline of making the grade when we should all be interested in the progress of all children.

  20.  Sensible measures in this area should look at the distance travelled by each child related to how far we might reasonably expect a child with that starting point and that set of attributes (disadvantages for example) to travel in the time.

  21.  Secondly, the proposal that every child should be measured against an improvement of two national curriculum levels is absurdly crude. It may be easy to understand, but will mislead most of those who see it, and will create new perverse incentives as damaging as those caused by the some of the present measures. For every complex and difficult problem there is a simple and straightforward solution . . . that is wrong.

  22.  That a child who is badly behind at the start of a key stage, a child who is a high flyer, a child with a strong leaning towards or away from a particular subject, a child with every sort of support, a child with a profound disability, a child with severe social disadvantage, a child simultaneously learning English, a child who learnt English at a previous stage should all somehow move on two levels bears no examination. In fact, any research that has been done into these and other interrelated factors is ignored here.

  23.  The information that is set out in Making Good Progress points to a further weakness in the proposals; that they would systematically favour selective schools and other schools that have a more able than average intake. This is illustrated clearly by looking at Key Stage 2 to Key Stage 3 mathematics:


  This diagram is highly simplified, some children make greater or less progress than indicated, but the arrows show the most common movements during Key Stage 3. Note that on the whole those children who start at lower levels mostly do not make two NC levels of progress, whilst those who start at higher levels mostly do. The effect at school level would be to create a measure that would imply that a school with a "good" intake is doing better than a less favoured school, almost independently of the excellence of the schools themselves.

  24.  Thirdly, the key stages are of different lengths, and the national curriculum levels not designed to be of equal size. So it is inevitable that, quite apart from the many individual differences between children, a target of two levels per key stage will be much harder to reach at some stages than at others. Figures in the document itself make it quite clear that this is the case. Adopting such a set of measures would therefore invite media attention of the most unwelcome and ill-informed kind: "only 30% of children make acceptable progress in English between age 11 and 14", for example.

  25.  Encouraging such misunderstanding cannot be to the advantage of children, schools or the Government itself. Many children will be given the false impression that they have "failed", when in fact they have made perfectly normal progress. Secondary schools will be painted as failing their pupils even when they have made above average progress. And the Government will be accused of allowing a systematic failure of education at Key Stage 3.

  26.  Targets framed in this way will set up new perverse incentives. Students who part-way through a key stage have already clearly made their two level improvement, or who clearly cannot do so, will not be targeted as intensively as those who may or may not make that improvement.

  27.  A two key stage improvement may be an appropriate aspirational target for many individual students, but it is not appropriate for all, and it is certainly not appropriate as an accountability measure for teachers, schools or Government at Key Stage 3.

  28.  Fourthly, the relationship between the national curriculum tests and GCSEs is not close and not well understood. The document seeks views as to how a measure based on a percentage of those moving up two levels on these two incommensurable scales should be formulated. It should not be formulated at all. Any such formulation will fail to measure anything meaningful and will create perverse incentives of the worst kind.

  29.  Fifthly, this whole section is predicated on the national curriculum tests as robust and reliable measures of attainment. These tests are better now than in their early days but are still not capable of bearing the weight of all the many uses to which they are already put. It is not sensible to erect a further edifice of measures and targets on them.

  30.  Finally, this section asserts that these new measures should be added to all the existing measures and not replace anything. This is simply wrong. The English education system already has more tests and measures than any comparable system, a larger proportion of scarce resources is diverted from actual learning into setting tests, preparing for them, administering them, analysing the results, reporting the results, and dealing with the inevitable misunderstanding of them by children, parents, governors, the media and others.

  31.  The assertion that nothing can ever be removed from a bureaucratic system does not sit well with recent attempts by Government to reduce bureaucracy and improve the intelligence of accountability systems. In this case it rests upon a separate assertion, made at page two, that it is the elaborate system of tests, targets and performance tables that has driven up standards in recent years. No evidence is brought to support that idea, which has taken on the aspect of a dogma. Indeed, on the very same page of the document it is undermined by the assertion that it is the Government's increased investment in education that has had the beneficial effect. This seems a more likely explanation: our schools are better led, are better staffed, have better facilities and are better resourced than before, and this has been reflected in better progress. Pupils and teachers are also more experienced at the tests, which is bound to have had a beneficial effect on scores nationally.

  32.  ASCL urges as a matter of general principle that initiatives should not be taken, in a system already at full stretch, without indicating what it is that they should replace. In this case progress measures are welcome but must replace some of the alternative measures that have now had their day, and done whatever good they may have been able to do. There are plenty of candidates . . .

E.  AN ALTERNATIVE MEASURE

  33.  As already stated ASCL welcomes the idea of a measure of progress. As a constructive response to Making Good Progress what is set out in this section is an outline of such a measure that would command the support of school leaders.

  34.  Any such measure should avoid the perverse incentives inherent in "the percentage of students who . . ." but should depend on the progress made by all students. It should also intelligently reflect the starting point of the student in question.

  35.  What is needed as a first step is a more complete and careful analysis of a cohort of students, say those that took the various tests in 2005 that will give an expected outcome for each student based upon the starting point. This will also answer the question, left open in Making Good Progress, of what progress can be expected between Key Stage 3 and GCSE. What ASCL proposes is the use of this statistical relationship as a baseline for expected performance against which performance in future years can be judged.

  36.  Thus each student's result can be compared to the expected outcome and a positive or negative "residual" determined. These residuals can readily be aggregated to give an average for a class, school, local authority and the country as a whole.

  37.  This should be familiar as the approach taken by value-added measures. Like them it would avoid perverse incentives and be based upon careful research into the actual performance of real students.

  38.  Traditional value-added measures have the drawback that they are cohort referenced*, meaning that they relate an individual or a group with the averages for the year group to which they belong. This has some disadvantages. The individual's score is partly determined by the performance of the peer group. Such measures do not really reflect change from year to year; a teacher, department, school, partnership or local authority can improve in performance but find that that is not reflected because others have improved too. And in particular they hide improvement in the system as a whole—the average residual for the whole group must by definition be zero.

  39.  So what ASCL proposes is different in a crucial respect: the performance of future students should be compared not to their own peers in their own year-group, but to the fixed 2005 reference group. In the sense in which we are using the term here this would make the measure norm referenced* rather than cohort referenced and therefore avoid the drawbacks outlined in the previous paragraph. It would allow for year to year comparison of the performance at all scales from the departmental to the national.

  40.  In particular any improvement would be reflected in the measure which could thus be used for any target-setting. The average residual for the nation as a whole would no longer necessarily be zero; if the education system improves new cohorts of students will do better than the 2005 group and this will be reflected in a positive residual. These residuals are expressed as fractions of a National Curriculum level (or GCSE grade) and are therefore relatively easy to understand if not to calculate.

  41.  There is at least one precedent for such an approach in the SAT (formerly Scholastic Aptitude Test), widely used and respected in the USA. Before 1941 this test was cohort referenced but after that date comparison was made each year with the cohort of 1941 thus making the test norm referenced.*

  42.  It is not likely that in the more rapidly changing modern world a reference group could be retained for more than sixty years, and the measure could be rebased from time to time, but it is envisaged that the same base should be used for, say, a decade at a time in order for systemic progress to be observed.

  43.  As far as the secondary phase of education is concerned ASCL suggests that the main measure of progress should be that between Key Stage 2 and GCSE. This reflects the most common patterns of secondary organisation with 11-16 and 11-18 schools. In such schools measures involving Key Stage 3 are clearly subsidiary and should be primarily for internal use rather than used to rate the school as a whole.

  44.  ASCL will be pleased to help develop these ideas further as part of its commitment to more intelligent accountability.

F.  PROGRESSION PREMIUM

  45.  This idea is particularly unwelcome. Teachers and their leaders are motivated by a desire to do right by those in their charge, not by a desire for a bonus. Such a premium, especially one built upon a measure that lacks full professional confidence, would either reward in a capricious fashion or would systematically reward those that need no such reward (ie those schools teaching the best supported pupils with the fewest disadvantages).

  46.  ASCL would remind ministers that the School Achievement Award was scrapped for very good reasons. They should not seek to reintroduce a similar, but equally flawed, reward.

  47.  ASCL strongly suggests that this idea be dropped forthwith.

G.  CONCLUSION

  48.  ASCL welcomes the basic idea of Making Good Progress and its aspirations. School and college leaders have always striven to help all their students achieve as much as they can.

  49.  However, the actual proposals contained in the document, especially those in section five, would not help in any way to do this, and would in fact do far more harm than good.

  50.  ASCL would strongly suggest that the whole of sections five and six, and some of section three as outlined above, should be set aside. There is a need for some genuinely new thinking about these important matters so that the whole system of assessment, testing, reporting and accountability can be amended to better serve the worthy aims of Making Good Progress.

  51.  ASCL stands ready to contribute to such thinking and trusts that the major amendments proposed above should be made before the pilot begins.

January 2007

*  Thanks to Professor Dylan Wiliam, Deputy Director of the London Institute of Education for the explication of the difference between cohort referenced and norm referenced, and for drawing attention to the norm referenced nature of the SAT. Norm referenced is often used as opposed to criterion referenced to describe measures that would more properly be called cohort referenced. To be clear: in this paper norm referenced means comparing the performance of a child or group of children with a fixed reference group (say those that took tests in 2005) whilst cohort referenced means comparing a child or group of children with those taking the tests in that year.

REFERENCE

    Wiliam, D. (2007). Balancing dilemmas: traditional theories and new applications. In A Havnes & L McDowell (Eds), Balancing dilemmas in assessment and learning in contemporary education (pp 269-283). London, UK: Routledge. (To be published later this year).







 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 13 May 2008