Select Committee on Children, Schools and Families Minutes of Evidence

Examination of Witnesses (Questions 140 - 159)



  Q140  Mr Chaytor: May I put my first question to Keith? The chief inspector may tell us in his annual reports that we have the best generation of teachers ever and that standards of teaching are rising year on year, but what is the relationship between that and the testing regime? Since the testing regime came in and the publication of league tables became the norm, standards of teaching appear to have risen. How do you explain that relationship?

  Keith Bartley: First, I do not think that it is a causal relationship. Secondly, I observe that the implementation of the national curriculum in the late 1980s had a significant wash-back effect on initial teacher training. There is no doubt in my mind that teachers who are moving through into qualified teacher status now have a higher level of teaching proficiency within that curriculum because the curriculum is now much more closely defined than ever before. Year on year, we have seen a rise in the quality of the training experience. Running alongside that, we are also seeing a rise in the levels of qualification of teachers. Last year's cohort of qualified teachers was the highest qualified that we had ever seen in our schools. As for the question of why those higher standards of teaching and better qualified teachers are not manifesting themselves in improved levels of achievement, I would argue that during that period we saw a significant increase in the value placed on the tests. At the same time as we have a curriculum—indeed, there is a very high level of encouragement from Government through their curriculum policies to broaden and experiment with that curriculum—all the time, and particularly at the ages of 11 and 14, youngsters are being narrowed down by a very narrow system of testing. That is the bit where we are not unlocking the achievable.

  Q141  Mr Chaytor: Why do you assume that an improved quality of teaching is not leading to improved levels of achievement? Formal evidence from league tables, Key Stage tests, GCSE and A-level results, and vocational qualifications suggests the opposite.

  Keith Bartley: What we have seen, all the way through, is that shortly after testing has been introduced, there is an increase in standards. Typically, in the first few years after the implementation of testing for a key stage, we see a steady increase in achievement and standards, and then we see it tail off. That is very much the case with any performance measure, whether it is to do with schools, or with the punctuality of trains. What happens is that people work towards a performance measure, and it quickly plateaus. That is why you need to have a much broader band of assessments that can be used.

  Mick Brookes: Performance at the end of Key Stage 2 has been largely stuck for about five years, and until we start to do things differently, it will remain stuck. There have been improvements, not only in children's knowledge and understanding, but in teaching. We must be careful not to polarise the issue. Going from the 1970s and 1980s, there was, indeed, a need for much greater veracity of thought and practice in our schools. Although that might have been a painful process for schools to go through, we have gone beyond it. We now have sophisticated systems of assessment, which we should use, and, according to Ofsted, we have better standards of teaching in our schools. That worked then, but we must consider where we go next.

  Mr Chaytor: Mary wants to come in.

  Dr Bousted: I should like to answer the question directly. Ofsted does not investigate the quality of the tests. It takes the test level data and says that they indicate a rise in standards. If you examine the recent research by Peter Tymms, which was in the Robin Alexander review, he undertook a long inquiry into how difficult it is to measure standards longitudinally. In fact, two years ago, ATL published, Standards in English Primary Schools: are they rising? by Colin Richards, which also examined the issue. Tymms and Richards said that standards in reading had risen marginally, but nothing like what the tests state. Tymms makes it quite clear that when the Key Stage 2 reading tests started, for the first four years they found it very difficult to get a foundation level. The baseline against which they tested whether standards had gone up or down changed. It is also clear from a paper by Mary Hilton, on English in education, that three years into the Key Stage 2 tests, the level of questions that asked for an inferential response—not, "Did this happen in the passage?" but, "Why did it happen; what were the motivations?", which is a much harder question—went down. Tymms argues that only since 2000 has the baseline level against which you measure year on year, by each cohort, become steady. If you consider the rise since 2000, it is nothing like as great as in the first four years, so Ofsted will say that standards have risen dramatically because it does not question the evidence base. The question that we must ask is, "What is the validity of the evidence base?"

  Q142  Mr Chaytor: May I link that to the point that you made earlier about 25% of pupils not getting the grade that their ability merits at the Key Stage test? Surely the question is, who defines what their ability merits? It must be true in any form of testing and assessment that the subjective judgment of the tester comes into it. There are bound to be errors, surely?

  Dr Bousted: There are, but you must consider the level of confidence error that is acceptable. Part of the problem that we have with the tests is that they test a very narrow range, so it is particularly difficult in science to get a valid pencil and paper written test that tests the key concepts in science. They are taken on a certain day, at a certain time, and you can have only very narrow test items. You could have tests with different test items that tested other parts of the syllabus, and they would be equally valid but give a completely different result. In other words, our system has gone for high reliability: it is likely as not—we will put a lot of effort into it, and be as sure as we can—that the kids who take the test will get similar grades if they have a similar level of ability. We have problems with that, but they are also highly invalid.

  Q143  Mr Chaytor: May I move on to the question of the role of teacher assessment? Perhaps the question is directed to Brian. What does the evidence say about the relationship between teachers' judgment of ability and the Key Stages 2 or 3 results? Would it be possible to abandon completely external testing and replace it with an entirely teacher-assessed system?

  Mick Brookes: Certainly, by moderated or accredited teacher assessment, but I think using—

  Q144  Mr Chaytor: Sorry, what do you mean by accredited?

  Mick Brookes: Accredited would mean using a bank of tests, so that if a teacher said, "I think 80% of children in my class have got Level 4, but how come the test doesn't say the same thing?", you then have that professional conversation with your member of staff, not about those who scored the same, who are easy, but about those who scored differently. You must ask, "What's happened here? Is it your assessment? Is it that they were sitting next to Fiona, and that is why they got a higher mark?" It needs to be brought back into the school and discussed in an atmosphere where professional integrity is encouraged. If we could do that we would have a system that is fit for our children and does not completely subvert the year 6 curriculum, for example. I could take you to a school where they cancelled Christmas for year 6 kids because they were so worried about—

  Q145  Mr Chaytor: You have not told the Daily Mail this?

  Mick Brookes: They want to know where it is, but I have not told them.

  Brian Lightman: I absolutely agree. You have to have a clear definition of the standard that makes a Level 4 or Level 5. That is a problem: there is a big discussion about what constitutes a Level 4. You can test a little bit of it or you can do an overall assessment. To do that, you need to be able to draw down appropriate, high-quality materials that can be used to assess reliably where the students are. That sort of thing needs to take place. That professional conversation has been missing over the years, because people have relied almost entirely on external tests, rather than sitting down together as groups of teachers and saying, "In geography, which is my subject, a Level 4 is x, y or z; this is what constitutes it, and we need this evidence and that evidence to see that." Part of that will be a test and other parts will involve looking at students' written work and, perhaps, using oral assessments. There will be a whole bank of things. All of that put together and properly moderated will lead to a much more robust assessment, which will go back to the formative side of things—we can then advise students about how to improve.

  Chairman: David, a last one.

  Q146  Mr Chaytor: Finally, in terms of high-stakes testing and the publication of league tables, if there were a movement towards the greater involvement of teacher assessment, should that information be published in the league table list, or are you saying that they should be taken out completely?

  Brian Lightman: The results of all that assessment?

  Mr Chaytor: Yes.

  Brian Lightman: I think we have to ask how much we need to publish in the form of league tables.

  Mr Chaytor: I am asking you how much you think we should be—

  Brian Lightman: My argument would be that there is far too much going into the league table. We are adding more and more. Parents need and have a right to an overall picture of how the school is doing and how their child is doing individually. The move towards real-time reporting, where parents can see how their child is doing individually, is exciting and positive. At the other end, the whole school accountability side of assessment—which is a completely different purpose—could be published in much more effective ways than league tables, which often do not compare like with like, talk about the context of the school or look at raw data.

  Chairman: I know that you all want to answer, but if you have anything to say that is different from what Brian said, that would be the most useful.

  Dr Bousted: I want to go back to the issue of teacher assessment. Part of this discussion is predicated—or, rather, most discussions are, but not this—on the idea that teacher assessment is unreliable and invalid and that test assessment is good. Currently, at Key Stages 1, 2 and 3, you have teacher assessment scores against test scores. The issue is that teacher assessment scores count for nothing: they are not used by Ofsted and are not in the performance league tables. Teacher assessment is, however, usually significantly and consistently lower by 2 or 3 percentage points, so there is no reason to believe that teachers would make things up in a properly trained and moderated system.

  Mick Brookes: May I say an unequivocal no from the NAHT? That information should not be published in the league tables that we have now. It simply tells you where rich people live and, sadly, where poor people live as well. It is deeply undermining for those communities, teachers and children. I cannot understand why a Labour Government have allowed this to go on for 10 years.

  Chairman: Adam.

  Q147  Adam Afriyie: Thank you, Chairman—or acting Chairman. This is my first official outing on the DCSF Committee, and I am delighted to be here. My background is that I come from the Science and Technology Committee, so I have been fascinated to hear some of the comments about how assessment works, the baselines, the changes over time, and whether things can be compared. I want to concentrate my two questions on the single level tests, to examine them as briskly as possible, and to obtain your views on the proposed progression tests and the league tables that they will be placed in. Is it a good thing to use single level tests, or is it better to stick with the key stages or take another route for assessment?

  Brian Lightman: The first thing is that placing single level tests in league tables implies that everyone will have to do them at the same time, which flies in the face of personalised learning. If we are talking about personalised learning, we must be able to apply those tests at the right time. Trying to force the entire cohort through that at a certain time makes nonsense of the idea. The other thing is that we can use new technology for genuine testing when ready, which we must do, and we then return to the idea of downloading a bank of tools.

  Q148  Adam Afriyie: It is interesting that you have made the point about testing when ready several times, and I have heard it loud and clear. Would you propose something more along the lines of allowing pupils to progress when they are ready or hit a key stage, or move on fast when they are ready, or is that just purely the testing format rather than what happens with pupils in the year in which they sit in school?

  Brian Lightman: The only thing I have an issue with in the way you expressed that would be if you are saying that they should be allowed to progress when ready. We, as teachers, want to ensure as much challenge as possible, so we do not want to tell children that they can take as long as they like. We want them to progress. That is one of our issues about two levels of progress within a key stage, because that is an arbitrary decision. Key stages are different lengths. For example, there is no comparison between Key Stages 2 and 3 in length and time, so why talk about two levels? Some students could progress three levels, but it would be unreasonable to expect others to do so. Also, the levels are not equal. The idea is not as simplistic as the music grade analogy that is sometimes used.

  Dr Bousted: Briefly, we think that making good progress is fraught with difficulties, and we do not think that the Government have thought through the matter. The single level test, taken up to four times a year, could lead to a huge proliferation of the test, and all the issues arising from that—narrowing of the curriculum and teaching to the test—could be ratcheted up to four times the present level.

  Q149  Adam Afriyie: So you think the Government are just plain wrong.

  Dr Bousted: Yes, just plain wrong. There is also a question about the validity of the levels. If a single level test is shorter, how valid is it and how worth while is the information that it provides? We agree absolutely with the Association of School and College Leaders. We all know that a less able child will find it far more difficult than a more able child to move two levels. That has been shown already in the current progression rates. Schools where less able children congregate are often in more challenging areas or where children have English as their second language. They would be penalised by a progression measure that they could never achieve. In schools where children arrive with Level 5 they are much more likely to be able to move two levels than those in schools where the majority of children arrive with Level 3. It is much more difficult for them to make that progress because they learn more slowly. Yet that school might be doing an equally good job, but be penalised by a completely inappropriate progression measure. That has been scientifically determined by the levels, so why the Government have chosen this measure, I do not know. Finally, a system of repeated and proliferation of testing is like excellence and enjoyment. The Government think that you can put in excellence and enjoyment, and have a high testing regime and a broad curriculum, and that just by putting them in the same title it will happen. They also think that as long as assessment for learning and progression level testing are in the same document, that is all right. You will not get assessment for learning in the system. They could have done so many things, but this is not the right thing for them to be doing.

  Q150  Adam Afriyie: Thank you. And Mick, do you agree with that view?

  Mick Brookes: I would like to encourage the Government to move on from where they are. The concept of having a bank of tests that are appropriate for a single level is right, but it is the way in which it is being rolled out. To do the right deed for the wrong reason is the greatest treason. The tests genuinely have to be for schools' use in assessing their children. That assessment is then reported to the right people: the parents, the children themselves and, of course, the governing body. The problem is trying to use the same tests to judge the performance of schools. I think that we have made that point.

  Q151  Adam Afriyie: I hear your point. You are arguing for separate tests or assessments for teachers, for the school and for the pupil—they are not necessarily the same thing.

  Keith Bartley: May I just put a slightly different take on that? At the moment, our reading of the Making Good Progress pilots is that they are still about testing when the system, rather than the child, is ready. Actually, if you were to place a greater emphasis on the validity of teacher assessment, and if you were to provide teachers with a bank of tests that they can draw on at an appropriate stage to confirm their judgment of where a child is in a particular aspect of his or her learning, you would change the system. The learners' needs would drive the assessment and, therefore, the subsequent teaching. Making Good Progress is a step in the right direction, but we would particularly like a robust evaluation of its impact and the possibilities that it is beginning to open up.

  Q152  Adam Afriyie: I think we already had the answer to my final question in previous answers. Do you consider that it is in any way possible to disconnect summative assessment of the children from the monitoring and performance of the school or the teacher? Is that possible?

  Mick Brookes: Yes, I think it is, by sampling. Of course, we must retain some idea of national benchmarking to see where things are. That is important because this is part of the public purse and therefore the public need to know how well education is faring across the country, but that could be done with sampling rather than going to every school. Going back to the previous question, I think an analogy with the driving test is a good one. Some people might need five lessons before they are ready to take the test; others will need 20 and some might need 120.

  Brian Lightman: I would agree with that.

  Q153  Lynda Waltho: My feeling is that testing and assessment is coming out as, "Yes, we need it, but perhaps not in the way we are doing it." I do not get the same glowing feeling about league tables. Without league tables and other value-added measures, can you suggest a measure of school performance that can be used by the Government for accountability but also by parents for their decision making? Does that exist? Is it possible?

  Mick Brookes: I think that parents are being grossly misled by raw, or league table, data, and, indeed, by some Ofsted assessments that are driven by exactly the same set of figures. That creates polarisation. When it comes to admissions, people will travel and do all sorts of strange things to get their child into the school of their choice because of that polarisation. Actually, their local school may be very good, but the only way they can find that out is by going there and having a look. We think there should be an emphasis on parents, who should make sure that they visit their local school and see the work that it is doing. As an experienced head teacher, I could walk into any school and within five minutes tell you whether it is a good school. Parents then need to look at the results, but they need to look at them in the context of their community. That is not necessarily about poverty. It is also about the expectations of the community in terms of education qualifications, which is something that Ofsted does not take into account. The level of higher education qualifications in the community is a key factor in whether the children will expect to progress in education. We are doing the job of moving those expectations on, and we need to keep doing it. Parents must go and see.

  Brian Lightman: What are the criteria for a good school, and what are the criteria that parents look for? I show parents around the school. I strongly encourage all parents to come round to our school before they make their decision about whether to send their child there. They want to know whether they are going to be happy, safe and encouraged to make the best progress they can. It is those types of things that they want to know. Over and above that, parents look at our school prospectus. They read the school prospectus, which contains, by statute, a whole range of indicators against the school's performance and they can see a detailed account of the school's performance. I know I am talking from a different context, where I work, but that is a much more valid approach than putting the information up, as it was done in the paper last week, where the top 10 schools in any authority—the best 10, they were called—were published, whether it was an authority with 100 schools or one with 10 schools. That is the way they published the league tables last week. That does not tell you which ones are the best schools. Looking at the broad picture of a school is the way in which you can judge its quality—and I think there are plenty. There is also the Ofsted report—I hope you know—and all the information in the public domain. There is a vast amount of information enabling a parent to see what a school is like.

  Q154  Lynda Waltho: So we are never going to be able to do it with a league table. Is that the feeling?

  Brian Lightman: I do not think that we are going to have the sort of accurate information, which gives a true picture, with league tables the way they are.

  Dr Bousted: The problem with league tables is that you can write whatever you want, but in the end the drop-down system for league tables is so pernicious in other respects. Let us consider a school in a challenging area that is working hard and making the good progress that we know a school can make. Variation between schools is much less than variation within schools. We know a school can make about 14% of the difference; the rest of the determining factors on a child's achievement come from their background, actually. And 14% is a lot; I am not undermining what a school can do. However, that means that schools in the most challenging areas have to work extremely hard to get the results that they do. It is up to Ofsted to go into that school. We have one of the most accountable systems in the world. Ofsted goes in and looks at whether the school has sufficiently challenging targets for pupils—whether it is sufficiently demanding of their progress, whether the behaviour in school is appropriate and whether the teachers are working well enough. Ofsted looks at a range of issues to do with the performance of that school. But if you are in that challenging school, doing very good work and adding clear value to the children's lives in that school and you are still, because of the nature of your intake, going to be at the bottom of your local authority league table, that is not a good thing. I will tell you another thing: it is not just not good for the teachers, but it is not good for the pupils going to that school, who need to feel good about themselves.

  Keith Bartley: May I share some of the research that has been commissioned from parents? MORI, in 2005, discovered that parents attributed very low value to league tables in assisting them to determine their choice of school. That was partly because they felt that the information that they portrayed was to some extent confusing. For others, it did not match what parents saw as being the really important things about schools. The list that Brian gave captured those things. We undertook some further research through the British Market Research Bureau about what parents found most helpful in determining the quality of a school. They looked for a much broader range, particularly in respect of contextual factors: the nature of the young people that the school educated, the kinds of aspirations it set out for those young people and the way in which it delivered against those. That wider bank and portfolio is what parents said they would find most useful. League tables, parents tell us, are questionable in terms of their value. The further dimension is that as our 14-19 curriculum unfolds, and particularly as we seek to offer young people a much broader range of both education and training opportunities through to the age of 18 or 19, the influence of an individual institution is going to be much harder to measure through league tables, because young people will have been involved in several institutions—maybe a college, maybe two schools or maybe a training provider—so their validity will become even less as our system better meets the needs of that older group of pupils.

  Q155  Lynda Waltho: In light of that, although it is slightly unfair to mention this because possibly only one of you may have read The Western Mail today—[Interruption.] Okay. Actually, it refers to Wales. It published comments by Professor David Hopkins, who has said that school tests and league tables should be brought back. In fact, it said that, "Statistics show children in England, where testing the two ages and league tables remain in place, performed better last summer in key exams like GCSEs and A-levels." Professor Hopkins said, "We very much know that the performance at seven correlates to success at 16 ... The previous system was too harsh but Wales went too far the other way." That is also backed up, to a certain extent, by Professor David Reynolds, who described Wales's PISA—Programme for International Student Assessment—ratings as "awful" and said they are falling behind England. I am sorry to zero in on you, Brian, but obviously I know that you have a more direct experience of this subject. In the light of that report and bearing in mind your comments earlier, what would you say in response to that?

  Brian Lightman: I have not seen that article this morning. However, one of the big discussions that I have had with the Welsh Assembly Government about the way that they have published the results this year is that I discovered a few weeks ago that what they understand as five As to C and what the English league tables describe as five As to C are completely different measures. That is an example of the type of thing that happens. In England, there are all kinds of qualifications that count towards the Level 2 qualification and in Wales they have not got round to including those qualifications. So, for example, my school does the DiDA qualification—Diploma in Digital Applications—that counts as four GCSEs here in England, but in Wales it does not count. So you have to be very careful about how you produce these figures and let them mean what you want them to mean after one year. This is one year's results and one indicator and we must be very careful about the way that we read things into those results. Obviously, in Wales as in England, we want our children to achieve the best possible standards. So we must look at a range of measures and not just one thing. At the moment, it is not possible to compare England and Wales. It is not possible to compare national curriculum tests or assessments in England and Wales, because the national curriculum for each country is different. So there is a real danger that things will be misinterpreted in order to put forward a particular point of view. I would be very careful about that.

  Mick Brookes: Of course, the mechanics of this system are bound to want to defend it; David Hopkins was one of those. But I think that he is absolutely wrong. I think that it is absolutely right to say that comparability between England and Wales is very difficult, but those mechanics need to look at other things. For instance, they need to look at the UNICEF report about the happiness of children in this country. They need to look at all the work that has been done by Robin Alexander on the primary curriculum. There is a lot of evidence now that says that we are simply focusing on too narrow an area and it is having an effect on children's lives and it is certainly having an effect on the curriculum. We must try to measure the things that we value, rather than valuing the things that we measure; that is an old cliché, but I think that it is true.

  Q156  Lynda Waltho: You talked there about the comparison between England and Wales, but the Organisation for Economic Co-operation and Development tests—

  Brian Lightman: Sorry, which tests?

  Q157  Lynda Waltho: Some 400,000 15-year-olds across 57 countries were tested for the first time in 2007. Wales came bottom in the UK for maths, reading and science, and trailed at 33rd place for maths, beneath Latvia and Lithuania. That is obviously a wider comparison, across a wider set of nations, in which England was rated along the same lines. I am just wondering about that test.

  Brian Lightman: But, you know, you get those results and you then have to start asking questions. At the moment, there has not been time in Wales for anybody to analyse the evidence base of that test. There will be questions asked about that test, but I cannot draw a conclusion about the results until we see the evidence base. It is the first time that those assessments were used in Wales and we need to look at the evidence base now and ask questions about it. I do not think that it would be fair to draw on the results of that one test for the first time and come to some conclusion that is perhaps making two and two into five before we have analysed what the evidence base is and what those results are saying.

  Mick Brookes: And I could direct you to Jersey, which has no SATs, no league tables and a totally different inspection system and which would be top of that league. I do not know whether it is in the table, but the results from schools in Jersey and staying-on rates in Jersey are much, much higher and it does not have the same system as we have, so it depends where you look.

  Chairman: Indeed, and Jersey does not have the variety of population. These tests are exactly the kind of tests that I have heard you arguing for. They are like the APU—Assessment of Performance Unit—tests. They are internationally moderated. Anyway, we will leave it there and I will give Sharon a chance to ask her questions.

  Q158  Mrs. Hodgson: One question sprang to mind on the back of the decision-making that parents go through and how they use the league tables to make that decision when perhaps they should be visiting the school and making the decision on a whole host of other things, one of which I believe should be the distance that the child would have to travel. My daughter has just started secondary school and is within walking distance of the school. It is a good walk, but it helps to keep her fit and she and my son do the walk together. One of my daughter's friends is late about twice a week because she has a horrendous journey. She probably passes between six and 10 other high schools to go to that one. Obviously, her parents have chosen the school for a whole host of reasons, but the experience that that girl—and probably many other children throughout the country—is going through must be affecting her learning in negative ways. I am thinking of the stress of constantly worrying whether she will catch the bus and getting into trouble when she gets to school late. Eventually that might affect her whole learning experience at school. My children were in school in London for a while and some children commute to school from outside London. They have a commuting distance that an adult would consider a chore. What are your comments on that?

  Dr Bousted: In London that is endemic. I get on the train every morning and my carriage is delightfully shared with schoolchildren commuting from one area to another. A long commute to and from school must affect the learning ability of children, notwithstanding what it is doing to the environment and everything else, and of course that relates to the school run as well. This is often about the parents' perceptions of a good school, and the perception of a good school is often based on the class of the intake.

  Mick Brookes: I was going to say the same thing. Rather than being based on the quality of teaching and learning, the decision is sometimes based on the fact that the school is full of the children of "people like us".

  Dr Bousted: It nearly always is.

  Q159  Mrs. Hodgson: So there are parents who want to get their children into the school for that reason?

  Dr Bousted: That is right.

  Chairman: You wanted to ask about Making Good Progress.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 13 May 2008