Examination of Witnesses (Questions 20-39)
DR STEVE
GIBBONS, DR
TOM BENTON
AND SIMON
RUTT
30 JANUARY 2008
Q20 Paul Holmes: Politicians in search
of the holy grail have said that the answer to problems with pupil
attainment and school improvement is diversity through the provision
of CTCs, faith schools, trust schools or whatever they are called.
However, all the evidence that you have given seems to indicate
that that does not matter and that it is the intake of pupils
that makes the difference. Is that a fair summary of what you
said?
Dr Benton: Sure, it is a fair
summary of all the research that we have done. Furthermore, if
you look at the outcomes for schools in terms of different subjects—for
example, English results or maths results—you can ask whether
the schools that are overachieving in maths are the same ones
that are overachieving in English. When you do that, you find
out that they are very different schools.[3]
There is a relationship between the two—there is a correlation
between overachieving in one and overachieving in the other—but
if you look at different subjects, you get different results,
which indicates that whole school changes may not be the most
important thing in driving results. It may be that subjects work
more individually than that. You have to think, how do we improve
English, how do we improve maths? A lot of it could be achieved
at the subject level rather than the whole-school-approach level.
Certainly in terms of school type, that is not a major driver.
Q21 Paul Holmes: Okay. You have said
that where you have diverse schools, especially if they are in
control of their admissions, they start to select by academic
selection, social selection and so forth. Is there any evidence
that diversity and selection of various kinds have an adverse
effect on other schools in the area?
Simon Rutt: On the admissions
side, we looked at communities that have more than one school.
So if a voluntary aided school is taking an unfair proportion
of pupils on free school meals, we found in a number of areas
the knock-on effect appeared to be that the community school in
the same area had a higher proportion of free school meals and
SEN pupils than the selective schools. It appears that if one
school takes fewer pupils on free school meals and SEN than you
would expect, the other schools in the area take more. That, in
turn, has a knock-on effect on attainment.
Q22 Paul Holmes: The programme for
international student assessment in OECD countries has consistently
said that the two best performing countries in the world are South
Korea and Finland. The one thing that they have in common is that
they have local schools, and not much else besides. Is the lesson
that the comprehensive system of local community schools is better
than diversity?
Dr Benton: When it comes to the
PISA countries comparing countries, there is a vast number of
differences in the education systems in different countries. Immediately
saying "These countries are the best, and it must be because
of the comprehensive system they have both got", is probably
too much of a leap to be certain about.
Q23 Paul Holmes: If you look through
the PISA studies, you generally find that the countries that have
selection, such as Germany, England and the USA, do very well
with academic pupils but have a huge tail of underachievement
compared with the countries that have more non-selective systems.
It is not just about the top two countries.
Dr Benton: Sure, I understand
what you are saying. However, you have not got an enormous number
of countries in those studies, so statistically it is difficult
to see how you can draw robust conclusions. Although it is interesting
to speculate along those lines, you could not see that in any
way as being a proof that a comprehensive system is the better
one.
Can I return to your previous point about
the negative influence of selection?
Paul Holmes: Indeed.
Dr Benton: There has been some
further research on the positive effect that selective schools
seem to have on pupils who get in them. When we compared them
with secondary moderns, we found that there seemed to be a converse
negative effect of a much smaller size that affects a greater
number of pupils. If you consider a local authority as a whole
and consider the relationship between the percentage of pupils
who are selected and overall achievement within the local authority,
I think that the effect would more or less balance out. It appears
that the positive effect on those pupils who get into selective
schools is perhaps balanced out by the effect on surrounding schools.
Q24 Paul Holmes: You mention inquiries
into the evidence on academies in the written evidence. Part of
the problem with looking at academies is that they have not been
running that long, so it is hard to tell the long-term impact
on intake and certain areas. However, there is evidence that in
those 24 academies the admission of pupils from deprived backgrounds
fell from 42% in 2002 to 36% in 2006. Is that just a readjustment,
because the academies were replacing failing sink schools that
had too high a proportion of pupils from such backgrounds, or
is it that the academies have started to become socially selective?
Is it too early to say?
Simon Rutt: I suggest that it
is too early to say. From those statistics, it is difficult to
say whether they are balancing themselves out to take account
of pupils applying to the school or whether they are now being
selective. By having that information we would be able to determine
whether they are starting to operate selection policies on pupils
getting into schools.
Q25 Paul Holmes: Academies are relatively
new, so we will have to see how the situation pans out, but CTCs
have been around for a lot longer. Are there any studies of the
CTCs, some of which have been in existence for 15 years or more,
examining those effects?
Dr Gibbons: I have looked at the
intake of CTCs compared with other schools. Between 1996 and 2002,
CTCs had a much more compressed intake in terms of the distribution
and level of achievement of the kids coming in. They were selective
and they had higher achieving pupils as well. The CTCs were in
our estimation de facto selective, but the mechanisms through
which that is working are not completely clear. They claim to
have a comprehensive intake but we found evidence that they do
not. They are extremely selective, and, although that is not to
the same extent as a grammar school, it is still significant.
Q26 Paul Holmes: So, there is clear,
uncontroversial evidence that CTCs have become selective in some
ways?
Dr Gibbons: Yes.
Q27 Chairman: The original framework
that the CTCs were given included the ability to band. You three
have all told the Committee that in order to give a school a fair
chance of achievement, you need a balance of abilities that reflects
the community rather than distorts the community. Is that the
case?
Dr Gibbons: The process by which
CTCs admit pupils is admittedly mysterious to me. I am not quite
sure. When I trawled through the admissions policy of the CTCs,
it was said that they were trying to pick a balanced intake from
the London community, yet they were allowed to slack on aptitude
and specific skills. How those two matters square, I do not know.
In the end, the policy winds up being slightly selective.
Q28 Chairman: Earlier, you told Paul
that, to obtain achievement, a balanced intake is needed.
Dr Gibbons: I did not mean to
say that.
Q29 Chairman: I thought that you
said 90% of the results from a school depend on its intake. You
said that if we represent our community and get a fair balance
of the community, we can do wonderful things to raise levels of
achievement. However, what about a preponderance of children who
are on free school meals, have SEN or are looked-after children?
We visited schools with 100% free school meals, let alone 65%
SEN. They find it difficult to raise levels. Is that the truth
or is it not?
Dr Benton: That is not quite what
we are saying. We are looking at individual pupils, so we can
see their characteristics and know what we expect them to achieve.
If we look at them at an individual level, that is where 90% of
the difference is. The make-up of the school is not so important,
but each individual's characteristics affect each individual's
chances of achieving later on. That is where most of the differences
between schools lie. Do you see what I am saying? From a pupil's
point of view, the people around the pupil are not as important
as the pupil's characteristics. There is some evidence of schools
with a higher average intake doing better than other schools.
There is an effect of having high ability kids around other pupils
in terms of the other pupils' achievements, but that is smaller
than the 90% figure, which is based on an individual's characteristics
affecting an individual's chances.
Q30 Paul Holmes: I have talked about
politicians looking for the holy grail, as in diversity. Another
holy grail that is often trotted out is super-heads who, through
their dynamism and personality, can transform a school regardless
of its intake. Is there qualitative, statistical evidence to back
up that statement or contradict it?
Dr Benton: We did a survey of
all schools in London. We asked teachers how good they considered
leadership in the school. We could then relate that to the attitudes
of pupils in terms of whether they liked the school and were committed.
We found a significant relationship between the two things. Although
what I said earlier about schools achieving differently in different
subjects might be expected to have some effect on the quality
of the school overall, it cannot be the holy grail. It cannot
be the only thing that drives performance forward. Schools do
differently in different subjects, so something must be going
on within individual subjects. There is certainly evidence that
the quality of leadership has a significant impact.
Simon Rutt: A lot of the statistical
analysis that we carry out on the national pupil database allows
us to explain a certain amount of the variation between schools
and within schools and what is actually happening by using pupil
effects and school characteristics. One thing that individual
research has attempted to get at, but what we rarely have at national
level, is parental involvement in education. Being able to put
that into some of the models that we undertake would be powerful
and would allow us to look with more variation at pupils to find
our their parental involvement and home environment. At the moment,
we have very few socio-economic indicators, but free school meals
is not an indicator of how committed a parent is to their child's
education. That would be another powerful piece of information
to put in, because it would help to explain variations between
pupils in schools with regard to parental factors and things outside
a school's control.
Q31 Chairman: Surely, there must
be a body of research that has looked at parental influence.
Simon Rutt: I am sure that research
projects have looked at it, although I do not know of them as
such, but it would be a very powerful piece of information to
have.
Q32 Chairman: Is that something that
you would like to have in order to further your research?
Simon Rutt: Absolutely.
Q33 Lynda Waltho: I want to drill
down to what you think is missing with regard to statistics and
information. How useful is the pupil level annual school census
in the work that you are doing, what gaps are there and what else
do you feel that it would be helpful to have information on?
Simon Rutt: That is an extremely
useful dataset. I think that it has improved the analysis of educational
research, and having that pupil information at a national level
has allowed much more robust and sophisticated analysis. With
regard to the information in it, such as information on behaviour,
attendance and exclusions, that will increase the information
and power of the dataset when it becomes fully incorporated into
the national pupil database. Attendance has just started to be
gathered at pupil level. I worked on the Excellence in Cities
evaluation, for which we collected information on pupil attitudes
and attendance, and I think that there are lots of things in the
research that could be used. On a number of occasions, it became
evident that schools were actually having an impact on some of
the other measures, despite not impacting on attainment straight
away. It takes a little bit of time for a change in culture and
ethos within a school to have an impact on attainment, but it
might have a more immediate impact on behaviour, attendance and
attitude to school. Change the attitude to school first so that
children want to come to school and learn and turn up enthused
by education, and then the attainment will change. In the Excellence
in Cities evaluation, we tended to find that some of those things
were having an effect first and that some of the behaviour was
changing, which would eventually, hopefully, lead to changes in
attainment. Therefore, when the attainment things come through,
along with fixed-term exclusions, which have been difficult to
get on there, that will also make the information powerful, along
with looking at behaviour in the school.
Dr Gibbons: I support those requests
entirely and think that the information on behaviour and attendance
is important. I will go back to what Simon said earlier about
information on admissions and on which schools pupils put down
as their second and third choices. That information would be really
valuable for understanding what really drives the selection processes
and makes different kids go to different schools. As it stands,
we only know which school a child ends up at, but not which school
he or she would have preferred to go to. We cannot really work
out whether the selection takes place on the parents' side or
the school's side, so those two things together are the most important
things that I would like to see.
Dr Benton: I agree with everything
that has been said. One thing that is also to be said is that
a lot of data have already been collected within schools—we
have talked a little about the attitudes data that we already
have—for one purpose or one evaluation. That information
could be used in secondary analysis of the data that already exist,
and it could be reanalysed for a new purpose, such as looking
at school type, selective schools and so on. There is a lot of
potential, therefore, for further analysis of the data that already
exist and for looking at some of the questions that we are considering.
Simon Rutt: One addition would
be English as an additional language. We used to collect information
on fluency in English at various stages of fluency up to being
bilingual. A lot of very powerful analysis was done because bilingual
pupils tended to achieve higher than native English speakers,
and those who are new to the country and have low levels of English
tend to struggle with the curriculum and underperform. The national
pupil database, at the moment, only collects information on whether
those pupils have English as an additional language. In running
analysis, an effect tends to come out that you know is not the
same for all pupils with English as an additional language. So,
if we could get fluency levels back on to the national computer
database, it would be a powerful resource, particularly for urban
areas where there are many refugees and asylum seekers.
Q34 Lynda Waltho: That was what I
was going to ask about. Language is quite a big issue. I am the
daughter of a school secretary and I can just imagine what my
mother might think now, after listening to all the extra things
that are going to be required. I know that school secretaries
take on much of the burden, so, sorry mum. That is great, thank
you very much.
Dr Gibbons: Another study, the
Longitudinal Study of Young People in England is not about
the population. It is not the pupil level annual school census,
or PLASC, but it is very useful. I would put in a word asking
for that to be continued and extended because it contains a lot
of the more detailed parental background information. It has detail
on parental involvement and attitudes to school. At the moment
it only follows one cohort year by year. I do not know what the
plans are with regard to extending it, but it would be useful
to see it followed up for different cohorts, and perhaps also
to see its scope extended, as it is a valuable data source for
answering these questions.
Q35 Mr Slaughter: The Government's
contention is that academies are either replacing schools or being
placed in low-achieving, often socially deprived areas, with the
idea of making a significant change in the format. They also contend
that, at least at national level, the percentage of children having
free school meals in faith schools is not much different from
in community schools; it is only slightly higher. I know that
because I heard Lord Adonis say it on the Today programme
this morning. Therefore, I had a quick look at one of my local
education authorities to see if that fact was borne out. Looking
at your CV, Mr. Rutt, it is an authority with which you will be
familiar. In brief, the percentage of free school meals in four
community schools was 56, 50, 42, and 41; it was 21 in one C of
E academy; and it was 6, 6, and 2 at three voluntary aided schools.
That is not, I would submit, a minor difference. It is an extraordinary
difference. It does not necessarily equate to a system of comprehensive
education, as I would understand it. My question to you all is:
how do you get to such an extreme system of stratification? That
may be more extreme than other LAs; I do not know. If it is in
any way representative, it is clearly more extreme in relation
to faith schools, which are an established part of the school
family, rather than academies, although, significantly, academies
seem to be in there as well. First, is it a problem? Is it something
that we should not have ended up with? If it is, is it the local
education authorities, politicians, parents, or the schools themselves
that lead to that degree of difference?
Simon Rutt: Given that I know
the local authority to which you are referring, I think that the
difference between the schools within that area has come about
because of the parental ethos and the culture in some of the schools.
Many people have not applied to go to those schools because of
pre-conceived perceptions of what the school is about, what it
is like, and how it will be for their children. I know that applications
to some of those schools are extremely high and that they do take
a balanced intake of those who apply; they split the performance
of those pupils into groups and their lowest performing groups
perform higher than the highest performing groups in other schools.
Is it the school's fault that such pupils apply, or is it the
local authority's fault for not ensuring that a broader range
of people apply? Hopefully, with the changes in admission policies,
a broader range of people will apply for those schools. That situation
has evolved over the years to become how it is. Parental perceptions
and the cultural ethos have allowed that to develop.
Chairman: Dr Benton is looking unhappy.
Dr Benton: No, I am not.
Q36 Mr Slaughter: I find the last
point difficult to accept. Are you saying that there is self-selection
in terms of parental applications?
Simon Rutt: I believe that there
is a degree self-selection for the schools that we are talking
about.
Q37 Mr Slaughter: Another aspect
is that even though voluntary aided schools make up half of the
schools in the LEA area, only 5% of children from the LEA area
go to those schools. Clearly, their catchment area must be wider
because a much higher percentage of pupils are going to community
schools. Is that a general feature of academies or faith schools?
Simon Rutt: Looking at voluntary
aided schools nationally and at a local level, they tend to have
a much wider dispersal of pupils. On average, they will come from
more communities than those in community schools. In London, they
come from even wider areas. There is a difference between inner
London and outer London and other urban areas. In inner London,
pupils will come from many more communities than that which the
school is in. That area tends to be wider for voluntary aided
schools than for community schools.
Q38 Mr Slaughter: What about academies?
Academies can be selective for 10% of their intake. That may or
may not be significant. Does the ethos of an academy, by having
an element of selection, a relationship with a sponsor or other
factors, have the same effect of discouraging applications from
a wider cross-section of parents?
Simon Rutt: I was reminding myself
about academies. Given that there were only a few academies in
the database that I was looking at, pupils came from a similar
sort of proportion of areas as in community schools. Many pupils
are from other communities, but not as many as in voluntary aided
schools. The intake was from smaller areas around academies; a
little bigger than community schools, but not as big as voluntary
aided schools.
Q39 Mr Slaughter: That does not answer
my question. Your answers slightly surprise me. I do not know
what the answer to my question is, but I would be surprised if
you were correct. What you effectively seem to be saying is that
we have a comprehensive system of education and parents choose
to turn that into a selective system for their own reasons. If
that is the case, I am asking whether it is likely to apply to
academies as well as to voluntary aided schools.
Simon Rutt: I have not done any
research to identify that, but I would think that as schools get
better, they will have more applications from parents wishing
to send their children to them, so the level of applications will
be higher. Does that lead to schools selecting pupils from that
application list? I have no evidence of that.
Dr Gibbons: I do not have any
evidence that would tell you anything specifically about academies,
but it is self-evident from the relationship between community
schools and house prices that an element of self-selection goes
on. Parents create a selective system out of the comprehensive
system by moving nearer to schools that are seen as good. That
drives up house prices and keeps out lower-income families. There
is a lot of evidence that school policy has a causal relationship
with house prices. That in itself is evidence that this selection
process is going on. Indeed, the distribution of achievement on
intake into community schools, where there is no element of selection,
is evidence that something like that is happening. Part of that
is driven by the geographical location of schools and the kind
of communities in which they are located, but there is a bit on
top of that which will be generated by people selecting themselves
into schools according to the kind of kids that are in there.
3 Note from witness: Saying that they are very
different schools may be overstating things. There is some relationship
between schools overachieving in one subject and overachieving
in another. However, the differences are big enough to reasonably
conclude that results in any particular subject are not particularly
driven by overall school characteristics such as school type. Back
| 
