Memorandum submitted by the Skills for
All Research Programme
In this written evidence we report on research
carried out for the Skills for All Research Programme 2001-04
relevant to the inquiry into 14-19 Education. This programme of
work was funded by the Esmée Fairbairn Foundation and carried
out by a team of researchers from the Centre for Economic Performance
(CEP), London School of Economics and the National Foundation
for Educational Research (NFER). Directors of the research were
Dr Hilary Steedman (CEP), Professor Lord Richard Layard (CEP)
and Dr Sheila Stoney (NFER). Additional financial support was
provided by the Anglo-German Foundation and the Economic and Social
Research Council.
The research focused on the 50% of young people
in England who do not go on to university. Skills for All research
examines the routes they take to higher levels of skills and education,
their motivation, the quality of provision and the contribution
of employers and colleges to their success.
1. One strand of the research investigated
the under-achievement of British young people in school and subsequent
marginalisation on the labour market. A series of seminars brought
together experts from a number of disciplines to give papers on
disengagement. The results were summarised and published as "Disengagement
14-16: Context and Evidence" (Steedman and Stoney 2004) http://cep.lse.ac.uk/pubs/download/dp0654.pdf
and seminar papers at http://cep.lse.ac.uk/events/seininars/motivation/default.asp
The main conclusions reported in this paper
were as follows. Disengagement is not new, but changes in the
demand for skills, and the importance of lifelong learning together
with a more inclusive employment and social agenda have made it
a high priority policy area. There is an extensive body of academic
research which documents the effect of negative attitudes to school
on attitudes to further learning. However, estimating the size
of the challenge presented by disengagement and distinguishing
the variety of needs of those in this group is still under-researched.
Nevertheless, we feel that, from our seminar papers and discussions,
some improved understanding has been achieved.
The disengaged who cause immediate concern are
located within the group that achieve less than five GCSE passes
at A*-C grades at 16. A very small group, the "out of touch",
(1-2%) have practically lost touch with school between 14 and
16. A larger group, difficult to estimate, but possibly to be
identified with roughly 20% of the cohort who claim to have no
GCSE qualifications at age 17/18/19 could be characterised as
the "disaffected but in touch" group. This group will
include the truanting group but, for the rest are mostly still
in touch with school. Some may be in danger of failing to reach
basic skills standards necessary for any further education and/or
training—others may be capable of achieving GCSE passes
at Grades A-C. A further group, also difficult to quantify but
perhaps close to the proportion who gain one or more but less
than five Grade C or higher passes at GCSE would account for up
to a further 20%. Within this third group whom we will characterise
as the "1-4 A-C Grade" group, some may have reached
their full potential but others will be capable of much more if
interest and enthusiasm can be aroused.
A vast amount of experimentation has taken place
as a result of initiatives taken by the previous government and,
after 1997, by the current government. Unfortunately, none of
this was planned and implemented in such a way as to provide robust
evidence of the efficacy of any particular type of provision.
Nevertheless, from our survey of evaluation of initiatives targeted
at the disengaged it has become clearer that these client groups,
although difficult to distinguish at the margin, do require different
approaches in order to achieve in basic skills and/or achieve
their potential.
The small "out of touch" group appears
to make some progress in alternative provision which provides
one to one contact, an adult approach and atmosphere and the opportunity
to mark progress through certification. However, success on even
the best of these programmes is limited and OFSTED have expressed
deep concern about variability in quality of provision.
The larger "disaffected but in touch"
group appears to respond to a wide range of initiatives which
often share the characteristic of taking them out of school whether
into a work-related setting (FE college, employers' premises)
or a setting associated with leisure and cultural activities (football
club, theatre). However their response was by way of general animation
rather than the securing of material benefits. Progress was mixed
as were experiences of work placements, etc, there was no "magic
bullet" and, where outcomes were assessed, there was mixed
evidence on improved attainment and progression to further education
and training. If anything, contact with the world outside school
appeared to increase the probability that students would choose
to leave rather than stay.
Ofsted was cautious on this type of intervention,
stressing the need for careful planning and monitoring of work
placements—evaluation had shown that many were not suitable
and some could be quite damaging. With this proviso, and based
on relatively small quantities of pupil-level achievement data,
Ofsted considered that well configured work placements might indeed
contribute to re-engagement and improving performance. The understandable
lack of expertise in teaching basic number and literacy amongst
staff in these "alternative" settings, coupled with
the absence of these pupils for some regular timetabled classes
in these subjects as a result of their attending their "alternative"
activities, must raise concern that some of the necessary foundations
for further education and training could be neglected to the considerable
detriment of the students.
The third client group identified, the "1-4
A-C Grade" has been targeted throughout the 1990s by pilots
and initiatives which focused on changing the mix of subjects
studied 14-16 to include vocational subjects. The thinking behind
these initiatives proposed that vocational subjects would re-engage
students' interests at a period in their development when they
were focusing on future roles in the adult world of work. Vocational
subjects would also be in tune with hobbies and out of school
interests and allow students to demonstrate aptitudes and capabilities
that were not called upon in the more "academic" subjects.
Many difficult challenges have to be overcome
in providing such courses, as is shown by the experience of Barking
& Dagenham LEA. Ofsted expressed concern about the capacity
of schools to offer such courses to the standard required (lack
of teachers with requisite vocational experience/skills) when
evaluating GNVQ in 1999. Nevertheless, starting with the introduction
of GNVQ Part 1 in the mid-1990s and Vocational GCSEs in Barking
& Dagenham, the overall judgement has been that such courses
can have a highly motivating effect on students' performance.
Systematic and robust evaluation of the impact of studying vocational
subjects on student motivation, achievement and progress has been
lacking. It is too early to learn from the IEP programme of evaluation
but it should eventually provide some of the hard evidence that
has been lacking until now.
What are the concerns raised from this overview?
Doubts were expressed about the underlying assumption that work-related
learning [aimed principally at the "disaffected but in touch"
group] is inherently more motivating for young people switched
off by the existing curriculum. Could it be that the real issue
is one that is more to do with pedagogical style, learning preferences
and the provision of adequate support? This hypothesis is worthy
of testing. We learned from the earlier paper on the Swiss experience,
for instance, that less academic youngsters appeared to flourish
more routinely in Swiss schools where the curriculum was broadly-based
for longer, was more nurturing in delivery style and had more
safety nets built into the system
It was argued that nothing that we have heard
in the seminars offers a cast iron way forward for the "disaffected
but in touch" group targeted by work-place learning measures,
but that it is not clear that any other country has a foolproof
method either. The danger is that of conflating these two different
groups [the "disaffected but in touch" group and the
"1-4 Grade C" group].
For the "1-4 Grade C" group, there
may be something of a magic bullet; namely better vocational options,
which many countries offer as sizeable programmes of three years
duration or more, apparently to good effect, leading many to Level
3, whereas considerable numbers of this group in the UK spend
only one year post-compulsory with no advancement on their Level
1 at GCSE.
For the second, the "disaffected but in
touch" it is not clear at present that there is an obvious
solution or what it might be if it exists. The danger that threatens
in the policy field is to apply the solution for the "1-4
Grade C" group (better vocational options) to the "disaffected
but in touch", with the result both that it doesn't work
for them (because no simple recipe will) and that it is discredited
for the "1-4 Grade C" group by association with the
disaffected.
We would be in a better position today to understand
the "disaffected but in touch" group and what Works
for them if a smaller number of initiatives had been introduced
during the period of experimentation, with provision for robust,
scientifically-based programmes of evaluation.
It was found, both in the US and in England,
that while motivation of groups targeted by work-related initiatives
frequently improved, they failed to show consistent improvement
on standard measures of achievement (eg GCSE points scores). Can
motivation/engagement improve and yet not be translated into increased
effort and therefore improved performance? Or has performance
improved but the improvement not been captured (if assessments
not fit for purpose, for example)? Does the range of assessments
used to try to capture improvement need to be broadened?
The OECD PISA study showed even our poor-performers
scoring better than their counterparts in a number of other countries.
Yet it is highly probable that half the English students scoring
at Level 3 on the PISA tests may fail to gain a Grade C GCSE pass
in English (the ONS shows a .8 correlation between PISA score
and GCSE English A*-C so we map directly across from PISA to GCSE).
Are we setting the bar too high too early? What is it about five
GCSEs anyway (the National Targets started out with four)? Other
countries with a more socially equitable distribution of educational
achievement, for example the Scandinavian countries, do not set
a formidable hurdle at 16. Denmark and Sweden have lower average
scores on PISA than the UK (largely because they have fewer very
high achievers at 15 than the UK). However, a far higher proportion
subsequently gain a Level 3 qualification.
Can we move more quickly towards de-coupling
K54 assessment from GCSE and encouraging a variation in length
of preparation for GCSE? Or, for example could we demand the same
standards as now obtain at C+ for the aspects of maths and English
which are vital for future life (for example probability, writing
letters) but be less insistent on all achieving the grade for
things like geometry and English literature? Fewer "failing"
ie not getting five A-C would mean less disaffection and discouragement.
We learnt that vocational qualifications can
improve the employment chances of the group with no or low level
GCSE qualifications (see McIntosh 2004a below). At the moment
we think that too many in this group are "turned off learning
and that means that we need to continue to work at all the challenges
raised by disaffection. But this is not the only problem, as our
poor record relative to other countries in post-16 qualification
demonstrates. Some of the reasons for failure to progress are
to be found in labour market characteristics and lack of transparent
routes. However, from discussion at the seminars it appears that
many of the disengaged lack information about post-16 qualifications,
training and subsequent labour market opportunities.
Careers guidance and information on training
and education routes to chosen occupations appear to still be
a major weakness. There was general consensus that the Connexions
service is having difficulty in meeting both mainstream and more
specialist needs of young people. It was not clear what the considerable
resources devoted to Connexions were "buying". There
was a strong feeling that the Connexions service is trying to
do too much and not meeting any need fully. These complaints have
been made—particularly in regard to information on further
education and training opportunities supplied by schools—for
at least the last 20 years. What do we need to do to make more
progress on this front?
Historical evidence (technical secondary schools,
TVEI) show vocational/technical subjects and programmes as being
particularly vulnerable to pressures to academicize content, to
problems of quality and the absence of natural champions ie employers
and trade unions. The papers which touched on Vocational GCSE
(Barking & Dagenham) and the OFSTED Final Report on GNVQ Part
1 pilots also hinted at the fragility of the supporting infrastructure
and problems of maintaining high quality learning experiences.
They also stressed the importance of ensuring that teachers have
appropriate vocational experience and skills and the opportunity
to make links with employers in relevant occupational areas. What
safeguards are in place to ensure that the GCSE in vocational
subjects initiative is supported by high quality appropriate course
content and taught by appropriately qualified teachers? How is
the need for equipment and resources to provide for the practical
elements in GCSE in vocational subjects being met? Can we be sure
that this technical/vocational initiative will finally receive
the sustained support necessary for survival?
2. In a study of labour market outcomes
for low achievers using Labour Force Survey data (McIntosh 2004a)
http://cep.lse.ac.uk/pubs/download/dp0621.pdf it was found that
relatively few of those who report no or low school qualifications
at age 17 subsequently gain a vocational qualification at Level
2 or Level 3. The paper goes on to suggest that those individuals
who left school with no qualifications but subsequently acquire
vocational qualifications at levels 2 or 3 have employment rates
quite similar to those who reached these levels via academic qualifications
at school (ie GCSEs and A levels respectively). The problem is
that few of the unqualified school leavers acquire these vocational
qualifications. Even by 2002, when the cohort studied are aged
23-25, approaching half of the poorly qualified school leavers
have not acquired any further qualifications (42% of males and
44% of females), with a further 26% of males and 27% of females
only reaching level 1 via the vocational route. Therefore, less
than one quarter of the poorly qualified school leavers actually
reach level 2 through vocational qualifications, with only 15%
of males and 13% of females reaching level 3. It would appear
therefore that vocational qualifications offer a second chance
to labour market success for those who leave school with no qualifications,
but at present too few such people are taking this chance. There
are some technical caveats attached to this study which arise
from the limitations of the data available. In short, we cannot
be sure that the observed effects do not arise from some unobserved
characteristics of the group in question bearing in mind that
they are in some sense a self-selected group. Nevertheless, these
data are the best we have at present and it seems unlikely that
there would be no or negative effect of gaining a vocational qualification,
only that the effect might not be as large as observed here if
all unobservables could be controlled for.
3. Since we found that vocational qualifications
may make a substantial difference to labour market outcomes for
young people with no or low school achievements, our research
for this programme also analyses the relative strengths and weaknesses
of full and part-time 16-19 provision for youth with below average
GCSEs (Foreman and McIntosh (2004 forthcoming)
http://cep.lse.ac.uk/research/skills/Skill-Publications/McIntosh-Foreman-2004.pdf
In this paper, which analyses data from the
Youth Cohort Study, we find that the work-based employment route
seems to have a lower success rate than full-time education or
government-sponsored training, though the difference just fails
to achieve statistical significance.
We then examine the factors associated with
whether individuals continue with their vocational education.
We find that achievement of the original qualification goal, vocational
study at a higher level and study in a full-time college of Further
Education are all associated with an increased likelihood of low
GCSE-achievers (no GCSE passes at A-C) continuing with their training,
but a reduced likelihood of high GCSE-achievers doing so. These
results, together with the fact that, overall, high GCSE-achievers
are less likely to remain in vocational education from one sweep
of the survey to the next, suggest that such individuals appear
to simply drop into Further Education to acquire a specific qualification,
before exiting the system again. Given that their school qualifications
already qualify them to level 2, and they may have opportunities
to acquire higher level qualifications elsewhere via the academic
route, we need not be too concerned about this group in Further
Education.
Attention should be focussed, therefore, on
the low school qualification-achievers, for whom the results suggest
that they can progress through the system, given appropriate conditions,
which appear to be in a full-time college setting, and with frequent
qualification attainments and time extensions where necessary,
along the way. Policy prescriptions therefore involve getting
more such young people involved in vocational training, and also
investigating why the work-based route seems to serve this group
in particular rather poorly. Such policies may then allow some
progress to be made on closing the gap in intermediate vocational
qualification achievement rates, between the UK and some of its
competitor countries.
4. Much of the Skills for All research focuses
on the potential of apprenticeship and other work-based learning.
We ask:
— How well does employer-supported
group training perform? We conclude that employer involvement
in the organisation of apprentice training can benefit small firms
in particular and increase the probability of relevant and appropriate
work placement (Gospel and Foreman 2002) http://cep.lse.ac.uk/pubs/download/dp0555
.pdf
— Does apprenticeship pay (rates of
return)? We conclude that, for men at least, an apprenticeship
together with an NVQ3 qualification offers a substantial return
(14%) although the same caveats attached to McIntosh 2004a apply.
(McIntosh 2004b) http://cep.lse.ac.uk/pubs/download/dp0622.pdf
— Does regulation improve work-based
training? We conclude that, in the case of the care sector, much
of the training is genuinely new skilling and that deadweight
effects are small. There are also gains in staff morale and greater
interest among staff in progression to the next level (Gospel
and Thompson 2003) http://cep.lse.ac.uk/research/skills/Skills-Publications/Gospel-Thompson-
May—2003.pdf
— How should apprentices and employers
under stress be supported? This study recommends producing good
practical guides to support the implementation of MAs in the sector
studied (retail). One would set out for training providers "what
works" in terms of support and intervention and the other
would provide similar guidance for employers. (Spielhofer and
Sims 2004) http://www.nfer.ac.uk/research/downloads/SKAJan04.doc
We also examine British apprenticeship in the
light of European best practice and find the absence of a statutory
framework for apprenticeship in the UK a particular obstacle to
achieving consistently good quality provision (Steedman 2001)
http://cep.lse.ac.uk/pubs/down/dp0513.pdf
We discover how companies in Britain and Germany
procure specialist ICT skills—German companies are currently
training some 60,000 ICT apprentices while in the UK the number
is less than 2,000 (Steedman and Wagner 2003) http://cep.lse.ac.uk/pubs/download/dp0575.pdf
Finally, we compare British performance in raising
qualification levels over time and across countries by carrying
out a third Skills Audit, benchmarking British progress in increasing
stocks of skills against that of other countries—France,
Germany, Singapore and the US. (Steedman, McIntosh and Green 2004)
http://www.dfes.gov.uk/research/data/uploadfiles/RR548.pdf
The main conclusions of this study, which was
commissioned by the Department for Education and Science and the
Department for Trade and Industry were that:
— At Level 2 and above proportions
of UK 19-21 year olds are higher than in Germany and the US. However,
Singapore and France have much higher proportions at Level 2 and
above.
— At ages 19-21, the UK also compares
well with all other countries at Level 3 and above. However, at
age 25-28 at Level 3 and above the UK lags behind France and Germany
— In the two continental European countries,
vocational qualifications continue to play an important role in
enabling more young people to reach Level 2 and Level 3 and above
by age 25-28.
— In the UK proportions of 25-28 year
olds with higher education (HE) qualifications are similar to
the US, only slightly behind France but 10 points below Singapore.
Germany has less than half the UK proportion of 25-28 year olds
with HE qualifications.
— In France, Germany and Singapore
substantial proportions of HE qualifications are vocational/applied.
In the UK and US proportions with short vocational/applied diplomas/degrees
are much smaller.
— The UK has the highest growth rate
of qualifications at Level 2 and above over the period 1994-2003.
— At Level 3 and above 1994-2003, UK
growth compares well with the other four countries. Only Singapore
has a significantly higher growth rate than the UK for the 25-28
age group.
— For 19-21 year olds at Level 2 and
above in the UK, the gap with France and Singapore has been roughly
halved since 1994.
— At Level 3 and above for the 19-21
age group Germany had an advantage of 14 percentage points relative
to the UK in 1994 and the gap with the US was of a similar magnitude.
The gap with respect to Singapore was slightly smaller. These
gaps have now disappeared.
— For 25-28 year olds at Level 3 and
above, not only the UK but also France and Singapore have experienced
rapid growth with the result that the UK is just about "keeping
pace" with those countries rather than closing the gap.
— Qualification levels in the UK increase
much more slowly after ages 19-21 than in France and Germany.
In these countries qualifications at Level 3 and above increase
substantially between ages 19-21 and 25-28.
POLICY IMPLICATIONS
International studies show that the UK does
not have more disengagement than other OECD countries but the
association with socio-economic deprivation is stronger in the
UK than in any other OECD country. The strength of this association
makes it particularly difficult to overcome social inequalities
in educational outcomes and emphasises the need to tackle disengagement.
It should not be assumed that work placements and vocational courses
are the answer to disengagement among all those affected since
existing evidence is unclear on this point. Evidence does suggest
that vocational options may help to raise the attainments of those
who currently gain between one and four GCSE passes at grades
A*-C. For other groups alternative teaching and leaming styles
and content may be equally effective. The puzzle of good UK scores
of 15 year olds on the PISA tests and subsequent low levels of
qualification at Level 3 relative to other countries may be explained
by the demotivating effect of GCSE "failure" on those
students who do not achieve passes at Grades A*-C. Less emphasis
on high-profile summative assessment at age 16 could also help
to increase achievement post-16.
Evidence from other countries continues to suggest
that the British shortfall in proportions of young people gaining
qualifications at Levels 2 and 3 could be improved by the availability
of clearer and more transparent progression on vocationally-oriented
study routes. We urgently need to understand how best to promote
success on vocational routes for young people with GCSE at grades
A*-C. Our research suggests that full-time provision may be more
effective than work-based provision in helping young people to
gain a recognised qualification. In particular, a vocational qualification
at Level 3 appears to greatly improve employment prospects for
this group of young people (GCSE at grades A*-C). Yet, relatively
few among them gain such a qualification. This is particularly
true of young women in this group who have a very high probability
of inactivity and lack of labour market experience.
The potential of apprenticeship in the UK is
under-developed and under-used as a route to skill for young people
when compared to other countries. Training providers with strong
links to groups of employers could be encouraged and supported
to provide high quality work experience. Government regulation
can also encourage investment in skills training by employers.
The returns to apprenticeship combined with a Level 3 qualification
suggest that this is an investment well worth making.
November 2004
| 
