Examination of Witnesses (Questions 87
- 99)
WEDNESDAY 30 JANUARY 2002
MISS JANE
OWENS AND
MR JOHN
RODGER
Chairman
87. I welcome Jane Owens and John Rodger to
our deliberations. Thank you for agreeing to appear before the
Committee to help us with our interesting inquiry into the Individual
Learning Accounts. Last week we had an interesting session with
two of the key civil servants and with some of the learning providers,
which you may have seen reported. We are particularly interested
in hearing your evidence because you carried out an investigation
into Individual Learning Accounts quite early in their development
and returned to the subject later on. We are here to learn, but
we shall keep our questions reasonably short, as many members
of the Committee want to ask questions, and perhaps you can give
reasonably brief answers. How did the original research come about?
Why did the department want you to carry out the research very
early in the life of the ILAs?
(Mr Rodger) We were approached by the
department to conduct what might be described as a barometer or
a dipstick research approach to look at some of the key activities,
characteristics, perceptions and processes that were going on
in terms of ILA operation. It was not designed as a thorough evaluation
in the broadest sense of the ILAs. It was very much to get a first
view of what was happening which may inform future planning, particularly
the roll-out and targeting which may give a focus to future evaluation
activity that may take place.
88. You are an independent consultancy. I am
delighted to see that it is based in Leeds. A large number of
members of the Committee are from Yorkshire so we are pleased
about that. How dependent on Government DfES contacts are you?
(Mr Rodger) We specialise in education and training.
We have a turnover of £1.5 million of which 10-20 per cent
is probably DfES. We work for a large number of agencies, government
agencies, local authorities and the private sector, so we have
a broad client base.
89. Last week we discovered that the balance
between the Government and the department was to make a new kind
of learning available with few barriers for new learners and at
the same time not having any quality control built into the courses
nor a substantial check on the quality of the learning that was
provided. Did you pick up serious concerns of that kind early
on?
(Mr Rodger) In terms of our evaluation we focussed
on the views of the account holders. We did not go into a lot
of detail, looking at the procedures that the department had established
for quality assurance of the programme. In a sense any quality
assurance aspects that we picked up were aspects of satisfaction
of the account holders. Certainly from our side, of the two investigations
that we carried out—one in February last year and the second
in July—we picked up high levels of satisfaction from the
account holders' point of view. From a wider perspective, we did
not pick up significant concerns from providers. There were one
or two issues that were raised, particularly regarding aspects
of qualifications, but we did not pick up any significant issues
raised by providers. However, the number of providers to whom
we spoke was relatively small. Overall, we were talking to 100
providers out of several thousands. It was a very qualitative
piece of research.
(Miss Owens) The 100 providers were across three of
the home countries. For England 33 or 34 providers were spoken
to. The only concerns that they were raising in terms of qualifications
for courses, was that some providers were either advertising the
fact that people could get £150 discount and not mention
the £25 contribution or in a number of cases in each of the
three countries in which we spoke to providers, people were claiming
an 80 per cent discount on courses that were not eligible for
that. For example, they were not information technology or numeracy
courses.
90. You are both highly qualified. I have look
at your biographies. This would have been a scientific sample.
In your survey you had a pretty good feel as to what was going
on in terms of those 100 providers and also in terms of the satisfaction
of the people taking the courses.
(Miss Owens) In terms of the providers, because of
the low numbers we were asked to speak to, we could not say that
it was a scientific analysis; it was more a toe-in-the-water feel
for any issues that may come out.
91. You say it was not a scientific analysis?
(Mr Rodger) In terms of statistical significance,
the main survey of account holders was statistically significant,
but we were talking about one thousand in each home country. The
results from that are statistically significant. In terms of the
providers, we only talked to 33 providers in England, so that
was qualitative in nature and you could not draw statistical significance
from the results that we produced.
92. What was the Government hoping to get out
of such an exercise? Presumably they set the parameters?
(Mr Rodger) Indeed. It was meant to be indicative
of types of things that may arise, to identify issues and areas
that they may want to investigate further on a more substantive
basis.
93. They received results that everything seemed
to be going very well.
(Mr Rodger) In terms of some of the main findings,
there were issues that with hindsight may raise some concerns.
For example, there were issues that approximately a third of account
holders were not aware that they had Individual Learning Accounts.
Particularly in the second survey, there were issues relating
to the fact that a quarter of people whom we assumed had redeemed
Individual Learning Accounts, we discovered actually had not.
At the time we suggested that that may be due to some data issues
in terms of information given by Capita, who managed the information
system. Our initial reaction was that it was probably a data issue,
or may be a recording issue in terms of the information that they
received. We recommended to the department that it was something
that they would want to investigate; to get underneath it.
Mr Shaw
94. In your key findings you have told us that
84 per cent of people who took up ILAs already possessed a qualification
and that two-fifths possessed a qualification equivalent to NVQ
level 4, and only 16 per cent had no qualifications at all. In
your research were you able to establish the differing levels
of satisfaction between those people with higher qualifications
or some qualifications and those without?
(Miss Owens) Yes. We asked the question about satisfaction
in the first survey that was carried out in the spring and there
were no significant differences in satisfaction between people
with various qualifications and those with none at all. Levels
of satisfaction were extremely high with the learning that was
being provided at that stage.
95. I am pleased to hear that. In the Green
Paper, the Government spoke of there being a universal approach
and a target approach. There is a concern that such a relatively
small group of people were able to access this training. That
is one of the things that we shall consider: whether there should
be more targeting. In reply to the Chairman you said that you
were not looking specifically at the established quality assurance
that the department had put in. What was your understanding of
that established quality assurance?
(Mr Rodger) It was a bit limited. We were brought
in to do a particular task.
96. Was your understanding limited?
(Mr Rodger) Through the information given to us by
the department, it was limited. We had been involved in doing
some developmental work with the TECs when developing this, so
we were aware of the structure that they had produced. We were
certainly aware that a light touch approach had been adopted.
One of the issues through a number of the TEC initiatives was
that the administrative costs to deliver the accounts was very
significant. In some cases it was more than the current value.
We know from that that a balance had to be struck in terms of
getting a minimum cost administrative overhead, but there were
also aspects of quality assurance. We knew that there was not
the same level of quality assurance in place as had been in place
in some development project.
97. That was fairly obvious to you?
(Mr Rodger) Yes.
98. When you were undertaking your research
were any alarm bells ringing in your head? As the Chairman has
said, you are experienced researchers and you are familiar with
this area. Were there any alarm bells ringing at all?
(Mr Rodger) Not really. In terms of what has come
out, the only potential alarm bells were around two things: one
that there was a third of people who were not aware that they
had accounts; and we had a quarter who were understood to have
redeemed, but had not redeemed. In terms of how people heard about
the accounts, in the second survey a small proportion emerged
in terms of door-to-door activity.
99. When you were discussing your findings with
the DfES, did you discuss that particular group of the door-to-door
people not knowing that they actually had an account? Did they
express any concern about that, saying that they were looking
at that or that they were worried about it?
(Miss Owens) At the point when we . . .
| 
