Examination of Witnesses (Questions 100
WEDNESDAY 30 JANUARY 2002
100. When was that?
(Miss Owens) When we presented the findings to them.
The number that we found who had heard about ILAs from someone
coming to their door, that was the first time that we heard about
that. That did not appear in the first survey. It was 5 per cent
of 218 people. We were looking at the early tens. From talking
about it, we assumed that it could be learning providers marketing
in a more pro-active fashion which with hindsight it appears to
101. Can you clarify the approach in relation
to the establishment of the two parts of research? You carried
out the first part and then there was a follow up part. How did
that happen and what was the differing focus?
(Miss Owens) First, the department wanted "an
early assessment of how well Individual Learning Accounts were
operating for individuals and providers". In a sense, it
was satisfaction, attitude and views. When the invitation to tender
first came out at the end of 2000, when we were bidding for it
along with others, it was stated that there would be further stages
of the evaluation. The survey carried out in spring 2001 was to
be the first stage. In July, just as we were finishing the first
report, and it was being tidied up for publication, the department
came back to us and said that they wanted "rapid limited
evidence on how early account holders"the people who
had opened their accounts early on in the national framework"and
later ones"people who had opened their accounts from
1 May onwards"to compare and contrast" to see
if the profile of people was similar and whether there were any
differences coming out. We understood that that would help to
inform the second year of evaluation to pinpoint any areas that
the department may want to look at.
102. Did you have any discussion about that
second piece of research and the reasons for that? There was one
piece of research, and you say that you were still pulling it
together and that it had not been published.
(Miss Owens) It had been presented to the department
in May so they were aware of the findings.
103. The brief for the second piece of research
came out of what you were finding on the first?
(Miss Owens) Yes, they came back to us and said that
they wanted some follow up work done to see whether the profile
of account holders had changed. Once more publicity had gone out
and more people heard about it; they wanted to see, for instance,
whether the number of people without qualifications may change.
104. Did you feel that the second piece of research
was picking up on the first piece or the findings from that?
(Miss Owens) Yes, to some extent. In the first piece
something came out in qualitative interviews with providers about
understanding that an ILA is for life. At that stage, it was not
just a one-off payment. We were interested to look into people's
understanding of what an ILA was. Did they understand that they
could use it again, in future years, and did they understand that
they could use it with different providers. There was also a steer
from the department as to what they wanted.
105. Did you suggest things to them, in terms
of the problems that you had identified, such as the quarter of
account holdersI think that was the redeeming point?
(Miss Owens) That came out in the second survey. That
did not appear in the first one. That was an issue in terms of
the sample that Capita, who ran the ILA centre, had provided to
us through the department, from which we could select people for
telephone interviews. The department had asked Capita to provide
us with approximately 1500 records of people who had opened and
used their accounts. When the 300 telephone calls took place it
was discovered that about 27 per cent of them said that they had
not used their account, but our understanding from Capita was
that they had used it.
106. Were there any issues of concern that you
identified in your first piece of research that were not picked
up for further follow-up work?
(Mr Rodger) Not really. One of the key issues in the
first piece was around the area of confusion. A number of individuals
were not quite sure what the entitlement was under the account.
We probed that further in the second piece of research and that
trend continued. They were not clear what their entitlement was
and whether they would receive any discount, £150 or whatever.
We assume that in many respects that was related to the way in
which they opened their accounts. The way in which it is meant
to work is that an individual opens an account and is therefore
empowered to purchase learning on an informed basis and he/she
shops around. Clearly, some did that but it is also quite clear
that others were perhaps introduced into a particular kind of
learning by a supplier and in many cases did not realise that
the account was part of that. They were simply entering into a
course and the supplier had introduced the discount aspect which
was the account, of which in some cases the individual would not
be aware. That was an assumption that we made from the evidence
presented. If you look at how people found out about accounts,
you are looking at almost half who heard about the account through
a supplier. If you look at the guidance element, three-quarters
of them did not take up any guidance of any sort and of those
who did again over half had guidance from a supplier. There was
a clear strong supplier push.
107. When was the second piece of research presented
to the department and when was it published?
(Miss Owens) The field work was done in the second
half of August and the beginning of September and the findings
went to the department in the middle of September.
108. When was it published?
(Miss Owens) The research brief was due to be published
in October, but I note that it was published on 14 January this
109. What were your feelings then? Was this
a client who had asked for a piece of research, a worried client
who said, "My God, something is happening and we want you
to do a follow-up"? Did you get that feeling of urgency?
(Miss Owens) I did not get that about the second survey,
no. They were just interested to find out about the profile of
the account holders and to do some follow-up work on issues that
had come out of the research.
110. You were not at a level of communication
with the department where you picked up the vibe that something
was going horribly wrong and that they were trying find out what
(Mr Rodger) No.
111. I want to clarify exactly what your remit
was from the Government. Putting the first and second pieces of
research to one side, how scientific an evaluation were you asked
to conduct? As I understand it, you carried out a UK-wide survey
of something like 4,700 account holders across the four home countries.
What was your remit from the Government? Was it a proper scientific
evaluation, in your view, or not?
(Mr Rodger) Certainly in terms of the account holders,
it was a scientific evaluation of perceptions and characteristics.
As I said at the beginning, sometimes when we do these sorts of
things we are asked to design an evaluation of a programme or
an initiative which is very holistic. In this case, we were brought
in to do a particular task, which may be one element of an overall
evaluation, and that was to look at the characteristics and perceptions
of account holders. That was the main scientific element. In addition
we undertook a qualitative review of provider perceptions. That
was our remit.
(Miss Owens) It was specified in the invitation to
tender that the total sample of account holders across the UK
should not exceed 4,500. In terms of the providers that stage
should be qualitative.
112. In your view, were the Government asking
you to look into the right question? Were they asking the right
things in order to evaluate whether the initiative of ILAs was
going to succeed and whether it was hitting the right targets?
Were the Government giving you wide enough parameters to allow
you, in your professional approach, to explore the issue further?
(Mr Rodger) I think so. If you look at some of the
headline findings from our evaluation, these are the key things
that they want to know about. It was looking at the number of
new learners who are coming in, looking at the characteristics,
looking at new aspects of guidance, looking at employer take-up,
looking at the proportion who may have done this anyway. All of
those are key evaluation matters that one would want to know about.
Perhaps in terms of the provider aspect of things, our remit was
not strong. If we look back on it I suppose you may say that perhaps
a more substantive piece of work should have been carried out
to look at the provider perspective of this. The numbers that
we looked at were perhaps not robust enough to give a firm conclusion.
(Miss Owens) In addition to the work that we were
commissioned to conduct, as far as I am aware two other surveys
were carried out in spring last year that the department commissioned
in relation to providers. So there was other work going on.
113. So you were looking at those who were taking
part, specifically at account holders, rather than looking at
it from the providers' point of view?
(Mr Rodger) Yes.
Mr Baron: You never had the opportunity or inclination
to look at whether strong enough structures were in place to stop
Chairman: To be fair to our witnesses, they
would have been given a remit by the department. Their inclination
would not have come into it.
114. I just wanted clarification.
(Miss Owens) We were asked to look at the programme
only in terms of people's perceptions of the process, what account
holders' perceptions of the process were, for instance, of the
Individual Learning Accounts Centre. Also they wanted to know
the perceptions of the providers to whom we spoke, but we were
not asked to investigate the process.
115. On the first UK-wide study that you carried
out in the four home countries, did you find any variation in
the results from the four home countries, and if so, what were
(Miss Owens) There were some variations. In many ways
the findings were remarkably similar across the four countries.
There were some differences in terms of gender, in the type of
learning that some people were taking, with, for instance, a much
lower number of people in Scotland participating in further education
than was the case in England and Wales. Satisfaction was similar.
How they first heard about Individual Learning Accounts was reasonably
similar. Sources of guidance were not quite as high in Scotland
and Wales, from learning providers, as they were in England and
Northern Ireland. But very few things actually stood out. In some
cases the proportion of people who said that they could have paid
for their courses without having an Individual Learning Account
was higher in two other home countries than in England.
116. I propose we move on now to the findings.
What strikes me from reading the material that you have supplied
is that there was a delay in publishing from October to 14 January
2002. What is the explanation for that?
(Mr Rodger) That is not untypical. In terms of the
results being made to the department and the time that it takes
for that to get into the public domain, I would say that that
is quite normal.
117. Is it quite normal in terms of what must
have been going on in the department at that time?
(Mr Rodger) We have no knowledge of that. If I did
not know what I know now and I had seen the report coming out,
I would have thought that that would be normal procedure.
118. I have two questions on the findings. Previously
we have heard from training providers that by the spring of 2001
they had said to the Government that they thought that the scheme
was flawed and that it was open to abuse. You are saying in both
of your surveys that you carried out in spring 2001 and in late
summer/early autumn 2001 that the Government never asked you to
look at that aspect of it at all?
(Miss Owens) We were not asked to look at providers
in the second survey. It was purely people with accounts in the
second survey. The only time that we spoke to providers was in
the first one and it was an early perceptions survey that they
wanted us to do. They did not ask us to focus on specific issues.
They had not raised the issue of there being problems. They asked
us to look at things like the type of learners that the providers
were seeing coming through, whether ILAs had had an impact on
the course costs and whether employers were contributing. They
also wanted us to tease out whether there were any other issues.
The providers quite willingly expressed any issues or concerns
that they had, but none of them raised the issues that have come
up more recently. The only issues that they raised, for instance,
was the slow website at times when they were entering details
and the fact that they felt that some providers were not marketing
Individual Learning Accounts in a proper manner, or were claiming
the 80 per cent discount for ineligible courses.
119. I have two questions on the findings from
your surveys. The ILAs were hopefully designed to draw people
into training, or back into training, who were not normally in
that area. Nationally about 16.5 per cent of the population have
no qualifications or training. One of the findings of your study
was that only about 16 per cent of people to whom you spoke came
from that background of no training at all. Is that correct?
(Miss Owens) That is correct. It went up to 18 per
cent in the second survey.