Select Committee on Education and Skills Minutes of Evidence

Examination of Witnesses (Questions 100 - 119)



  100. When was that?
  (Miss Owens) When we presented the findings to them. The number that we found who had heard about ILAs from someone coming to their door, that was the first time that we heard about that. That did not appear in the first survey. It was 5 per cent of 218 people. We were looking at the early tens. From talking about it, we assumed that it could be learning providers marketing in a more pro-active fashion which with hindsight it appears to be.

Ms Munn

  101. Can you clarify the approach in relation to the establishment of the two parts of research? You carried out the first part and then there was a follow up part. How did that happen and what was the differing focus?
  (Miss Owens) First, the department wanted "an early assessment of how well Individual Learning Accounts were operating for individuals and providers". In a sense, it was satisfaction, attitude and views. When the invitation to tender first came out at the end of 2000, when we were bidding for it along with others, it was stated that there would be further stages of the evaluation. The survey carried out in spring 2001 was to be the first stage. In July, just as we were finishing the first report, and it was being tidied up for publication, the department came back to us and said that they wanted "rapid limited evidence on how early account holders"—the people who had opened their accounts early on in the national framework—"and later ones"—people who had opened their accounts from 1 May onwards—"to compare and contrast" to see if the profile of people was similar and whether there were any differences coming out. We understood that that would help to inform the second year of evaluation to pinpoint any areas that the department may want to look at.

  102. Did you have any discussion about that second piece of research and the reasons for that? There was one piece of research, and you say that you were still pulling it together and that it had not been published.
  (Miss Owens) It had been presented to the department in May so they were aware of the findings.

  103. The brief for the second piece of research came out of what you were finding on the first?
  (Miss Owens) Yes, they came back to us and said that they wanted some follow up work done to see whether the profile of account holders had changed. Once more publicity had gone out and more people heard about it; they wanted to see, for instance, whether the number of people without qualifications may change.

  104. Did you feel that the second piece of research was picking up on the first piece or the findings from that?
  (Miss Owens) Yes, to some extent. In the first piece something came out in qualitative interviews with providers about understanding that an ILA is for life. At that stage, it was not just a one-off payment. We were interested to look into people's understanding of what an ILA was. Did they understand that they could use it again, in future years, and did they understand that they could use it with different providers. There was also a steer from the department as to what they wanted.

  105. Did you suggest things to them, in terms of the problems that you had identified, such as the quarter of account holders—I think that was the redeeming point?
  (Miss Owens) That came out in the second survey. That did not appear in the first one. That was an issue in terms of the sample that Capita, who ran the ILA centre, had provided to us through the department, from which we could select people for telephone interviews. The department had asked Capita to provide us with approximately 1500 records of people who had opened and used their accounts. When the 300 telephone calls took place it was discovered that about 27 per cent of them said that they had not used their account, but our understanding from Capita was that they had used it.

  106. Were there any issues of concern that you identified in your first piece of research that were not picked up for further follow-up work?
  (Mr Rodger) Not really. One of the key issues in the first piece was around the area of confusion. A number of individuals were not quite sure what the entitlement was under the account. We probed that further in the second piece of research and that trend continued. They were not clear what their entitlement was and whether they would receive any discount, £150 or whatever. We assume that in many respects that was related to the way in which they opened their accounts. The way in which it is meant to work is that an individual opens an account and is therefore empowered to purchase learning on an informed basis and he/she shops around. Clearly, some did that but it is also quite clear that others were perhaps introduced into a particular kind of learning by a supplier and in many cases did not realise that the account was part of that. They were simply entering into a course and the supplier had introduced the discount aspect which was the account, of which in some cases the individual would not be aware. That was an assumption that we made from the evidence presented. If you look at how people found out about accounts, you are looking at almost half who heard about the account through a supplier. If you look at the guidance element, three-quarters of them did not take up any guidance of any sort and of those who did again over half had guidance from a supplier. There was a clear strong supplier push.

  107. When was the second piece of research presented to the department and when was it published?
  (Miss Owens) The field work was done in the second half of August and the beginning of September and the findings went to the department in the middle of September.

  108. When was it published?
  (Miss Owens) The research brief was due to be published in October, but I note that it was published on 14 January this year.


  109. What were your feelings then? Was this a client who had asked for a piece of research, a worried client who said, "My God, something is happening and we want you to do a follow-up"? Did you get that feeling of urgency?
  (Miss Owens) I did not get that about the second survey, no. They were just interested to find out about the profile of the account holders and to do some follow-up work on issues that had come out of the research.

  110. You were not at a level of communication with the department where you picked up the vibe that something was going horribly wrong and that they were trying find out what it was?
  (Mr Rodger) No.

Mr Baron

  111. I want to clarify exactly what your remit was from the Government. Putting the first and second pieces of research to one side, how scientific an evaluation were you asked to conduct? As I understand it, you carried out a UK-wide survey of something like 4,700 account holders across the four home countries. What was your remit from the Government? Was it a proper scientific evaluation, in your view, or not?
  (Mr Rodger) Certainly in terms of the account holders, it was a scientific evaluation of perceptions and characteristics. As I said at the beginning, sometimes when we do these sorts of things we are asked to design an evaluation of a programme or an initiative which is very holistic. In this case, we were brought in to do a particular task, which may be one element of an overall evaluation, and that was to look at the characteristics and perceptions of account holders. That was the main scientific element. In addition we undertook a qualitative review of provider perceptions. That was our remit.
  (Miss Owens) It was specified in the invitation to tender that the total sample of account holders across the UK should not exceed 4,500. In terms of the providers that stage should be qualitative.

  112. In your view, were the Government asking you to look into the right question? Were they asking the right things in order to evaluate whether the initiative of ILAs was going to succeed and whether it was hitting the right targets? Were the Government giving you wide enough parameters to allow you, in your professional approach, to explore the issue further?
  (Mr Rodger) I think so. If you look at some of the headline findings from our evaluation, these are the key things that they want to know about. It was looking at the number of new learners who are coming in, looking at the characteristics, looking at new aspects of guidance, looking at employer take-up, looking at the proportion who may have done this anyway. All of those are key evaluation matters that one would want to know about. Perhaps in terms of the provider aspect of things, our remit was not strong. If we look back on it I suppose you may say that perhaps a more substantive piece of work should have been carried out to look at the provider perspective of this. The numbers that we looked at were perhaps not robust enough to give a firm conclusion.
  (Miss Owens) In addition to the work that we were commissioned to conduct, as far as I am aware two other surveys were carried out in spring last year that the department commissioned in relation to providers. So there was other work going on.

  113. So you were looking at those who were taking part, specifically at account holders, rather than looking at it from the providers' point of view?
  (Mr Rodger) Yes.

  Mr Baron: You never had the opportunity or inclination to look at whether strong enough structures were in place to stop fraud?

  Chairman: To be fair to our witnesses, they would have been given a remit by the department. Their inclination would not have come into it.

Mr Baron

  114. I just wanted clarification.
  (Miss Owens) We were asked to look at the programme only in terms of people's perceptions of the process, what account holders' perceptions of the process were, for instance, of the Individual Learning Accounts Centre. Also they wanted to know the perceptions of the providers to whom we spoke, but we were not asked to investigate the process.

Jeff Ennis

  115. On the first UK-wide study that you carried out in the four home countries, did you find any variation in the results from the four home countries, and if so, what were the specifics?
  (Miss Owens) There were some variations. In many ways the findings were remarkably similar across the four countries. There were some differences in terms of gender, in the type of learning that some people were taking, with, for instance, a much lower number of people in Scotland participating in further education than was the case in England and Wales. Satisfaction was similar. How they first heard about Individual Learning Accounts was reasonably similar. Sources of guidance were not quite as high in Scotland and Wales, from learning providers, as they were in England and Northern Ireland. But very few things actually stood out. In some cases the proportion of people who said that they could have paid for their courses without having an Individual Learning Account was higher in two other home countries than in England.


  116. I propose we move on now to the findings. What strikes me from reading the material that you have supplied is that there was a delay in publishing from October to 14 January 2002. What is the explanation for that?
  (Mr Rodger) That is not untypical. In terms of the results being made to the department and the time that it takes for that to get into the public domain, I would say that that is quite normal.

  117. Is it quite normal in terms of what must have been going on in the department at that time?
  (Mr Rodger) We have no knowledge of that. If I did not know what I know now and I had seen the report coming out, I would have thought that that would be normal procedure.

Paul Holmes

  118. I have two questions on the findings. Previously we have heard from training providers that by the spring of 2001 they had said to the Government that they thought that the scheme was flawed and that it was open to abuse. You are saying in both of your surveys that you carried out in spring 2001 and in late summer/early autumn 2001 that the Government never asked you to look at that aspect of it at all?
  (Miss Owens) We were not asked to look at providers in the second survey. It was purely people with accounts in the second survey. The only time that we spoke to providers was in the first one and it was an early perceptions survey that they wanted us to do. They did not ask us to focus on specific issues. They had not raised the issue of there being problems. They asked us to look at things like the type of learners that the providers were seeing coming through, whether ILAs had had an impact on the course costs and whether employers were contributing. They also wanted us to tease out whether there were any other issues. The providers quite willingly expressed any issues or concerns that they had, but none of them raised the issues that have come up more recently. The only issues that they raised, for instance, was the slow website at times when they were entering details and the fact that they felt that some providers were not marketing Individual Learning Accounts in a proper manner, or were claiming the 80 per cent discount for ineligible courses.

  119. I have two questions on the findings from your surveys. The ILAs were hopefully designed to draw people into training, or back into training, who were not normally in that area. Nationally about 16.5 per cent of the population have no qualifications or training. One of the findings of your study was that only about 16 per cent of people to whom you spoke came from that background of no training at all. Is that correct?
  (Miss Owens) That is correct. It went up to 18 per cent in the second survey.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 6 March 2002