Select Committee on Innovation, Universities, Science and Skills Minutes of Evidence

Examination of Witnesses (Question Numbers 20-39)


17 JULY 2008

  Q20  Mr Wilson: A risk-based system, so that you can target your resources more effectively and ensure that standards are more robust by doing that.

  Dr Jackson: This is an issue we are currently discussing in the agency. The current cycle of audits is due to be completed by 2011. We are discussing what methodology we should adopt after 2011. One of the models we are currently looking at is a risk-based proportionate model, with scrutiny of institutions based on pre-existing evidence about their quality and standards.

  Q21  Mr Boswell: Perhaps I could come back to something about the grades "no confidence", "limited confidence" and "confidence". I do not want to say anything about degree classification obviously, but are you happy that is the right sort of framework? Would it be that the level of confidence is sufficiently refined to drive some institutions to do better, rather than run the risk of resting on their laurels? In an ideal world, would you design it this way or would you seek to modify it? To put it another way, to give you an opportunity at a rounded answer: Can we set a threshold which is adequate and only that, or are we looking to drive the system upwards?

  Mr Williams: We have a classified system.

  Q22  Mr Boswell: That was a rounded up way of saying that.

  Mr Williams: We use a confidence judgment. I think it is important to recognise precisely what a confidence judgment is. Most processes of this sort take a snapshot. They are essentially historic, looking backwards. They are looking at what is done—essentially, the day before yesterday. That is not much use unless you can be absolutely sure that what happened yesterday is going to continue to happen tomorrow.

  Q23  Mr Boswell: This is not much use to your student sample: they want to know when they are there that it is going to be all right.

  Mr Williams: Our judgment of confidence is essentially a judgment we make as informed peers, or as our peer reviewers do, of whether this is meeting, at present, the satisfactory maintenance of standards and quality and whether we think in our judgment it will continue. Of course that is really quite difficult to predict, but we want to see that everything is in place, so that if individuals leave or things change there will still be a sufficient infrastructure which will be likely to carry on maintaining the standards and the quality. That is what we do. Under those circumstances, the danger of having a graded profile is that the costs would be greater than the benefits.

  Q24  Mr Boswell: Have you ever been judicially reviewed on the assurances that you have given?

  Mr Williams: No.

  Q25  Mr Boswell: Nobody has ever sought the legal route?

  Mr Williams: No. We have had some sabres rattled sometimes but nobody has ever come forward.

  Q26  Dr Evan Harris: Is there an appeals process?

  Mr Williams: There is an appeal against no confidence.

  Q27  Dr Evan Harris: Who hears the appeal?

  Mr Williams: The appeal is heard by a subgroup of our board but it does take external evidence as well. It has independent external advisers to make sure that the story that our colleagues tell is valid.

  Q28  Dr Evan Harris: I was intrigued by this idea that a student who wants to pass on concerns about the quality of the institution and wishes to do so in confidence is referred invariably to the Office of the Independent Adjudicator. Let me give you the example. If a student contacts you to say, "Can I tell you in confidence that I have concerns about widespread plagiarism in this institution and that is not being picked up" and there are mechanisms to pick it up, you will deal with that.

  Mr Williams: That is our area of responsibility. If it is a complaint about the treatment of the student within the institution or their individual experience, that is something that we recognise as the responsibility of the Office of the Independent Adjudicator.

  Q29  Dr Evan Harris: The example the Chairman gave was of a student who said he had been treated too well. I think the Office of the Independent Adjudicator would deal with harms to students, not unjustified benefits. Surely you should not be referring a case like that, even if it is an individual experience, to a complaints procedure when the student is not making a complaint on their own behalf. It would be curious.

  Mr Williams: It is difficult to answer on, in a sense, bizarre and hypothetical cases. It happened, presumably. I think I was given a grade above what I should have been—but many, many years ago—but I am not a judge. I am not a competent academic judge. I am not an academic.

  Q30  Mr Boswell: I take it you do talk to one another. If the office for student complaints gets something which looks like being systematic in an institution rather than just one person who did not get on with their supervisor, or, conversely, if you think this is really something which has its background in an individual clash or difficulty, you can presumably talk to one another and pass them over.

  Mr Williams: Yes. Although I think it is fair to say that it is only recently that our links with the OIA have become really effective. We now have a very good working relationship.

  Q31  Dr Evan Harris: I am still concerned. I must say that I thought the OIA deals with complaints once they have gone through the university's own procedure and only then.

  Mr Williams: That is right. It does.

  Q32  Dr Evan Harris: I think the example the Chairman gave is clearly inappropriate for the OIA. I am concerned that you would think that you could directly refer to the OIA, when you can really only do that once you have gone through the university's complaints procedure. I can understand that you should say this is something you should raise with the university first.

  Mr Williams: Yes.

  Q33  Dr Evan Harris: Have you ever thought of using other techniques to judge quality than the ones you have? Some people use what are considered powerful techniques of quality assurance, such as the mystery shopper. Or is that just too radical for your austere organisation?

  Mr Williams: If we wanted to use the mystery shopper we would have to be very clear about precisely what we were doing, why we were doing it, how we were doing it, and how it would validate the outcomes.

  Q34  Dr Evan Harris: Yes, assuming you could do all that—I am not saying you should do something that is useless—would you consider that? Have you ever considered it?

  Mr Williams: It is something we have thought about. We have also thought about things like unannounced visits. You have to be very careful as to how you are going to use the evidence you get. The mystery shopper is after a particular piece of information, so it could be quite a useful technique for looking at specific questions, but it is not going to tell you very much about the overall way in which the institution is managed.

  Q35  Dr Evan Harris: As you know, it is the threat of it: if institutions know that they might be out there. Most of the thrust of it is to raise standards out of fear. That is the idea.

  Mr Williams: Of course the press do this from time to time. It is an interesting technique but we would need to be sure how we were using it.

  Q36  Dr Evan Harris: My other question relates to the use of the audit procedure. Your external audit is essentially a form of peer review.

  Mr Williams: Yes.

  Q37  Dr Evan Harris: If peer review is fine and reckoned to be good enough, then that is reassuring for you. But if there were questions about peer review, that would lead to concerns about how you did your business. Are you happy that peer review, as practised in this country, is as good as it can be?

  Mr Williams: I think the answer to that question is: It is not a perfect system but it is a good system.

  Q38  Dr Evan Harris: Could it be improved?

  Mr Williams: Everything can always be improved.

  Q39  Dr Evan Harris: Really.

  Mr Williams: We believe in continuous improvement. To improve is part of our mission.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 27 November 2008