Üjf199ÝOral evidenceÌTaken before the Innovation, Universities, Science and Skills CommitteeÊ(Innovation, Universities, Science and Skills Sub-Committee)ÊÈon Thursday 17 July 2008ÌÜjf27ÝMembers presentÌÜjf27ÝMr Phil Willis, in the ChairÌÜjf55ÝMr Tim BoswellÈDr Evan HarrisÌMr Rob WilsonÌÜjf22ÝÜjf50ÝÜcf2ÝWitnesses: Ücf3ÝMr Peter WilliamsÜcf1Ý, Chief Executive, Ücf3ÝMr Douglas BlackstockÜcf1Ý, Director of Administration and Company Secretary, Ücf3ÝDr Nick HarrisÜcf1Ý, Director of Development and Enhancement Group, and Ücf3ÝDr Stephen JacksonÜcf1Ý, Director of Reviews, Quality Assurance Agency (QAA), gave evidence.ÌÜjf12ÝÜcf3ÝChairman:Ì Good morning and welcome to our witnesses this morning: Peter Williams, the Chief Executive of Quality Assurance Agency; Dr Stephen Jackson, Director of Reviews at QAA; Dr Nick Harris, Director of Development and Enhancement Group at QAA; and Douglas Blackstock, Director of Administration at QAA. Welcome to you.ËÜjf65Ý17 July 2008	Mr Peter Williams, Mr Douglas Blackstock, Dr Nick Harris and Dr Stephen JacksonÌÜjf12ÝÜcf3ÝMr Boswell:Ì A point of order, Mr Chairman, if I may. May I mention to the Committee a minor interest I have in this matter as a member of the governing board of the University of Wales Institute Cardiff. I mention that now specifically because our application for research degree awarding powers is currently under consideration by QAA.ËÜjf12ÝÜcf3ÝChairman:Ì Could I declare that I am a member of the Court of University of Birmingham. I do not quite know what that means, but I put that on the record.ËÜjf12ÝÜcf3ÝDr Evan Harris:Ì I am a member of the Court of Oxford Brookes University.ËÜjf12ÝÜcf3ÝMr Wilson:Ì I think I am a member of the Court of Reading University.ËÜjf12ÝÜcf3ÝMr Boswell:Ì I am a member of the Court of Nottingham University.ËÜjf13ÝÜcf3ÝQ1 Chairman:Ì You can see that we are incredibly well qualified to be able to discuss this. I would like to thank you particularly for coming at short notice. We very much appreciate it. This is a one-off inquiry looking at the three reports which you published in June. The reports have become a focus for allegations that universities are marking softly", that they are awarding higher grades because of the pressures of league tables, and, also, the particular accusation that they are attracting international students because they are paying unregulated fees. There has been a lot of speculation in the media following comments you also made. As the Committee that has responsibility for scrutinising higher education, we thought it was important to give you and your colleagues the opportunity put the record straight, but also for us in our scrutiny role to see that the standards within higher education and particularly the role of the QAA are being maintained and upheld. That is the purpose of this session. As always, I think we are trying to approach our work in terms of adding value to the system rather than simply trying to find fault. What is the purpose of the Quality Assurance Agency? It appears to me that you provide rather more consultancy services than you do acting as an auditor or inspector.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Chairman, could I say that we very much welcome this opportunity to discuss our work with you. We are pleased that the Committee feels this is a proper area for it to look at. You ask whether we are essentially consultants or auditors. The answer to that is that we are auditors principally. I wrote an article once about what kind of dog is the QAA: guard dog, watchdog, lap-dog and so on, and I think we should consider ourselves a watchdog principally; that is to say, we go and look at institutions. We have a remit to find out and check that they are managing their quality and standards in an appropriate way and in a proper way bearing in mind their legal powers. That is the principal purpose of our job. Having said that, I think it would be remiss of us if, within that, we were not to use the information we get from those inspections to inform the sector of what is going on, what is good practice, what is not so good practice. That, essentially, is what these three reports are about.ËÜjf13ÝÜcf3ÝQ2 Chairman:Ì The taxpayer has a real interest in the work you do.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ3 Chairman:Ì Whilst we understand you are not a government-appointed body_and perhaps we will return to whether you should be later_you have no teeth, have you? You are just a toothless dog, if I might continue with the canine analogy.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is interesting that, once we get on to the canine analogy, we always end up on the dental analogies. The answer to that is that you really need to talk to those institutions to which we have awarded the judgment of no confidence" or limited confidence" in our audits, because the impact on them is very severe; that is to say, the effect on two institutions has been that one institution has changed its name and the bank manager of the other reduced their credit rating.ËÜjf13ÝÜcf3ÝQ4 Chairman:Ì Ofsted can effectively close a school; ETIÜepÜcf5ÝÜfu1ÝÜnhÜrs can close a college. I am not aware of any higher education institution that has been closed as a result of your work.ËÜfo1ÝÜepÜcf5,8,8.5ÝÜnhÜcf1ÝÜixEducation and Training InspectorateËÜrsÜfeÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Higher Education institutions are independent legal entities. They would only be closed if they could not afford to keep going. It is not for us to close anything down. We have in the past been the cause, ultimately, of the resignation of a vice-chancellor but it is not in our remit to close universities down.ËÜjf13ÝÜcf3ÝQ5 Mr Boswell:Ì Perhaps I could ask two questions. One is on the funding model. Is there any driver, in terms of the funding model, where somebody who has been funded by you might lose support? The second one is at the academic level. I am aware_but perhaps it would be helpful if you could confirm for the Committee_that once degree awarding powers, whether taught or researched, have been awarded, on your recommendation, they are not withdrawn. Is that still the case? Can you say, in principle, that because something dreadful has gone wrong_and I am not talking about some historic case, I am talking about a hypothetical case_this should not in such circumstances be withdrawal of degree awarding powers or even, ultimately, a university?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We now receive something like two-thirds of our income from funding councils across the UK. Given that we are contracted to deliver the reassurance they need under the 1992 Act, we do that through a contract with them. If we come up with a negative limited confidence" or no confidence" judgment, we report the outcome to HEFCE and it is up to them then to decide whether or not to exercise any sanctions upon the institution. I should say that, before that happens, with one of those negative judgments there is an action plan that has to be produced by the institution. That is monitored by us and ultimately signed off by us.ËÜjf13ÝÜcf3ÝQ6 Mr Boswell:Ì That would not be different conceptually from the kind of action plan driven by an Ofsted inspection in relation to further education provision or, indeed, in schools. It is the same sort of approach: something is wrong, we need to think about what we should do and, ultimately, there is a sanction.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf14ÝÜcf2ÝÜcf4ÝDr Jackson:Ücf1Ý In principle that is right. The action plans contain details about how the institutions will respond to recommendations in the report. We identify in the report the issues that need to be addressed, then we expect the action plan to say how those issues are addressed and the time scale in which they would be implemented.ËÜjf13ÝÜcf3ÝQ7 Chairman:Ì Peter, it assumes that the discussions you have are with the vice-chancellors and senior academics within university institutions. What happens when a whistleblower comes along and says that something unacceptable is happening within a university department? Can they come to you? Do they come to you? What do you do about it? Or do you simply brush it under the carpet?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No. I think the first thing to say is that we do not just talk to vice-chancellors and senior staff. At our audits_and Stephen could describe those better if you wish_we talk to people at all levels in the institutions, including students. Students are invited to write their own statement to us about what they are getting. In the case of whistleblowers, when the new framework for quality assurance was devised in 2002 it left a gap. That gap was precisely the whistleblowing gap. We did these audits. We did them once every three years and the agreement was that we would move to once every six years. The question I posed was: What do we do in the meantime if something goes wrong? In order to fill that gap, we have developed a process called causes for concern". That process allows whistleblowers to tell us, provided they have evidence_and we are not prepared to accept complaints without evidence, because there are a lot of reasons why people might want to be difficult, shall we say_ËÜjf13ÝÜcf3ÝQ8 Chairman:Ì It is the same with politicians.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý This is a graded process, a proportional response process, whereby people can come to us and we look at the evidence. When we think there is something in the evidence that is worth pursuing, we will discuss it, in the first instance, with the head of the institution concerned. If we are not satisfied with the response to that, we will then escalate it until we get a full new audit. That will then be subject to review, which will be published and the outcomes will be publicly seen.ËÜjf13ÝÜcf3ÝQ9 Chairman:Ì Has that ever happened?ËÜjf14ÝÜcf2ÝÜcf4ÝDr Jackson:Ücf1Ý In one case.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý The process only came in last year. We only introduced it last year. The individual case was a college of further education, over its higher education awards. There was a negative Ofsted inspection report, where the funding council was concerned about the quality of standards and HE provision and asked us to use this procedure in order to provide assurance that the students and HE programmes were being properly looked after.ËÜjf13ÝÜcf3ÝQ10 Chairman:Ì If a student wrote to me about the quality of the masters course provision, saying it was so poor, and another student wrote to me to say that he had been awarded a third class honours degree when he should not have got anything because he was so appalling, if they wrote to you what would you do about it? That is what I have advised them to do.ËÜjf14ÝÜcf2ÝÜcf4ÝDr Jackson:Ücf1Ý We would not do anything. In the first case student complaints are handled by the adjudicators.ËÜjf13ÝÜcf3ÝQ11 Chairman:Ì We have been through that route. It has not worked.ËÜjf14ÝÜcf2ÝÜcf4ÝDr Jackson:Ücf1Ý If students or other members of staff in an institution were able to provide us with evidence of serious concerns, then that would trigger the cause for concern" process.ËÜjf13ÝÜcf3ÝQ12 Mr Boswell:Ì I want to come back to the second part of my question: the academic sanction as opposed to financial.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý This is whether degree-awarding powers can be withdrawn. There is a distinction to be drawn between non publicly-funded institutions and publicly-funded institutions. The criteria for awarding powers are the Government's not QAA's but we use those criteria when assessing institutions for whether or not they should get powers. The publicly-funded institutions are all subscribers to QAA, therefore they will submit to audit.ËÜjf13ÝÜcf3ÝQ13 Mr Boswell:Ì So far as I know, the Privy Council, which is the body responsible for ultimately awarding the powers, has never withdrawn powers. But it would be open to you, if you were concerned, either in conjunction with ministers or maybe on your own recognisance, to write to the Privy Council to say that the situation is so bad that_ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We could, in theory, although I think that would be going along quite a long road of inquiry. When it comes to the privately funded, they are also required to be subscribers to QAA. Those powers are granted on a six-year renewal basis. At the end of six years, they come up for review anyway.ËÜjf13ÝÜcf3ÝQ14 Mr Boswell:Ì That is a licence.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is a kind of licence. There is an interest in deciding whether or not after six years they should become permanent in the same way as everybody else, so as not to distinguish, unnaturally, in a sense, between publicly-funded and non publicly-funded. I am not aware that in public debate there has been any serious canvassing of the suggestion that anybody's powers, be it Oxford or some privately-funded institution, should be converted into a licence_I suppose anticipating there might be some concerns about academic freedom if that were to happen. It is not the British way; it is the Continental way but it is not the way we have done it here.ËÜjf13ÝÜcf3ÝQ15 Mr Boswell:Ì It is similar. I have one final question about Bologna. Given that we now have the Bologna Process which is designed to provide comparability, are there any concerns about Bologna. If there is an unsatisfactory situation would you report that to anybody? If so, who would take an interest?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý All the requirements of Bologna are already incorporated within our inspections and institutions. If something is deficient in Bologna, it would be deficient at home as well. We would probably see it first as home-deficient rather than Bologna-deficient.ËÜjf13ÝÜcf3ÝQ16 Mr Boswell:Ì A director of a university who had maybe exchange students with a British university would not normally get engaged with anybody else. If necessary he would write to QAA and say, We are having trouble with our postgraduates."ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is right.ËÜjf13ÝÜcf3ÝQ17 Mr Wilson:Ì You have been pretty clear about the judgments you have; that is no confidence" and limited confidence". You were formed in 1997. In those 11 years, with how many institutions have you used those powers so far?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý A round of audits started between 2002 and 2006, when all the English institutions were audited. There were six judgments of limited confidence" and one judgment of no confidence".ËÜjf13ÝÜcf3ÝQ18 Mr Wilson:Ì In those four years, how many would have no confidence" at the moment?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý One we had no confidence in.ËÜjf13ÝÜcf3ÝQ19 Mr Wilson:Ì How many have limited confidence at this moment?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think I am right in saying that all the limited confidence" and the no confidence" from the last round of audit have now done their action plan to our satisfaction and are now back in good standing.ËÜjf12ÝÜcf3ÝMr Wilson:Ì Are there some institutions that you think should have a more rigorous inspection, because they show signs of being more of a worry?ËÜjf12ÝÜcf3ÝMr Boswell:Ì Risk based.ËÜjf13ÝÜcf3ÝQ20 Mr Wilson:Ì A risk-based system, so that you can target your resources more effectively and ensure that standards are more robust by doing that.ËÜjf14ÝÜcf2ÝÜcf4ÝDr Jackson:Ücf1Ý This is an issue we are currently discussing in the agency. The current cycle of audits is due to be completed by 2011. We are discussing what methodology we should adopt after 2011. One of the models we are currently looking at is a risk-based proportionate model, with scrutiny of institutions based on pre-existing evidence about their quality and standards.ËÜjf13ÝÜcf3ÝQ21 Mr Boswell:Ì Perhaps I could come back to something about the grades no confidence", limited confidence" and confidence". I do not want to say anything about degree classification obviously, but are you happy that is the right sort of framework? Would it be that the level of confidence is sufficiently refined to drive some institutions to do better, rather than run the risk of resting on their laurels? In an ideal world, would you design it this way or would you seek to modify it? To put it another way, to give you an opportunity at a rounded answer: Can we set a threshold which is adequate and only that, or are we looking to drive the system upwards?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We have a classified system.ËÜjf13ÝÜcf3ÝQ22 Mr Boswell:Ì That was a rounded up way of saying that.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We use a confidence judgment. I think it is important to recognise precisely what a confidence judgment is. Most processes of this sort take a snapshot. They are essentially historic, looking backwards. They are looking at what is done_essentially, the day before yesterday. That is not much use unless you can be absolutely sure that what happened yesterday is going to continue to happen tomorrow.ËÜjf13ÝÜcf3ÝQ23 Mr Boswell:Ì This is not much use to your student sample: they want to know when they are there that it is going to be all right.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Our judgment of confidence is essentially a judgment we make as informed peers, or as our peer reviewers do, of whether this is meeting, at present, the satisfactory maintenance of standards and quality and whether we think in our judgment it will continue. Of course that is really quite difficult to predict, but we want to see that everything is in place, so that if individuals leave or things change there will still be a sufficient infrastructure which will be likely to carry on maintaining the standards and the quality. That is what we do. Under those circumstances, the danger of having a graded profile is that the costs would be greater than the benefits.ËÜjf13ÝÜcf3ÝQ24 Mr Boswell:Ì Have you ever been judicially reviewed on the assurances that you have given?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No.ËÜjf13ÝÜcf3ÝQ25 Mr Boswell:Ì Nobody has ever sought the legal route?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No. We have had some sabres rattled sometimes but nobody has ever come forward.ËÜjf13ÝÜcf3ÝQ26 Dr Evan Harris:Ì Is there an appeals process?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý There is an appeal against no confidence.ËÜjf13ÝÜcf3ÝQ27 Dr Evan Harris:Ì Who hears the appeal?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý The appeal is heard by a subgroup of our board but it does take external evidence as well. It has independent external advisers to make sure that the story that our colleagues tell is valid.ËÜjf13ÝÜcf3ÝQ28 Dr Evan Harris:Ì I was intrigued by this idea that a student who wants to pass on concerns about the quality of the institution and wishes to do so in confidence is referred invariably to the Office of the Independent Adjudicator. Let me give you the example. If a student contacts you to say, Can I tell you in confidence that I have concerns about widespread plagiarism in this institution and that is not being picked up" and there are mechanisms to pick it up, you will deal with that.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is our area of responsibility. If it is a complaint about the treatment of the student within the institution or their individual experience, that is something that we recognise as the responsibility of the Office of the Independent Adjudicator.ËÜjf13ÝÜcf3ÝQ29 Dr Evan Harris:Ì The example the Chairman gave was of a student who said he had been treated too well. I think the Office of the Independent Adjudicator would deal with harms to students, not unjustified benefits. Surely you should not be referring a case like that, even if it is an individual experience, to a complaints procedure when the student is not making a complaint on their own behalf. It would be curious.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is difficult to answer on, in a sense, bizarre and hypothetical cases. It happened, presumably. I think I was given a grade above what I should have been_but many, many years ago_but I am not a judge. I am not a competent academic judge. I am not an academic.ËÜjf13ÝÜcf3ÝQ30 Mr Boswell:Ì I take it you do talk to one another. If the office for student complaints gets something which looks like being systematic in an institution rather than just one person who did not get on with their supervisor, or, conversely, if you think this is really something which has its background in an individual clash or difficulty, you can presumably talk to one another and pass them over.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. Although I think it is fair to say that it is only recently that our links with the OIA have become really effective. We now have a very good working relationship.ËÜjf13ÝÜcf3ÝQ31 Dr Evan Harris:Ì I am still concerned. I must say that I thought the OIA deals with complaints once they have gone through the university's own procedure and only then.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is right. It does.ËÜjf13ÝÜcf3ÝQ32 Dr Evan Harris:Ì I think the example the Chairman gave is clearly inappropriate for the OIA. I am concerned that you would think that you could directly refer to the OIA, when you can really only do that once you have gone through the university's complaints procedure. I can understand that you should say this is something you should raise with the university first.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ33 Dr Evan Harris:Ì Have you ever thought of using other techniques to judge quality than the ones you have? Some people use what are considered powerful techniques of quality assurance, such as the mystery shopper. Or is that just too radical for your austere organisation?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý If we wanted to use the mystery shopper we would have to be very clear about precisely what we were doing, why we were doing it, how we were doing it, and how it would validate the outcomes.ËÜjf13ÝÜcf3ÝQ34 Dr Evan Harris:Ì Yes, assuming you could do all that_I am not saying you should do something that is useless_would you consider that? Have you ever considered it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is something we have thought about. We have also thought about things like unannounced visits. You have to be very careful as to how you are going to use the evidence you get. The mystery shopper is after a particular piece of information, so it could be quite a useful technique for looking at specific questions, but it is not going to tell you very much about the overall way in which the institution is managed.ËÜjf13ÝÜcf3ÝQ35 Dr Evan Harris:Ì As you know, it is the threat of it: if institutions know that they might be out there. Most of the thrust of it is to raise standards out of fear. That is the idea.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Of course the press do this from time to time. It is an interesting technique but we would need to be sure how we were using it.ËÜjf13ÝÜcf3ÝQ36 Dr Evan Harris:Ì My other question relates to the use of the audit procedure. Your external audit is essentially a form of peer review.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ37 Dr Evan Harris:Ì If peer review is fine and reckoned to be good enough, then that is reassuring for you. But if there were questions about peer review, that would lead to concerns about how you did your business. Are you happy that peer review, as practised in this country, is as good as it can be?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think the answer to that question is: It is not a perfect system but it is a good system.ËÜjf13ÝÜcf3ÝQ38 Dr Evan Harris:Ì Could it be improved?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Everything can always be improved.ËÜjf13ÝÜcf3ÝQ39 Dr Evan Harris:Ì Really.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We believe in continuous improvement. To improve is part of our mission.ËÜjf13ÝÜcf3ÝQ40 Dr Evan Harris:Ì What is going on out there that improves peer review?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Our training, our feedback, our evaluation and our annual updating of reviewers.ËÜjf13ÝÜcf3ÝQ41 Mr Boswell:Ì For clarity, we are taking about your peer review process as part of external audit.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ42 Mr Boswell:Ì We are not talking about your views on peer review.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No.ËÜjf13ÝÜcf3ÝQ43 Mr Boswell:Ì That is conducted at the course level.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý This is what we do.ËÜjf13ÝÜcf3ÝQ44 Mr Boswell:Ì Your own systems.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes, our own internal quality assurance.ËÜjf13ÝÜcf3ÝQ45 Chairman:Ì On 23 June you produced the outcomes from institutional audit, the three reports. You noted in your reports and, indeed, in your press release that accompanied them that there had been much solid achievement". On 23 June, Peter, you gave an interview to the BBC in which you said that the system was robust, but then you made a number of criticisms in language which might be characterised as headline-grabbing. You said, for example, that the degree classification was arbitrary and unreliable" and you went on to say that it was rotten". Do you believe that key parts of the UK's higher education system are defective? When somebody from QAA says the system is arbitrary and unreliable" and rotten" you can hardly blame the media for then picking that up and saying that is what you said.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Those particular quotes were taken, as you will appreciate, from a much longer discussion with a journalist, and of course he picked out the plums_perfectly reasonably.ËÜjf13ÝÜcf3ÝQ46 Chairman:Ì That is what they do.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Indeed, they do. But it was part of a longer conversation. What surprises me is not that they picked it up in June but that they did not pick it up in April 2007 when I said exactly the same thing. I said rather more robust things, reported on BBC Online, and nobody at all commented. This is the specific question of how a student's achievement is recorded. It is not about the general higher education system in this country, which I believe is robust, is solid, and is good.ËÜjf13ÝÜcf3ÝQ47 Chairman:Ì This is degree classification.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is degree classification. That is the only thing I was talking about when I said that the system was rotten_and I agree that rotten is a colourful adjective to use. I have strong feelings about the degree classification system which I have made clear on a number of occasions.ËÜjf12ÝÜcf3ÝChairman:Ì That goes really to the heart of confidence in our higher education system, does it not? Degree classification employers think is important. People will get jobs or their futures depend on that degree classification.ËÜjf12ÝÜcf3ÝMr Boswell:Ì And the grading of universities.ËÜjf13ÝÜcf3ÝQ48 Chairman:Ì And the grading of universities is that. When people are accused of inflating the number of firsts_and we seem to have had a miraculous improvement in student achievement in some of our universities_that does not give confidence, does it? Is that what you were talking about?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý All I was really doing was echoing the discussions in the two Burgess committees and now the implementation group, which came to the same conclusion: the system is not fit for purpose. It is as if that system was designed for other times, for a smaller higher education world. We have now reached the end of the display by" date and we have now pretty well got to the use by" date.ËÜjf13ÝÜcf3ÝQ49 Chairman:Ì Why did you not say in your reports that in future you will not in fact inspect or audit universities until they reform their degree classification to give confidence in the system? Surely you can drive this.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We are driving it, in a way, in the influence we have had on the Burgess group and the implementation process which is ongoing. Our Ücf2ÝQuality MattersÜcf1Ý paper which we brought out in April 2007 left nobody in any doubt about why we thought the system was time-expired. The decision on reclassification is one for the sector; it is not one for individual institutions. It would be extremely difficult for a single institution to move unilaterally on that. In the early 1990s the University of London tried to do it. It gave itself five years, I think it was, for a transition period to get rid of degree classifications in favour of a common transcript. It did not work. It did not happen because people were not prepared to jettison that with which they were comfortable.ËÜjf13ÝÜcf3ÝQ50 Mr Boswell:Ì Perhaps I could pick up on that, not historically but conceptually. Clearly there is a strong attraction for employers, and probably for students as well, in having some assurance of some coherence about what they have. I am no expert on the alternative system and I am not wishing to advocate this, but, given that you can see the present system is strained to the limits or gone beyond the limits_partly because of the greater number of higher education institutions and the diversity which I welcome_is there any common core that ought to be kept out of that, so that, if we take the institutions which are geographically equidistant from my home, the University of Oxford and the University of Northampton, both of which I know very well, there is some sense in which an attainment which is excellent in one is reflected or could be said to be comparable with comparable excellence at another. Is there something you want to salvage out of the degree classification to provide some common currency?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý This is contested territory. A lot of people have different views on it and mine is just one view. I think we have to agree the basic level at which a British degree is valid.ËÜjf13ÝÜcf3ÝQ51 Mr Boswell:Ì That is the threshold.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is right, the threshold. Once you get above the threshold, then it is much more important that you are clear about what it is you have. Of course, diversity means you get different things_not that one is better than the other but they are different from each other_and the important thing is that it is clear what it is you have. I think that information about a student's achievement is more important than trying to give them a brand, a five-point scale which is claimed to be the same across the whole country. It cannot happen. It is logistically not possible.ËÜjf13ÝÜcf3ÝQ52 Dr Evan Harris:Ì Forgive me, Tim, if this was not your question, but is the question whether an employer who has two candidates, one with a first class degree from Cambridge and another with a first class degree from another university which is much more recent_ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Uttoxeter is an example.ËÜjf13ÝÜcf3ÝQ53 Dr Evan Harris:Ì Uttoxeter, yes. I am not sure I have ever been. Would an employer be correct in thinking that those students had the same value of qualification?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think it would be rather foolish to assume that. There is no common definition of what a first is.ËÜjf13ÝÜcf3ÝQ54 Dr Evan Harris:Ì Let us say there are allegations of parochialism, so that a major employer had to have observed the awarding institution, would you say that they would notice the difference, if they were not able to distinguish in appointments between a first class degree from Cambridge and one from Uttoxeter, as you have put it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is impossible to say.ËÜjf13ÝÜcf3ÝQ55 Dr Evan Harris:Ì Is it really impossible to say?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think it is. It is very, very difficult. The evidence suggests that there is no consistency between subjects in institutions or between institutions.ËÜjf13ÝÜcf3ÝQ56 Dr Evan Harris:Ì I do not understand. If you need to get, say, the equivalent of three As at A-level to get into one institution and you can get into another institution with something like three Ds at A-level, and you both end up with a first class degree, is it reasonable for employers to take cognisance of that? If not, what is the point of having that classification or, indeed, any classification, unless it does distinguish between ability? Exams, surely, should be the same standard.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý In this country we have 118, I think it is, awarding bodies. That is 118 individual institutions with the powers to award their own degrees and to set their own standards. They do that as autonomous bodies. There is not, in a sense, a national curriculum and a national examination.ËÜjf13ÝÜcf3ÝQ57 Dr Evan Harris:Ì I thought your job was to check that people were not being awarded first class degrees undeserved in one institution. In a consumerised market, people can say I've got a greater chance of getting a first class degree there, given my entry qualifications."ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No, that is not part of our job. Our job in that is to say to the institution, Do you have some rules and regulations about what a first is? Can you tell us? Can you show us what you mean by a first? Do your assessment systems guarantee that only the people who meet those are given a first? Is that process operating satisfactorily?"ËÜjf13ÝÜcf3ÝQ58 Dr Evan Harris:Ì If some research was done with new graduates that showed objectively there was a massive discrepancy in the standard of knowledge attained in that subject between two people holding first class honours degrees, would that suggest that your system was not sufficient to identify that? Would you be concerned by it? Would you commission your own research?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We have done that. The April 2007 paper essentially is saying that.ËÜjf13ÝÜcf3ÝQ59 Dr Evan Harris:Ì You have taken a graduate with a first from one place and a graduate from another_lots of them to make it statistically significant_tested them and looked at what the results say.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No, we are not a research organisation of that sort.ËÜjf13ÝÜcf3ÝQ60 Dr Evan Harris:Ì Do you think that research should be done? Would it be awkward for you?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It would not be awkward for us, but we do not do research of that sort.ËÜjf13ÝÜcf3ÝQ61 Dr Evan Harris:Ì Should it be done?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Research of that sort is always interesting. You have to watch the method very carefully, because there are methodological issues there about where you choose them from, what year you choose them from and so on.ËÜjf13ÝÜcf3ÝQ62 Chairman:Ì This is descending into farce really. It is more than interesting; it goes right to the heart of what we are talking about. You are saying that an individual institution can award as many firsts as it can, provided it satisfies its own criteria as to what a first means. Somehow you are saying that the broader public, employers and the international community, do not make a distinction between a first at Oxford and a first at Uttoxeter. Nobody believes that, do they?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý There are a number of things you have said there. The first is that it is not quite as bad as that because there is an external examination system which brings external eyes to play on the standards. The external examiner system is a very valuable system. It is a very good way of a university knowing where it sits in relation to other institutions that the external examiner himself or herself knows. But by its very nature it is a limited system. It cannot provide a kind of nationwide security when we are dealing with, whatever it is, 600,000 students a year in 118 awarding bodies in England.ËÜjf13ÝÜcf3ÝQ63 Chairman:Ì That is what quality assurance is about, is it not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No. It is not about a single monolithic_ËÜjf13ÝÜcf3ÝQ64 Mr Wilson:Ì You seem to be saying you are completely satisfied with the external examiner system as it currently stands, but there are a lot of people who just think it is bringing in mates on the course team to shore up what is going on.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I did not say I am entirely satisfied with the external examiner system. Our reports suggest there are a number of things that need to be looked at. I am saying it is a valuable and useful part of the system. What you are all saying essentially backs up our argument that the degree classification system is past its usefulness. It is not doing what it claims to do and therefore should be changed. And it is being changed. The implementation committee is in place and it is doing its work in order to change that because everybody agrees that it is not now fit for its original purpose.ËÜjf13ÝÜcf3ÝQ65 Mr Boswell:Ì Could I go back to the technical side of this, which in a sense it harks back to an earlier exchange we have already had about your institutional confidence. Your view on degree classification is that it is not a system that is going to work but there needs to be some base-load threshold standard which enables you to say this institution is offering something that is called a degree and not something that is called a recreational attainment certificate. That does require some degree of objectivity of criteria.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ66 Mr Boswell:Ì In a sense_and I am making this point only for the purpose of the argument_I would be interested to know how you can apply that, which presumably will read across into a new system. If you can apply it at the level of threshold adequacy, if you like, why could you not do so in further areas? Is it because you do not want to get to intrusive on the institution or you become less confident about your findings or you are liable to be sued on it or whatever?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý In answer to your first question, there is in place something which we call the Academic Infrastructure. I do not want to bore you with the detail of that, but one part of that is the Framework for Higher Education Qualifications, which is a set of descriptors of the level that should be achieved in order to be awarded with a degree. That goes from certificate to diplomas, to honours degrees, to masters degrees, to PhDs.ËÜjf13ÝÜcf3ÝQ67 Mr Boswell:Ì And that is, as it were, a reasonably objective test that you can apply reasonably, with confidence.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is a reasonably objective test. It is very generic because it covers all degrees of any subject. But that is backed up by subject benchmark statements, which are an indication of the expectations in terms of characteristics of graduates in individual subjects. That provides an external boundary, if you like, within which institutions operate. They are free to operate, the autonomy allows them to operate within that, but there is that boundary. It is that boundary, in a sense, that we police. Within it, they are free to do what they like.ËÜjf13ÝÜcf3ÝQ68 Mr Boswell:Ì Could you just go on to talk about the development above that.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Institutions are free to develop whatever they like. The difficulty is when they all use exactly the same five-point scale but in different ways. That is the problem. It is not consistent. It is the lack of consistency that is the problem that we see.ËÜjf13ÝÜcf3ÝQ69 Mr Boswell:Ì So we have created a system_and I suspect this has persisted but maybe has intensified_whereby the fiction has been that the University of Cambridge and the University of Uttoxeter (shall we say, neutrally) are identical and what they are applying is the same.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ70 Mr Boswell:Ì But it probably never was the case and it certainly cannot be the case now.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. They do different things in different ways for different purposes.ËÜjf14ÝÜcf2ÝÜcf4ÝDr Nick Harris:Ücf1Ý I wonder if I could come in here, because I think we have to take a couple of examples. Essentially we are talking about a set of steps that build up to this classification and this assumption that all first class degrees are the same. No, of course they are not. All of you come from different backgrounds. Different students are following different programmes and in different universities, and the purposes of those programmes and the needs of the employers who employ those students are quite different. For example, a physiology student and a philosopher would not expect to be on the same type of programme and be assessed with exactly the same criteria. There is a different balance between theory and practice. Some will have more theory, some will have more practice. Exactly the same might be said of someone with a classics or with an agricultural degree: you would not expect the same criteria to be being used and the same outcome but a first class agriculturalist in the agricultural environment should, I would have thought, be recognised, as would a first class classicist in their own environment.ËÜjf13ÝÜcf3ÝQ71 Dr Evan Harris:Ì I do not think any of us have been questioning the distinction between subjects. It is the same subject. For example, if there was the doubt that there is about this at A-level, if universities or employers just did not know how to deal with whether an A from one examination board was easier to get to for a given standard than an A from another, there would be outrage, even if the language used was less than the language you have used in your report_and I do not blame you for saying what you need to say_would there not? I do not understand why there is the difference because, as I understand it, the percentage of A-level work funded by the taxpayer is the same, once you account for private schools and private tuition, as the percentage of higher education funded by the taxpayer. I do not understand why university autonomy, which is what is cited_which is to protect from political interference over course content_should be used as the reason for not being worried about ensuring that the classifications one would get are meaningful.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý The research evidence seems to suggest that that is the way it is and that is why we think it is not a good system or one of the reasons that we think it is not a good system.ËÜjf13ÝÜcf3ÝQ72 Dr Evan Harris:Ì Do you accept that university autonomy is not a good enough reason for the taxpayer, who has an 80%, let us say, stake in funding this, not to be concerned about the fact that, at best, there is doubt about the comparability of degree classifications from different institutions for the same subject?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think the taxpayer should take comfort from the fact that this has been identified by the higher education community itself as a concern and is doing something about it.ËÜjf13ÝÜcf3ÝQ73 Mr Wilson:Ì I think we are grappling towards an all-embracing question. From your long experience in higher education, are degree standards higher or lower now than they were, say, 10^20 years ago?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think it depends on how you define standards there. It sounds like a weasely answer, but it is not meant to be.ËÜjf13ÝÜcf3ÝQ74 Mr Wilson:Ì It does sound like a weasely answer. You are responsible for quality assurance in the sector.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Perhaps I could ask you how you are defining standards in that question.ËÜjf13ÝÜcf3ÝQ75 Mr Wilson:Ì You define the standards, do you not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We do.ËÜjf13ÝÜcf3ÝQ76 Mr Wilson:Ì That is your job.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We do, and we define them very precisely. I will tell you how we define them. For our operational purposes, we define academic standards as predetermined and explicit levels of achievement which must be reached for a student to be granted a qualification.ËÜjf13ÝÜcf3ÝQ77 Chairman:Ì The point I made earlier: the individual institution decides that very point, the criteria by which they award.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. That is their prerogative as autonomous institutions.ËÜjf12ÝÜcf3ÝMr Wilson:Ì That could be as low as they like.ËÜjf12ÝÜcf3ÝMr Boswell:Ì Above a threshold.ËÜjf13ÝÜcf3ÝQ78 Mr Wilson:Ì Above a minimum threshold.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Above a threshold.ËÜjf13ÝÜcf3ÝQ79 Chairman:Ì When University M says, We have to get more firsts in order to move up the league tables to attract students" they can arbitrarily do that with absolute ease.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is the legal state of affairs, yes. But, of course, if they do not have firsts, there is nothing for them to do. They cannot do it.ËÜjf13ÝÜcf3ÝQ80 Mr Boswell:Ì In a sense, you are suggesting we disarm this arms race which has taken place by banning the essential component, the degree classification.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is one consequence of it_one beneficial, in my view, consequence.ËÜjf13ÝÜcf3ÝQ81 Mr Boswell:Ì To go on from that slightly_because I think we are going to get into what one might call the politics of this in a minute_could we get a view from you as to whether all this has happened, as it were, objectively and in the air? Because there are highly complex conceptual issues, like what a first class degree would be, and, in a sense, the diversity of the system is broken because the challenges are now too great. Or is it in fact a more sordid process, where one or two persons have_as some people in the press are suggesting_manipulated the system to pump up their institution and will indirectly destroy public confidence. Is this an academic argument or a cynical manipulation that is going on_not by you, but by those you identify?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I would like to start with the academic argument, because I think there is an academic argument here. The academic argument is that, essentially, over the last 15 to 20 years, higher education has moved from what one might describe as a norm-referenced classification arrangement to a criterion-referenced classification arrangement; that is to say, that hitherto or previously it was common for quotas to be applied to graduates from year to year, course to course.ËÜjf13ÝÜcf3ÝQ82 Mr Boswell:Ì With 13% firsts.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is right, we would expect to see 5%, 6% or 7% firsts or whatever. There tended to be quite a nice normal distribution curve. Over the last 10 or 15 years we have moved away from that which, in a sense, gave no account of how good you were as a student, merely where you were in the rank order of the cohort of that particular year, to what we might call a criterion-referenced system where the question is: What have you achieved? That does allow for improvement in standards. As students are learning more and being able to do well in their assessments, so that will be reflected in the outcome. I am fairly sure that is part of the reason_and of course institutions will not tell us. We do not have direct evidence yet of what you have described as the more sordid approach. If we were to encounter that, we would want to know precisely why and how a university was operating the decision, essentially, to go back to a norm-reference system but a norm-referencing against other institutions rather than within a class.ËÜjf13ÝÜcf3ÝQ83 Dr Evan Harris:Ì Is the data available_it must be_of the percentage of, say, firsts for any given subject across each institution?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ84 Dr Evan Harris:Ì It is on your website.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes, HESA produces it.ËÜjf13ÝÜcf3ÝQ85 Dr Evan Harris:Ì You would expect, would you not_or am I wrong?_that a university with significantly higher entrance standards and decent education would have a much higher proportion of first class degrees awarded than a university with lower entry criteria and the same decent standard of education?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. That is what the evidence suggests.ËÜjf13ÝÜcf3ÝQ86 Dr Evan Harris:Ì You said universities were sorting this out. Diana Warwick, the Chief Executive of the higher education representative body, Universities UK, is quoted in the BBC story as saying Universities are already debating the classification of degrees". Is debating" higher education talk for sorting it out"? Or is it the same as we use here, which is just talking about it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý One of the reasons I am keen to discuss this is to ensure that the progress is continued. It is being implemented.ËÜjf13ÝÜcf3ÝQ87 Dr Evan Harris:Ì It is being more than debated.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý More than debated. The debate has happened.ËÜjf13ÝÜcf3ÝQ88 Dr Evan Harris:Ì After your press release and the news coverage, what contact did you have from DIUS, from the Government Department?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý DIUS wanted me to explain to them what the import of the press release and the BBC online account was. I explained to them the circumstances.ËÜjf13ÝÜcf3ÝQ89 Dr Evan Harris:Ì And that was it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý They thanked me.ËÜjf13ÝÜcf3ÝQ90 Mr Wilson:Ì Could I come back, because we did not get to an answer to my question as to whether you thought standards have gone up or down. I know we have talked about the individual awarding powers. In your experience, taking into account the conversation we have just had, have they gone up or down?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I do not know. What I can say is that they are appropriate for today_just as the standards of 1890 would not be appropriate today.ËÜjf13ÝÜcf3ÝQ91 Mr Wilson:Ì Do you not think it is a little bit worrying that you, representing your organisation, do not know whether standards are going up or down.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý The definition that we use for standards does not allow the question of whether they go up or down. Standards are arbitrary. Standards are fixed. If you are saying have they changed in some places they may have done, in other places they may not have done.ËÜjf13ÝÜcf3ÝQ92 Chairman:Ì They are only fixed by an individual institution.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes, but they have to be fixed by that institution; that is to say, the standards are there, that is our standard.ËÜjf12ÝÜcf3ÝChairman:Ì That is not the question that Rob Wilson is asking you. He is asking you how can we determine whether, overall, the cost of X plus Y is the number of students now getting firsts or upper seconds compared with what they were doing ten years ago. The question is: Does that mean that standards in higher education have risen?ËÜjf13ÝÜcf3ÝQ93 Mr Boswell:Ì I do not wish to obfuscate the question, because that is very pointed, but I think it is also necessary to ask, because we need to get feedback from you: Within all the constraints that we all understand about academic autonomy, is there at least some merit in considering whether there should be an objective standard? You say that it is not a question you can answer because there is no common standard of standards. Ought there to be a common standard so you could answer the question?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý To a proper common standard would apply a national examination. A national examination, because otherwise you would always be comparing different_ËÜjf13ÝÜcf3ÝQ94 Mr Boswell:Ì You would never be able to work out whether it had met the standard or not.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Exactly. A common standard would also undermine fundamentally the diversity of the system.ËÜjf13ÝÜcf3ÝQ95 Mr Wilson:Ì What is your personal opinion? Not your opinion in the position as head of this organisation, but your personal opinion of standards and whether they have risen or declined?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý My personal opinion_and it is a personal opinion_is that there has been some raising of standards. How much, I do not know. I think the students of today are very concerned indeed about the consequences for them of their degree class and are working hard to ensure they get the best possible level of degree. That suggests to me that standards have risen or at least have not significantly fallen.ËÜjf13ÝÜcf3ÝQ96 Mr Boswell:Ì Are you able to give us a similar appraisal as to the differentiation of standards? We have established there is no single standard, but clearly there are a lot more institutions offering degrees which, in a sense, are making a claim by doing so. Is it your impression that the spread has now changed? Some of the serious players and the ones who have a huge reputation in place are going up, shall we say, and some of the others who are dependent on recruitment and clearing and whatever might be saying, Oh, well, we need to keep bums on seats and show that we can function as a viable institution." If you are broadly happy about the system, are you worried that nevertheless we are creating more, if not defaulters, people who are giving rise to concern?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý One of the sets of figures I have is that which talks about the award of firsts and upper seconds according to type of institution. I am hesitant about saying this because it does give the impression that there is, if you like, a league table of types of institution and the degree classification system is almost a self-fulfilling approach. It does show, however, that there is a clear differentiation between, if you like, groups of institutions in the different groupings, self-created groupings, the clubs, and that there is only one group, I think it is fair to say, which in the last five years has shown any major increase in the award of firsts and upper seconds. The others have remained pretty static.ËÜjf13ÝÜcf3ÝQ97 Dr Evan Harris:Ì Which one is that?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý That is the 1994 group, and there is quite a differentiation between them.ËÜjf13ÝÜcf3ÝQ98 Dr Evan Harris:Ì Let us say that universities were dependent for their income and therefore their viability on the number of students that they accepted, and therefore, indirectly, the number of students who applied, and that there was evidence that students applied to those universities where they thought for any given standard they had a better chance of getting a good degree classification, and there was on the internet information available that told them which was the university which gave them the higher chance, all other things being equal, because it gave more upper second and first class degrees, if all those things were in place_and I believe they are_would that not be bound to influence the behaviour of universities? It is bound to.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It might well do. I am not sure I am necessarily the right person to ask that. I would say, however, that we would wish to see, as we undertake our audits, how institutions manage their admissions to ensure that there is not, if you like, one of those forces negatively influencing the other.ËÜjf13ÝÜcf3ÝQ99 Dr Evan Harris:Ì Do you look at that or is that someone else's job to look at?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is one of the things we can look at. You must appreciate in our audits there are a number of things we always look at and there are other things we look at if we see reasons for asking those questions. We cannot cover everything, or we would be there all day every day, so we have to be selective in what we look at. But it is an area we are certainly able to look at, partly because it is an area covered by our code of practice on admissions. Our code of practice on admissions offers a reference point to institutions as to how they should manage their admissions process.ËÜjf13ÝÜcf3ÝQ100 Mr Boswell:Ì Can I probe the other side of this? I think the inference of what has been said, and it is the whole philosophy that you started with, is that institutions have a professional academic responsibility themselves, and you can advise them on that, you can draw attention to deficiencies, and the external examiner and peer review system should help with that. Is it your impression that in reality, if X gets a really bad review and there is a suggestion of academic padding, if I can put it that way, the institution will grip that, if only because other VCs or Universities UK are saying, You really cannot go on doing this kind of thing", quite apart from any pressure you put on? Is there a sort of academic name and shame issue there? Can I couple that with a question about between the departments? My suspicion, and you may want to respond to this, is that this is unlikely, though there could conceivably be a vice chancellor instructing all departments to whitewash more people, but it may well be that one or two departments are anxious about their recruitment and are anxious to fill up their numbers rather than the whole institution? If this is essentially an academic problem which they have to discharge what peer pressures and sanctions are there, maybe by way of discreet words and raised eyebrows, that are likely to drive some remediation if these situations arise?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I do not think I am well qualified to answer your question in terms of what vice chancellors talk about when they are in huddles together. My guess_and it is a guess; I do not know_is that they do not spend a lot of time saying, You are letting the side down". When it comes to your second question, this is an area where I think we want to do more work because it is conjecture. At the moment there is not sound evidence one way or the other, but that is an area where I think we do need to do some specific inquiry, in a sense, to follow up some of the comments that have emerged from the press coverage, because there has been a lot of noise emerging and it is too much noise simply for us to ignore.ËÜjf13ÝÜcf3ÝQ101 Chairman:Ì Absolutely.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We intend to follow this up and try and find out whether there is a problem or whether it is noise and, if it is, what is the nature of the noise. I find it impossible to believe that vice chancellors are handing down edicts to the whole institution because edicts like that cannot be kept secret. Audit trails will emerge. If there is a difficulty, and, as I say, we have only got hearsay evidence so far, most of it anonymous, my guess is that it would be lower grade folk fairly far down the institution but, as I say, that is a guess. We need to have that evidence.ËÜjf13ÝÜcf3ÝQ102 Dr Evan Harris:Ì Can I turn to the question of international students? Do you accept that some universities are entirely dependent for their financial viability on attracting international students?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Given the percentage of students the departments receive I think it is probably true of some of them.ËÜjf13ÝÜcf3ÝQ103 Dr Evan Harris:Ì Given that, is it possible to be certain or even confident, or even limitedly confident, that an institution that depends for its viability and therefore all the leaders in it for their mortgage payments on getting a supply is likely to be influenced primarily by how it maintains that supply and resource to keep itself going?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý They have to have an eye for the long term.ËÜjf13ÝÜcf3ÝQ104 Dr Evan Harris:Ì We would like them to have an eye for the long term.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think they have, and I think that they are aware that if they get a bad reputation their overseas market could be affected, so it is not in their long term interests to deliberately act in that way.ËÜjf13ÝÜcf3ÝQ105 Dr Evan Harris:Ì Yes, I agree with you. Let us accept what you say. The bad reputation might be not awarding degrees to international students who did not pass muster whether or not it was because of the language. That might, in this world of websites and so forth and overseas students who know exactly what they are doing and who are spending a lot of money, mean that they would be very interested in knowing whether they are wasting their money. That is what you mean, is it not, that they are under huge pressure to award degrees to meet the expectations; otherwise they will get a very bad reputation in the medium to long term and therefore would lose the attraction? That is what you mean?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No, that is not what I mean. What I mean is that if they follow the route that you are suggesting, of pushing people through to get degrees, it does not take long for that to be known and for the credibility of the qualification to be undermined.ËÜjf13ÝÜcf3ÝQ106 Dr Evan Harris:Ì What makes you think your scenario of poor reputation is more immediate, more of a threat, more obvious than what I have just said, which I put to you as a reasonable case, that they would be torpedoed by their reputation abroad as someone to whom you pay all your money and you do not get your degree?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We have not seen either scenario in reality.ËÜjf13ÝÜcf3ÝQ107 Dr Evan Harris:Ì Has any research been done that you know of into which of those is more likely?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Not that I am aware of, no.ËÜjf13ÝÜcf3ÝQ108 Dr Evan Harris:Ì Do you think there should be research, because it is a good question, is it not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ109 Dr Evan Harris:Ì I always ask this question.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think there should always be more research into things in general.ËÜjf13ÝÜcf3ÝQ110 Dr Evan Harris:Ì Who should do that research?ËÜjf14ÝÜcf2ÝÜcf4Ý Mr Williams:Ücf1Ý There are a number of organisations that might do it.ËÜjf13ÝÜcf3ÝQ111 Chairman:Ì But, Peter, in your interview with the BBC on 23 June, this infamous or famous interview, you said that in terms of international students there were concerns on the part of audit teams that the number of international students was being increased in an unsustainable fashion in some institutions; yet when we look through your report Üdc9Ýthe words unsustainable", and indeed sustainable",Üdc0Ý do not appear at all.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No. That was a gloss.ËÜjf13ÝÜcf3ÝQ112 Chairman:Ì But it is an important gloss, is it not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes, it is.ËÜjf14ÝÜcf2ÝÜcf3ÝChairman:Ì Because the accusation is, and you know exactly what Evan Harris has said_ËÜjf13ÝÜcf3ÝQ113 Mr Boswell:Ì It also implies that you would know what a sustainable rate would be.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think what we are interested in are the ways in which institutions are managing their growth. There are two issues here, I think. One is the way in which they recruit their overseas students and the other is the way they look after them when they are here. It is quite clear from our audit reports and our outcomes report that they do a lot for the students when they are here. The question that crosses my mind, and the one which I think we all want to be pursuing more, is do they have to do more than they should be expected to when they are here? In particular I am thinking of English language capacity. Is there a question_and I am not saying there is; it is an open question and one we need to investigate_that students coming here are having to have a lot of language support because the institutions' ways of ensuring that the students have appropriate English language capacity before they allow them in are not acting as effectively as they might be? We have seen one or two instances of that possibility but we have also seen, and I bring to your attention the report produced by UKCOSA, which is now UKCISA, called Ücf2ÝBenchmarking the Provision of Services for International Students in Higher Education InstitutionsÜcf1Ý, where it talks about the ways in which institutions are dealing with the requirements of English language both before and at the acceptance stage and then subsequently when the students are in the UK.ËÜjf13ÝÜcf3ÝQ114 Chairman:Ì There is a fundamental issue here, Peter, and that is that in your interview with the BBC, which set a lot of these hares running,_and I am not saying that is a bad thing at all because we would not be having this debate if you had not said those things_you did make these comments about sustainability and you also indicated that there was a widespread problem with agents and that there was this view that if they pay their fees they get a degree, and that is the most damning indictment of the overseas student market where #3.7 billion comes into the UK as a result of that.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý In what the BBC reported me actually saying I think I used the word might". I am not saying this is the case. There have been problems with agents; that is well known, and those problems were so well known that the British Council decided to start a course for agents in order to raise the standard of agents. Agents are and have been a problem. What we are saying is that a problem agent is a serious problem for an institution and we are just alerting them to the need to ensure that the agents they use know what they are doing and are under proper control. That is an important message to give to the sector.ËÜjf13ÝÜcf3ÝQ115 Mr Boswell:Ì Thank you. I am glad that the agent issue has come up, but can I just build on it to ask your perception of a slightly different matter, and we could spend a second inquiry on it but we will not, and that is the comparability between UK sourced degrees and degrees which are franchised abroad? I do know that separately you go and you look at British higher education provision in Malaysia, for example. I also know the institution I refer to, of which I am a board member, offers degrees on its own account, degrees through an associate college in London, and is increasingly going to offer degrees offshore in the Far East. I happened to see the graduates there only the other day and was very interested in the large participation which seemed very positive. Are you satisfied (a) that you can make sure that a degree which is offered in place A is of the same value as place B, whichever those are, and (b) that in effect institutions are not creating situations where there is a soft option that people will take up in one place? Can you possibly also say, if you can draw on your experience when you are doing a country audit, There are bad agents performing here", and that you will get somebody_not yourselves_to do something about that?ËÜjf14ÝÜcf2ÝÜcf4ÝDr Stephen Jackson:Ücf1Ý As you say, we have an audit methodology here which looks at the quality of standards of provision by institutions operating overseas. If you look at our website, at the report and the links between UK institutions and partner organisations_and in the past we have been quite critical of a number of institutions and the way in which they are managing those things because we recognise that there is a degree of risk associated with partnership arrangements_we have detailed in our code of practice how we expect institutions to manage those relationships and we have procedures for following up any identified issues which come out of our audit activity, so we are very conscious of the reputational risk associated with this type of activity.ËÜjf13ÝÜcf3ÝQ116 Mr Boswell:Ì Because it is contagious as well, is it not?ËÜjf14ÝÜcf2ÝÜcf4ÝDr Stephen Jackson:Ücf1Ý It is the bad apple argument, is it not? If there is evidence of poor practice for one particular link it does have implications for other institutions as well.ËÜjf13ÝÜcf3ÝQ117 Mr Wilson:Ì Can I bring you back to this notion of sustainability because I do not think you really got to grips with what you believe is sustainable? By saying something is unsustainable it is suggesting that you somehow have a measure of what is sustainable and what is unsustainable for an institution. Can you be more specific on what you mean by sustainability"?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. What I mean is that an institution which would be predominantly populated by overseas students would have a quite different dynamic to it from one that did not. Of course, we all fervently want to see overseas students in the UK in higher education institutions, they do nothing but add lustre to them, but I think we have to be careful that we do not by chance find ourselves in a position where we are essentially operating overseas universities in the UK, unless that is what the university wishes to do and unless it recognises the needs and demands that that will create for it. My concern is that recruitment without consideration of the effect of recruitment of overseas students on the overall dynamic of an institution is likely to create its own difficulties and I think we have to be careful about that. Things are sustainable if proper protection is put in place to ensure that the standards and quality_and I think here we are talking as much about quality as we are about standards_are not to be jeopardised.ËÜjf13ÝÜcf3ÝQ118 Mr Wilson:Ì When you are talking about sustainability you are talking about the dynamic of an institution, the culture, the ethos?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ119 Mr Wilson:Ì It is not the financial viability of the institution that you were referring to when you were talking about sustainability?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No. I tread very warily in the area of finance.ËÜjf13ÝÜcf3ÝQ120 Mr Wilson:Ì Why do you think that is something that your organisation should be commenting on? Is it not a decision for the individual institution about where they want to take their students from?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Of course it is, but one of our jobs is, if you like, to provide warnings, to say, Look: be careful", to act as a kind of collective conscience for the sector so that through publications like the outcomes report we can send messages about things which are not big problems yet, and we hope they will not become problems_And if you are careful they will not become problems, but do please look for these things, do please be careful".ËÜjf13ÝÜcf3ÝQ121 Mr Wilson:Ì And you feel that is more your role than any other particular body's role?ËÜjf14ÝÜcf2ÝÜcf4Ý Mr Williams:Ücf1Ý Yes, because we have access to the information which can raise those warnings.ËÜjf13ÝÜcf3ÝQ122 Dr Evan Harris:Ì I just want to come back if I may to something which the Chairman asked about the market in international students because I think he said that there is a market which is worth #3.7 billion and it is supported and maintained. Is market" the right term, because my understanding of a market is that you pay some money, the market decides to provide you and you get what you want, and if you do not get what you want then the market is not working? It should not be called a market, should it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams: Ücf1ÝMarket" is now used widely in higher education and one has to be rather careful about_ËÜjf13ÝÜcf3ÝQ123 Dr Evan Harris:Ì Not by me it is not.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No, no; you are the exception perhaps that proves the rule. What is offered in higher education is an opportunity, or a series of opportunities, and that I think is what has to be made clear to everybody. That is not just to do with overseas markets; it is to do with home students as well. Universities offer an opportunity to learn and to be assessed on your learning.ËÜjf13ÝÜcf3ÝQ124 Dr Evan Harris:Ì The reason I do not call it a market is that it implies there is no non-market force either present or that ought to be present, and that if people just see it as a market it will just be, We will offer this, you will get a degree, we will meet the demand and you will be satisfied". My final question is about this issue of autonomy because the argument has been used that there should not be any greater regulation, regardless of how well you do, of standards in higher education institutions because, although they are mainly funded by the taxpayer, they are autonomous, they are independent. Therefore to have more rigorous, more intrusive regulation of their standards (although I would share their view in respect of academic freedom) would threaten their autonomy. My question to you is, does not crying autonomy" also imply freedom from appropriate questions as well, like financial questions? Because I do not understand why taxpayers' representatives expressing an interest in standards is more of a threat to autonomy than the fact that universities have to rely on getting international students in to keep them alive, to keep them viable? I would like you to comment on whether you think autonomy and independence are an adequate response from the sector to suggestions that there ought to be more regulation of their standards.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think you have to look at what regulation might look like and whether or not regulation would help or hinder. My view is that significantly more regulation would hinder because it would, if you like, encourage a compliance approach, which is the death knell of innovation. I think the compliance model for regulation is one where people teach to the test.ËÜjf14ÝÜcf2ÝÜcf3ÝChairman:Ì It sounds very familiar.ËÜjf14ÝÜcf2ÝÜcf3ÝMr Boswell:Ì Tick boxes.ËÜjf13ÝÜcf3ÝQ125 Dr Evan Harris:Ì Let us take medical research just as an example. It is heavily regulated but very innovative. You can have innovation with regulation because regulation breeds confidence and enables people to think that they are not wasting their time in an arbitrary system because there is confidence in the fact that they are going to be treated fairly and rewarded fairly in their work or study.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes, but I think the regulation of research is different from the regulation of academic standards and teaching alone.ËÜjf13ÝÜcf3ÝQ126 Dr Evan Harris:Ì Is it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think so. I think it is a different neck of the woods.ËÜjf13ÝÜcf3ÝQ127 Dr Evan Harris:Ì So what is beneficial in one is detrimental in the other? Why?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Because the nature of the exercise is different. It is a different task.ËÜjf13ÝÜcf3ÝQ128 Dr Evan Harris:Ì Yes, I know it is a different task, that is self-evident, but what is so different about it that it means that what is beneficial in one area is detrimental in another?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I think you have to go back to what teaching and learning in higher education are for. It is about providing opportunities for students to learn. It is about the development of knowledge. It is about the conservation and transmission of knowledge.ËÜjf13ÝÜcf3ÝQ129 Dr Evan Harris:Ì It is the same in schools but we regulate schools. Do you think it is an argument for deregulating schools?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý No, because we are talking here about organisations that have the powers to make awards. Schools do not have those powers. Universities do. That makes them really quite different in kind.ËÜjf13ÝÜcf3ÝQ130 Dr Evan Harris:Ì Okay, but it is sufficient to say that any greater checking up on whether what they are doing is appropriate would harm them?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Let us look at where that does happen. That happens_or happened_over most of Europe where there was a national curriculum. The degrees were owned by the state and the institutions essentially taught to the agreed national curriculum. The countries of Europe are moving at a huge rate. They are galloping away from that model because it has held them back and dragged them down. They have welcomed the Bologna Model because it offers them much more flexibility, much more self-reliance, much more opportunity to innovate and to develop their own self-confidence.ËÜjf13ÝÜcf3ÝQ131 Dr Evan Harris:Ì They might not like it but there are other people involved here, are there not? Obviously, everyone wants themselves to be less regulated, do they not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý They do.ËÜjf14ÝÜcf2ÝÜcf3ÝDr Evan Harris:Ì I am just saying overall.ËÜjf14ÝÜcf2ÝÜcf3ÝChairman:Ì I think we have perhaps come to a stalemate on that.ËÜjf13ÝÜcf3ÝQ132 Mr Wilson:Ì Can I say that I am very pleased that you have raised the issues that you have. You have created a very important debate in a number of areas, but the discussion of grade inflation, worries about too many overseas students and concerns about standards might create a negative impression overseas and I just wonder whether you think that what you have said will have negative repercussions for our universities overseas.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý I sincerely hope not because I think what this does, what we do, what you do, shows just how seriously the UK takes its standards. We could not have this discussion in most countries of the world. What we can do here is open them out and say, Look: we care deeply about our standards. We care deeply about our higher education. We want to make sure that everybody takes this at a level of seriousness which goes well beyond the UK", and that is what I think we do and that is what I think the work of the QAA helps to do, but it is also immensely valuable, as I said at the beginning, that a committee like yours is willing to engage with these questions because it adds to the recognition of the seriousness of standards and quality, which is what we are all keen for. I sincerely hope that in the overseas world they will say, Wow! We can have confidence in the UK higher education because they are not taking this lying down. They are thinking about the issues, they are worrying about the issues, they are doing something about them".ËÜjf13ÝÜcf3ÝQ133 Mr Boswell:Ì Thank you for this. I think we have had a good rattle around some really difficult issues. There were some fine words and some fairly provocative remarks that you made in your report, and that is part of your job, but what happens now? Who is going to grip this and do something about it?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Let me just say that I think there are five things we want to do, partly in the light of our own work.ËÜjf13ÝÜcf3ÝQ134 Mr Boswell:Ì In parallel with that can you suggest, delicately, also that others may need to do with you?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes. This is certainly something we cannot do entirely on our own. We will need to call on others to do it. It is partly in the light of our own work and partly in the light of the noise in the system, the things that are coming out. The first thing we are going to do is try and make our causes for concern process more visible so that it is easy for people to find out about it. There is a downside to that because we are not resourced to deal with an avalanche, should an avalanche happen. I hope one does not come but we will cope with that if it does and if necessary we will seek additional resources from our funders. The second thing we are going to do is that we wish to extend our causes for concern process so that we do not just wait for complaints or concerns to come to us, but when we see, if you like, a critical mass of concerns in the ether we then take action to see whether or not there is something substantial or substantive in those concerns. The third thing we are going to do is develop an action plan to make inquiries in a series of these areas, and that is where we are going to need other systems.ËÜjf13ÝÜcf3ÝQ135 Mr Boswell:Ì And this might, for example, be the interdepartmental issues that we talked about?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý All of those key issues that are emerging. As I say, we are not set up as a research organisation so some of it we will be able to do ourselves, some of it we will need to go and commission others to do for us. The other thing we will do is that we will ask complainants, and you see if you can do this, to let us have the evidence because without the evidence we are really rather stuck. If we have evidence which is usable then we can take action, but our difficulty is that a lot of the noise in the system is not usable. We have analysed 335 responses to the media pieces in the last three weeks and of those we think 12% have sufficient evidence in them were we to get the real evidence.ËÜjf13ÝÜcf3ÝQ136 Dr Evan Harris:Ì You are not a research organisation but you can add to your list of things to do publishing your recommendations of what research needs to be done?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ137 Dr Evan Harris:Ì It has come up a couple of times.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý Yes.ËÜjf13ÝÜcf3ÝQ138 Mr Boswell:Ì We might like to have a note on that when you are ready.ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý We are about to go from this meeting to our board meeting in which we will be putting these proposals to our board.ËÜjf13ÝÜcf3ÝQ139 Chairman:Ì I think 12.5% is a fairly significant percentage to be taken up, is it not?ËÜjf14ÝÜcf2ÝÜcf4ÝMr Williams:Ücf1Ý It is 40-odd.ËÜjf13ÝÜcf3ÝChairman:Ì I think that demonstrates that something needs to be done. On that note could I thank you very much indeed, Dr Stephen Jackson, Peter Williams, Douglas Blackstock and Dr Nick Harris. I think it has been an incredibly useful session this morning. We would like as a committee to be kept informed of your next steps because I think we intend to keep a very close eye on this as we may well return to the issue of teaching and learning in universities because that is really the heart of what you have been saying. Thank you very much indeed this morning.ËÜjf90ÝÜjf22ÝÜteÜjf199ÝÜjf165ÝÜjf27ÝÜcf3ÝLetter from Peter Williams, Chief Executive, Quality Assurance Agency to theÊChairman of the CommitteeÌÜjf4ÝThank you for your letter of 8 October requesting detailed information about the progress of the enquiries into the press stories of the summer relating to academic standards. I should be grateful if you'd allow me until the end of the month to prepare a report on progress to date.ËÜjf4ÝI should like to take this opportunity to inform you about some other aspects of our recent work in this area. Following the Select Committee's hearing in July, and the attendant publicity, we were expecting to receive a large number of potential causes for concern. To meet this expected demand we produced a users' guide" and simplified access to the procedure on our website (http://www.Qaa.ac.ukicausesfor concernldefault.asp). Since then, however, we have received about a dozen letters and other communications from people with concerns, some of which have been referred to us by third parties. Most of them are personal grievances or relate to employment disputes, some relate to non-HE matters or to strongly held personal views on higher education in general, and only a very small number appear, prima facie, to justify any further investigation by us. These we are pursuing.ËÜjf4ÝWe have also begun our enquiries into the specific areas of concern that were mentioned in the media during the summer. We expect to have a progress report on these by December.ËÜjf4ÝWe shall, of course, send you a copy of this.ËÜjf4ÝFinally, I noticed at the July hearing that there appeared to be some misconceptions among the members of your Committee about the distribution of responsibilities for quality and standards in higher education and about the distinction between those two concepts; but that was not the occasion for a digressive exposition of these rather complex matters. If you and your colleagues would find it helpful, I should be very happy to provide a background briefing(lasting about 30 minutes) for your Committee, to describe, explain and clarify the quality assurance landscape and its main features and concerns. Please let me know if that would be useful to you and your colleagues.ËÜjf4ÝI look forward to hearing from you.ËWith best wishesËPeter WilliamsÈChief ExecutiveËÈÜcf2Ý14 October 2008ÈÜjf22ÝÜteÜjf199ÝÜjf27ÝÜcf3ÝLetter from Peter Williams, Chief Executive, Quality Assurance Agency to the Chairman of the CommitteeÌÜjf4ÝFurther to my letter of 14 October I am now writing, as promised, with more information about our follow-up to the various stories about academic quality and standards in higher education that appeared in the press during the summer and have been received by QAA since.ËÜjf4ÝThere have been three principal sources for these stories: direct approaches to QAA; material received from Professor Geoffrey Alderman; and the BBC. The direct approaches to QAA include redirected enquiries from third parties.ËÜjf31ÝDirect Approaches to QAAÌÜjf4ÝSince the start of July we have received 12 direct approaches in relation to academic quality and standards. I am attaching a table listing these, together with information about the nature of the concern, our analysis of it and the action we have taken. You will see from this that a number of these are personal grievances against institutions rather than systemic problems; others relate to employment disputes; and some reflect the complainant's personal, sometimes perhaps idiosyncratic, views on education. Having analysed carefully all these submissions, we have followed up four with the higher education institutions or the students concerned.ËÜjf31ÝCases Forwarded to us by Professor AldermanÌÜjf4ÝIn addition to the direct approaches to QAA, Professor Alderman, at our request, has forwarded to us six instances where correspondents have contacted him with stories about academic quality and standards, following publication of news stories about his inaugural lecture at the University of Buckingham. He has told us that he has received other communications, which he will not forward to us until he has obtained permission from the authors. We do not know how many of these there are, but believe it is probably a small number.ËÜjf4ÝOf the six communications forwarded to us by Professor Alderman, we have judged three appropriate for further enquiry. The other three we have excluded either because the concerns were very old or because they were general rather than specific. The attached table indicates the action taken in respect of the first three cases.ËÜjf31ÝMaterial from the BBCÌÜjf4ÝMost of the stories during the summer appeared on the BBC Online website. The great majority of these were unattributed or without any information that would allow us to follow them up (ie no higher education institution was named and no other usable evidence was included). Given the claims of the BBC journalist that hundreds of comments had been received by the website on this topic, we invited him, immediately following your Committee's hearing on 17 July, to send us all these accounts so that we could follow up any that were sufficiently specific. He advised that he might not be able to do so, citing the need to protect the confidentiality of his sources, the need to get prior consent from the complainants, and advice from the BBC lawyers about the nature of the allegations made. We have recently followed this conversation up with a formal request, but have not, as yet, had a response.ËÜjf4ÝIn the absence of this material we are unable to pursue any particular lines of enquiry arising from the comments made to the BBC. However, we are re-analysing the content of the previously published BBC readers' responses, using qualitative data analysis software, to see if there are any common themes or trends discernable.ËÜjf31ÝOther ActivityÌÜjf4ÝAs I informed you in my previous letter, we are also undertaking research into five areas where the interest in standards and quality has been concentrated: student contact hours, international students' English language capacity; international students' recruitment; external examining; and student assessment. We hope to have an interim report by early December, and to complete the project in the New Year. I shall keep you informed of our progress.ËÜjf4ÝPlease let me know if you require any further information or clarification of this letter.ËYours sincerelyËPeter WilliamsËChief ExecutiveËÈÜcf2Ý30 October 2008Ücf1ÝËÜemËÜpg0r,cwmem1ÝÜcc57p6ÝÜjf31ÝCases Received Through Direct ApproachesÌÈÜmp10ÝÜcc57p6ÝÜcp9.5,10.5ÝÜdt6p0,0p6g,7p0,0p6g,14p0,0p6g,14p0,0p6g,14p0ÝÈÜbtÜcf2ÝCase numberÜntDate of incidentÜntAlleged Cause for ConcernÜntInitial analysisÜntFollow up actionÜcf1ÝÜetÈÜbt2008.01EËEnglandÜntJuly 2008ÜntStudent was rejected for an MA in Islamic Studies due to age. Minimum age for entry was 23.ÜntUniversities are responsible for setting their own admissions criteria; however this case may breach the recently implemented age discrimination legislation.ÜntComplainant was referred to the University's internal complaints procedure and the Equality Challenge Unit.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.02EËËEnglandÜntSeptember 2005ÜntStudent was dissatisfied with dissertation marking practices. Informed that their dissertation would not be marked until a specified level of credits was achieved. Student believes dissertation was marked before this was the case.ÜntPersonal grievance and no evidence of widespread threat to academic standards.ÜntPersonal grievance, appropriate response sent.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.03WËËWalesÜntJuly 2008ÜntStudent complained about the leniency of marking. They did not believe that they should have passed their course and alleged that the University was deliberately passing students who should fail.ÜntQuestion of academic judgement on standards. No evidence of malpractice or systemic procedural failure. No substantive evidence.ÜntDetailed personal letter sent from Chief Executive.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.04EËEnglandÜnt2000ÜntStudent failed dissertation twice and decided to transfer to different university. First university would not provide credit for the two years of completed study.ÜntCredit systems are not compulsory and the university does not have a system in place.ÜntResponse sent from Communications.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.05EËEnglandÜntMay 2003ÜntComplaint also sent to HEFCE. Staff member alleged widespread plagiarism at the university.ÜntAllegation is eligible for further investigation; however the complainant signed a compromise agreement following a related Employment Tribunal that would put QAA at risk of prosecution if any evidence were solicited. Complainant has refused to submit a formal complaint unless protection against litigation can be guaranteed.ÜntResponse sent from Chief Executive to explain that no protection against litigation is possible. Advice sought from QAA's solicitor.ËÜcf3ÝAllegation currently with HEFCE.Ücf1ÝÜetÜel3ÝÜbt2008.06EËEnglandÜntAugust 2008ÜntMember of public declared there was no validity in teaching Modern Art as a subject.ÜntQuestion of academic judgement. Universities are autonomous and decide which courses and subjects to teach.ÜntPersonal letter sent from Chief Executive.ËÜcf3ÝCase closed.Ücf1ÝÜetÜbt2008.07EËEnglandÜntNovember 2007ÜntStudent was dissatisfied with examination arrangements.ÜntCollege is not an HEI and has no relationship with QAA.ÜntPersonal letter sent from Chief Executive. Complainant was referred to QCA.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.08EËEnglandÜnt1981^2007ÜntStudent complained that they had shown no improvement in recent marks at university since failing examinations at College of HE in 1981.ÜntFurther information is required. This is most likely a personal grievance.ÜntPersonal letter sent from Chief Executive.ËÜcf3ÝCase continuing.Ücf1ÝÜetÜel3ÝÜbt2008.09EËEnglandÜnt2004ÜntReferred from HEFCE. Member of university staff advised QAA that an external examiner for a Music course had been pressurised into changing their report.ÜntAllegation investigated through Causes for Concern procedure.ÜntPreliminary investigation undertaken. Allegation upheld, but effluxion of time and subsequent changes to university practice rendered further action unnecessary. QAA in discussion with University re publication of investigation report.ËÜcf3ÝCase continuing.Ücf1ÝÜetÜel3ÝÜbt2008.10EËEnglandÜnt2005ÜntAllegation that evidence was falsified for a QAA Foundation Degree review.ÜntFurther information is required. Director of Reviews has written to the Vice-Chancellor.ÜntWaiting for response from Vice-Chancellor.ËÜcf3ÝCase continuing.Ücf1ÝÜetÜel3ÝÜbt2008.14EËEnglandÜntNovember 2005ÜntStudent submitted a project in June 2005 as part of BA course. It is alleged that the work went missing until March 2007 and that the student never received a mark for this work. Student then deferred until a later date but did not fulfil the requirements of Stage 1 of the award. The course has changed extensively since 2005 and it is now not possible for the student to complete the qualification.ÜntComms referred the enquirer to the Office of the Independent Adjudicator for Higher Education as they had exhausted the internal complaints procedure with the institution and this is a personal grievance.ÜntEmail response sent by Comms.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt2008.15EËEnglandÜntSeptember 2008ÜntStudent enrolled on the Postgraduate level certificate in Teaching in Lifelong Learning" but has now been told that the qualification awarded is an Extra Mural Certificate", but the same course and the same level, and fully accredited.ÜntAppears to be confusion re nomenclature and status of qualification.ÜntClarification sought from institution re naming of qualifications.ËÜcf3ÝCase continuing.Ücf1ÝÜetÜmp40ÝËÈÜcc57p6ÝÜjf31Ý     Cases Received from Professor G. AldermanÌÈÜcc57p6ÝÜdt6p0,0p6g,7p0,0p6g,14p0,0p6g,14p0,0p6g,14p0Ý     ÈÜbt     Ücf2ÝCase 		numberÜntDate of incidentÜntAlleged Cause for ConcernÜntInitial analysisÜntFollow up actionÜcf1ÝÜet     ÈÜbt     2008.11EË     EnglandÜnt2006^08ÜntReferred by Professor G. Alderman. QAA was asked to re-examine responses received from a university re academic standards in the light of recent adjudication in linked Employment Tribunal.ÜntConsidered by QAA prior to notification by Prof. Alderman. QAA is confident that the university reviewed the circumstances of this case carefully at the time and revised its procedures appropriately. ET ruling subject to appeal.ÜntNo further action required.ËÜcf3ÝCase closed.Ücf1ÝÜetÜel3ÝÜbt     2008.12WËËË     WalesÜntJune 2007ÜntReferred by Professor G. Alderman. Member of staff was pressurised to revisit marks for course examinations.ÜntFurther information is required before action can be taken. No formal allegation has been made to QAA by the member of staff in question. Professor Alderman has been advised to ask the member of staff to contact QAA directly if they wish to make a formal complaint.ÜntWaiting for response from member of staff.ËÜcf3ÝCase continuing.Ücf1ÝÜetÜel3ÝÜbt     2008.13EËË     EnglandÜntAugust 2008ÜntReferred by Professor G. Alderman. A student who failed to get the required offer grades was refused discretionary entry to a degree course because the university would not recognise one of their A-levels, as this was assumed to be in the student's mother tongue. Allegation that the possible discounting of native language A-levels was not made clear to the student prior to application.ÜntFurther information is required. Student has appealed to the university.ÜntUniversity asked if this is standard policy, and if so, where it is publicly stated. Appears to be an isolated discretionary decision by an individual, but unsupported by explicit and published institutional policy. Further clarification has been sought, but QAA has since been informed that the student has made a formal written complaint direct to the university. QAA will therefore await the result of the internal investigation before deciding whether further enquiries are appropriate.ËÜcf3ÝCase continuing.Ücf1ÝÜet     ËÜpg0ÝÜte