Students and Universities - Innovation, Universities, Science and Skills Committee Contents

Annex A

The Quality Assurance Agency for Higher Education (QAA)

Note for the IUSS Select Committee from the Chief Executive, Mr Peter Williams

31 March 2009

  At its evidence session on 9 March 2009, a member of the IUSS Select Committee, Dr Evan Harris, asked me to provide a note on a comment I made to the effect that QAA, in considering the future development of Institutional audit, might wish to look at more primary, rather than secondary, evidence.

In their early days (1991-2001), academic audits scrutinised directly the policies and procedures that HEIs themselves used to assure their own academic standards and quality. This involved an analysis of the documents used internally for the purpose, such as policy statements, procedural requirements or guidance, and monitoring reports, as well as evidence of the policies and procedures in action, through individual case studies involving the random (or targeted) sampling of departments. Audit teams would "trail" a variety of themes through the selected departments, asking staff and students about their experience of the different procedures, and reviewing additional documentation (such as external examiners' reports) of the auditors' choosing.

  With the maturing and increasing effectiveness of institutions' own internal quality assurance mechanisms, and the advancement of both national and international approaches to quality assurance, the institutional audit process has evolved to place more emphasis and reliance on institutions' own self-evaluation of the effectiveness of their policies and procedures in providing reliable assurance of quality and standards. The task of external audit in these circumstances has been to investigate institutions' own claims, as put forward in their self-evaluations. While this has undoubtedly paid dividends in forcing institutions to reflect carefully on their own practice, it has nonetheless shifted the focus of audit enquiry from showing policies in action, to one where the institution's own narrative is the subject of the scrutiny.

  I believe that this movement has probably now gone as far as it usefully can, and take the view that the next sequence of institutional audits should look once again at practice on the ground, using the "primary" evidence of "live" documents and focused interviews at all levels, as well as institutions' own accounts of their practices (the "secondary" evidence).

  We have also begun to think through what should be the key areas for scrutiny in the next form of external review of quality and standards. If we adopt the more direct approach described in the preceding paragraph, then we could concentrate on, for example, the way in which institutions use the Academic Infrastructure (see Annex E)[379] to calibrate their academic standards against national norms; the way in which external examiners ensure comparability of outcome with other institutions; and the way in which the areas we have highlighted in our current thematic enquiries are being tackled by institutions.

  There are, of course, disadvantages in scrutinising intensively a small number of areas, albeit important ones. Given the limited time and money that are available to QAA to carry out its reviews, depth of enquiry would lead to the sacrifice of breadth in the coverage; we already have to decline requests from many interest groups who want us to include their particular specialism in our audit enquiries. The alternative would be longer and more burdensome reviews, which would, I have no doubt, give rise to complaints based on the expectations created by the many proponents of "better regulation", that external watchdogs should adopt a more risk-based approach and target their activities on "weaker" organisations (however these are identified).

  External quality assurance in higher education aims to provide a kind of "assay office" service, "hallmarking" institutions that are considered to be meeting the necessary standard. Thus, one can speak of QAA "assaying" all UK degrees at the minimum threshold nine carat gold level (less bullion content but more serviceable and useful than purer gold), while recognising that some institutions may be offering 18 or 24 carat products. At present we believe that all degrees meet the necessary minimum standard, but that does not prevent some institutions from awarding degrees at a standard above the minimum. The limitation of this system, as we have said on many occasions, is that currently only one set of simplistic descriptors (degree classes) exists to cover the academic achievements of a diverse and ever-changing student population undertaking a huge and heterogeneous range of studies.

379   Not published. Back

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2009
Prepared 2 August 2009