Annex A
The Quality Assurance Agency for Higher
Education (QAA)
Note for the IUSS Select Committee from the Chief
Executive, Mr Peter Williams
31 March 2009
At its evidence session on 9 March 2009, a member
of the IUSS Select Committee, Dr Evan Harris, asked me to provide
a note on a comment I made to the effect that QAA, in considering
the future development of Institutional audit, might wish to look
at more primary, rather than secondary, evidence.
In their early days (1991-2001), academic audits
scrutinised directly the policies and procedures that HEIs themselves
used to assure their own academic standards and quality. This
involved an analysis of the documents used internally for the
purpose, such as policy statements, procedural requirements or
guidance, and monitoring reports, as well as evidence of the policies
and procedures in action, through individual case studies involving
the random (or targeted) sampling of departments. Audit teams
would "trail" a variety of themes through the selected
departments, asking staff and students about their experience
of the different procedures, and reviewing additional documentation
(such as external examiners' reports) of the auditors' choosing.
With the maturing and increasing effectiveness
of institutions' own internal quality assurance mechanisms, and
the advancement of both national and international approaches
to quality assurance, the institutional audit process has evolved
to place more emphasis and reliance on institutions' own self-evaluation
of the effectiveness of their policies and procedures in providing
reliable assurance of quality and standards. The task of external
audit in these circumstances has been to investigate institutions'
own claims, as put forward in their self-evaluations. While this
has undoubtedly paid dividends in forcing institutions to reflect
carefully on their own practice, it has nonetheless shifted the
focus of audit enquiry from showing policies in action, to one
where the institution's own narrative is the subject of the scrutiny.
I believe that this movement has probably now
gone as far as it usefully can, and take the view that the next
sequence of institutional audits should look once again at practice
on the ground, using the "primary" evidence of "live"
documents and focused interviews at all levels, as well as institutions'
own accounts of their practices (the "secondary" evidence).
We have also begun to think through what should
be the key areas for scrutiny in the next form of external review
of quality and standards. If we adopt the more direct approach
described in the preceding paragraph, then we could concentrate
on, for example, the way in which institutions use the Academic
Infrastructure (see Annex E)[379]
to calibrate their academic standards against national norms;
the way in which external examiners ensure comparability of outcome
with other institutions; and the way in which the areas we have
highlighted in our current thematic enquiries are being tackled
by institutions.
There are, of course, disadvantages in scrutinising
intensively a small number of areas, albeit important ones. Given
the limited time and money that are available to QAA to carry
out its reviews, depth of enquiry would lead to the sacrifice
of breadth in the coverage; we already have to decline requests
from many interest groups who want us to include their particular
specialism in our audit enquiries. The alternative would be longer
and more burdensome reviews, which would, I have no doubt, give
rise to complaints based on the expectations created by the many
proponents of "better regulation", that external watchdogs
should adopt a more risk-based approach and target their activities
on "weaker" organisations (however these are identified).
External quality assurance in higher education
aims to provide a kind of "assay office" service, "hallmarking"
institutions that are considered to be meeting the necessary standard.
Thus, one can speak of QAA "assaying" all UK degrees
at the minimum threshold nine carat gold level (less bullion content
but more serviceable and useful than purer gold), while recognising
that some institutions may be offering 18 or 24 carat products.
At present we believe that all degrees meet the necessary minimum
standard, but that does not prevent some institutions from awarding
degrees at a standard above the minimum. The limitation of this
system, as we have said on many occasions, is that currently only
one set of simplistic descriptors (degree classes) exists to cover
the academic achievements of a diverse and ever-changing student
population undertaking a huge and heterogeneous range of studies.
379 Not published. Back
|
