Memorandum 52
Submission from the Executive Committee
of the Quality Strategy Network
SUMMARY
1. In summary, this submission argues that:
— The methodologies used by UK HEIs to
determine degree classifications are consistent and appropriate,
and that these are well monitored by QAA— The degree classification
system remains the best available means of summarising student
achievement
— Degree classification is a good indication
of portability, when supported by the Higher Education Achievement
Record (HEAR)
— Plagiarism is well understood by HEIs,
which have developed sophisticated means of combating this.
PREAMBLE
2. The Quality Strategy Network is the membership
body for senior quality managers within UK HE. This submission
has been prepared by the QSN Executive Committee on behalf of
the Network.
3. Given the particular experience and expertise
of network members, we have confined ourselves to commenting on
the questions relating to academic standards, which come under
the heading "degree classification".
INTRODUCTION: THE
HE SECTOR IN
THE UK
4. The UK HE system is very diverse. It
is essential that this diversity should be maintained if the sector
is to meet the challenges of an unprecedented participation rate.
Individual institutions within the UK sector have different missions,
enabling the sector to offer a broad subject coverage and encourage
participation from students from a wide range of backgrounds.
A homogenous system would not be able to deliver this range of
educational opportunities. Given this diversity, it is essential
that there should be one agreed and coherent national QA[163]
system, within the parameters of which all providers of HE operate;
it is noteworthy that this is the situation which currently prevails.
5. Even within institutions there may be a wide
variety of objectives and targets; and some of these may even
be contradictory (see HEFCE's submission to the Denham review
of HE). There is no objective measure of the "best"
institution; success cannot be measured in simplistic terms, such
as measures of financial performance or student achievement, but
can only be assessed in relation to the specific mission.
6. This diversity is essential if we are
to deliver the wide range of graduates which the future of our
country needs (see HEFCE Strategic Plan). Hence the most successful
graduates from business-facing universities, Million + or the
specialist sector may have a very different skillset from the
"top" graduates from the Russell Group. This does not
mean that all are not equally worthy of First Class Honours when
judged against the criteria set out for their particular award.
THE BACKGROUND
TO THE
SETTING OF
DEGREE STANDARDS
IN UK HEIS
7. While it is essential that individual
degree-awarding institutions should exercise their autonomy in
determining the detailed criteria for the award of degrees, they
do so within a well-established framework provided by elements
of the Academic Infrastructure, the development of which has been
one of the major achievements of the QAA. This infrastructure
includes the Framework for Higher Education Qualifications and
the Scottish Credit & Qualifications Framework, which set
out general criteria for the award of qualifications from sub-degree
to doctorate level; Subject Benchmarks, which recommend core subject
content for awards in particular disciplines; and the Code of
Practice, which details best practice in a range of aspects of
institutional management, from the appointment of external examiners
(who represent the longest established cornerstone of the system
for externally assuring standards, see paragraphs 20ff below)
to the admission of students. In addition, there has been a successful
emphasis on the provision of accurate information to students,
for example programme information and comparative information,
such as the National Student Survey (NSS), and quantitative data
provided through Unistats.
8. An increasing range of awards are also subject
to the scrutiny and requirements of a professional body. There
are presently some 900 accrediting bodies on record in the UK,
which exercise considerable influence on both curriculum content
and standards. To give just one example: any institution which
wishes to award professional degrees in Social Work must have
those awards examined by the General Social Care Council. Such
scrutiny not only provides objective external input into the development
and review of degree courses, but also helps to ensure an element
of consistency across the sector. Institutions have developed
systems to deal successfully with the complexity of integrating
these professional requirements with the common HE standards and
their own missions. Any attempt to implement a centralised approach
would have implementation costs out of proportion to its value,
and would be likely to lead to major issues with professional
accrediting bodies.
EFFECTIVENESS OF
QUALITY ASSURANCE
AGENCY IN
MONITORING DEGREE
STANDARDS
9. We are not aware of any evidence that
degree standards are inappropriate, or that standards are any
less secure than at any time in the past.
10. We have a mature quality assurance system
in the UK. It has been developed systematically over the last
two decades, and is now well-understood and well-established.
A rolling programme of Audits, Continuation Audits, Subject Reviews
and now Institutional Audits, alongside Enhancement-Led Institutional
Review (ELIR) in Scotland and Institutional Review in Wales, has
consistently given the sector a clean bill of health, with only
a fraction of disciplines or institutions found to be unsatisfactory.
These processes have taken place alongside HEFCE Assurance Audits,
and reviews conducted by bodies such as the Higher Education Regulation
Review Group (HERRG), which sought to reduce the administrative
burden on institutions. There may be further scope for improvement,
but the principles are now well accepted. The individual decisions
of autonomous institutions are not reviewed, but the full process
of decision-making is considered, to ensure that it is fit for
purpose.
11. This maturity supports the increasing
focus on the enhancement of student learning opportunities, and
the promotion of student engagement, which are the current priority
for all institutions. This is not simply a matter of listening
to student feedback and acting upon it, although that is standard
practice in all institutions with which we are familiar. It is
about how we engage students fully as participants in more aspects
of institutional life and decision-making. At the same time, there
is a focus on the student experience: how we ensure that time
spent within HE is both rich, and valuable and relevant to subsequent
employment and personal development. Discussions about "quality
enhancement", and how best it can be delivered, have been
the central focus of the two most recent QSN annual conferences,
each attended by around 100 senior quality managers from across
the UK HE sector.
12. Peer review is an essential aspect of
the national QA system; indeed given the complexity of the HE
sector, the variety of institutions and the competing internal
objectives, it is difficult to see who the expert reviewers might
be, if not current (or very recent) professionals within the system.
We have all seen the few alarmist reports—given undue weight
perhaps by the THE letters page—which suggest that nameless,
faceless bureaucrats require the completion of ever-increasing
mountains of paperwork, usually involving the meaningless ticking
of boxes. This is not a system we recognise, nor one in which
we work. Indeed QSN perceives a trend towards internal processes
which are more efficient and add value, and the Network is active
in promoting these.
13. QAA recruits, and trains, established
and experienced members of staff from within HEIs to perform the
audit function, and they are well aware of the challenges faced
by institutions, for example in identifying, implementing, and
measuring the success of enhancement initiatives. Nevertheless,
where an institution is found to be lacking, either in terms of
academic standards, or in the delivery of high quality education;
or in its processes to assure these, a judgement of less than
full confidence is delivered. Our members have wide experience
of this system in practice; based on that experience, we believe
it to be a sound system which delivers reliable judgements. Indeed
we are concerned that any other process—for example one
which included more tick-boxes—would lead to universities
from which creativity and risk-taking, as well as frank debate,
were eliminated. The dynamism of the sector would thereby be compromised;
and the sector, and above all its students, would be the poorer
for this.
14. QAA has taken many years to reach its
current position, which provides assurance of, and public information
about, the standards and quality of provision across the HE sector.
It does this without infringing upon the core concept of the autonomous
institution. We fully accept the importance of making information
about the outcomes of QA processes available within the public
domain. We do retain some misgivings about the value of this information,
as currently published, to non-expert users, who are not well
placed to interpret the differences which may be perceived, and
we welcome the initiatives which are currently underway, for example
via the work of UCAS through the Unistats website, to provide
information in a form more accessible to those outside the sector,
such as students, prospective students, parents and employers.
15. We note that the UK's approach to quality
assurance has had a significant influence on developments elsewhere,
including across Europe through the Bologna Process and in jurisdictions
such as Malaysia and Saudi Arabia. We also note the recent confirmation
by the European Association for Quality Assurance in Higher Education
(ENQA) of QAA compliance with agreed European standards. This
is an indication of the esteem in which the system is held.
CURRENT METHODS
OF STANDARDISATION
16. We are of course aware of the debates
about academic standards, and the assertion that these have fallen
over recent decades.
17. We acknowledge the proportions of students
achieving higher classifications of degree (First and Upper Second
Class Honours), and understand public surprise that this proportion
has risen given the massification of HE. However, we note a number
of points which may have impacted on this:
(i) The onus on widening participation has enabled
a previously untapped pool of talented students to enter HE who
would not have done so hitherto.
(ii) The range of subjects available has greatly
increased. This has drawn in a range of professionals whose skills
were not previously acknowledged to be at degree level, but whose
competence was not in doubt. Hence the comparison over time does
not compare like with like.
(iii) This range of subjects has also increased
the range of skills and competences which are expected. For example,
students may be assessed on their achievement of practical outcomes
(such as through work placement), rather than on scholarly essays
or extended dissertations.
(iv) Institutions are much more explicit about
the learning outcomes which are to be demonstrated, and the assessment
criteria to be applied. Even 20 years ago, this was almost unknown.
Hence students are clearer about what they need to do in each
piece of assessed work, and can target their efforts accordingly.
(v) Institutions often use a range of assessment
methods. Hence whilst previously an ability to succeed in written
examinations, based on the academic essay, was of paramount importance,
students may now achieve well by being strong in coursework, presentations,
and so on. Many of the skills and abilities tested in this way
are of equal relevance to a student's future contribution in employment
than were the narrower range of skills tested by more traditional
methods.
18. Hence we question the assertion that
standards have fallen over time, as the respective systems are
so different that they are not susceptible to such a simplistic
comparison.
19. Institutions have internal mechanisms
to ensure the standardisation of marks in relation to the national
expectations of degree-level study. These may include some or
all of double marking; moderation; the use of generic grading
descriptors to articulate the expectations at each class of award;
and comprehensive validation and periodic review processes, drawing
on views from colleagues external to the institution, to ensure
that each course has outcomes which are appropriate to a degree
in this subject.
20. This internal decision-making about
student performance is validated by the external examiner. External
examiners bring professional experience and objectivity to bear,
and have a key role in confirming that the marks and grades awarded
are appropriate, consistent across institutions within the sector
and over time, and in accordance with the regulations. No work
will be outside the remit of at least one external, and we are
not aware of any evidence that the system does not work well.
We do not believe that there is a cadre of expert professionals
available to do this work other than those currently serving as
external examiners, and the commitment which senior academics
demonstrate in acting as external examiners (for limited reward)
shows the seriousness with which this role is taken.
21. QAA auditors do not interview external
examiners. However, they have access to their reports; and can
review the criteria for the appointment of externals; can check
any institutional training or induction; can read any guidelines
or procedural notes; and can check how an institution has responded
to any or all annual reports. Failure to engage appropriately
with external examiners would almost certainly lead to a judgement
of limited confidence at Institutional Audit or equivalent.
22. We note that there was a QAA proposal
in 1998, to develop a national Register of External Examiners
(in response to the 1997 Dearing Report), and that this suggestion
has recently been resurrected by Professor Roger Brown, now of
Liverpool Hope University. We fail to see the benefits of this
proposal. On a basic level, its administration would create a
significant bureaucratic burden, and we do not believe that a
single central body would be effective in keeping such a major
database updated. More significantly, institutions take great
care in selecting examiners with the specific knowledge necessary
for their own programmes, which assures subject alignment. The
use of an extended network of contacts to identify potential examiners
ensures that the available "pool" of examiners is regularly
extended through the appointment of those who have not previously
examined, but who are recommended by colleagues and subsequently
inducted into the process. It is hard to see how a central register
would be able to match the efficacy of this process, especially
in new or growth areas where the number of potential examiners
may be restricted.
23. We are aware of no evidence that institutions
select external examiners who are insufficiently critical or who
are too close to the institution or course team; the need for
objectivity is explicit in the QAA Code of Practice, and institutional
criteria for appointment will make clear that this professional
relationship should be supportive, but requires a degree of distance.
It should also be noted that many examiners have indicated that
they would not wish to join any centrally held Register. Their
engagement as fellow academics, operating a critical standards
safeguard, is based on their commitment to the educational and
subject community, but they have reservations about inclusion
as part of a national database.
PORTABILITY
24. The portability of a degree will relate
primarily to its holder, and his/her professional skills, rather
than the subject itself. In a society which is heavily service-industry
based, the ability to conduct research in a critical and analytical
manner, to solve complex problems, and to present complex information
in different ways to a range of audiences, are of critical importance.
Degree classification is determined both by the dedicated subject
knowledge which is demonstrated, but also by the application of
professional or graduate skills (time management; marshalling
of arguments; problem-solving; teamwork; communication and so
forth) to this knowledge.
ADVANTAGES AND
DISADVANTAGES OF
DEGREE CLASSIFICATION
SYSTEM
25. The value of the classification system
is that it gives a quick and easy reference point for students,
employers and other stakeholders. The system is well established,
and its outputs easily recognised. It is essential that this be
supplemented through a detailed transcript and record of performance,
but these will necessarily be as complicated as they are detailed.
26. We acknowledge the weaknesses of the current
classification system, and the anomaly of using such a broad brush
approach to summarising the achievement and attributes of students,
and we await with interest the pilot of the proposed HEAR. However,
as was evident in the sector's response to the Burgess Report,
there is little support for any of the possible alternative systems
which were discussed; and an acknowledgement that, if universities
do not offer summary judgements of performance in some way, then
employers and others will devise their own, which may be less
reliable than those which the awarding institution can offer.
It is unrealistic to suppose that an employer will have either
the time, or the expertise, to interpret a complex document such
as the HEAR for each applicant, although this will provide valuable
additional information for those who are shortlisted, for example.
27. We note that students, at institutional
level, also value the degree classification. This may in part
be based on pre-existing expectations, but it would be counter-intuitive
to change this system unless it is clear that the alternative
offers significant advantages, such as qualitatively greater objectivity.
We do not believe this to be the case with any alternative proposal
at this time.
METHODOLOGIES USED
BY UK HEIS
TO DETERMINE
DEGREE CLASSIFICATIONS,
AND THE
POTENTIAL METHODOLOGIES
FOR STANDARDISATION
28. We would accept that there are many
diverse methodologies for the determination of degree classification,
and note that this is in part a function of institutional autonomy.
Many of these methodologies are of long standing, and have been
developed in response to specific institutional priorities. There
are significant similarities between the majority (including the
accepted grade boundaries). However there may be value to reviewing
existing practice, and developing guidelines and practical advice
which would secure greater consistency of approach within and
between HEIs, especially around borderline cases.
PLAGIARISM
29. We are aware of regular reports of an
increase in student plagiarism, but we believe that institutions
are well aware both of the issue, and of approaches to respond
to it. Sector-wide groups regularly discuss issues such as plagiarism,
and we are convinced that this matter cannot be resolved through
interventions from outside the sector.
January 2009
163 Throughout this document, we have used "quality
assurance" or "QA" to refer to the systems and
processes used by institutions, and the sector more broadly, to
guarantee the standards of awards, to review the quality of the
student learning experience, and to plan for enhancements to that
experience. Back
|
