Memorandum 16
Submission from the Student Assessment
and Classification Working Group (SACWG)
1. Research, much of it conducted by SACWG,
has shown the following.
— The type of assessment task set for students
influences the grades that they receive for their work.
— Assessment criteria are, in practice,
fuzzier than is often acknowledged.
2. Further, there is a lack of clarity across
the sector as a whole regarding the rationales for assessment
regulations and practices. Yorke et al (2002, p.278) observed:
The higher education sector does not know enough
about what its grading methodologies and award algorithms are
actually valuing and how they operate to produce the results that
they do.
3. All of these points weaken the role of
the honours degree classification as an index of a student's overall
achievement.
4. It is inappropriate to impose a standardised
approach to assessment on autonomous institutions which offer
diverse programmes to diverse cohorts of students. Nevertheless,
developmental work aimed at clarifying and evaluating assessment
regulations and practices ought to enable the sector to advance
them on a more collective basis than has hitherto been the case.
WHAT IS
SACWG?
5. The Student Assessment and Classification
Working Group [SACWG] is a small and informal body of academics
and administrators who share an interest in assessment. Its membership
has evolved over time, and the organisational hub of the Group
is Oxford Brookes University, where Dr Chris Rust acts as convener
of the Group. The membership of SACWG is annexed to this submission.
6. SACWG was formed in 1994 and took as its main
purpose the investigation of issues that had hitherto been largely
ignored: honours degree classifications (with particular reference
to the modular schemes that had relatively recently been adopted
in the erstwhile polytechnics); the implications for grading of
different kinds of assessment demand; assessment regulations and
related matters. It has also undertaken occasional commissions
of research. The Group has, since its inception, run seminars
and workshops on relevant topics, and has published a number of
academic papers. It was commissioned to report to the Burgess
Group (which was considering the future of the honours degree
classification) on issues relating to degree classifications,
other national approaches to final awards, and assessment regulations.
SACWG is currently investigating the assessment of work-based
learning in foundation degrees.
SOME ASPECTS
OF SACWG'S
RESEARCH
7. Quite small variations in the way in
which degree classifications are determined (the "award algorithm")
can have more effect on the classification of some students than
is probably generally realised. Running a set of results through
other institutional award algorithms produces different profiles
of classifications (Woolf and Turner, 1997).
8. A number of institutions permit a small proportion
of module results to be dropped from the determination of the
class of the honours degree (provided all the relevant credits
are gained). Dropping the "worst" 30 credit points from
the normal 240 of the final two years of full-time study might
raise one classification in six, and (separately) changing the
ratio of weightings of results from the penultimate year to the
final year from 1:1 to 1:3 might change one classification in
ten, the majority of changes being upwards (Yorke et al,
2004).
9. Marks for coursework assignments tend
to be higher than those for formal examinations, though some instances
were found where the reverse was the case (Bridges et al,
2002). Simonite's (2003) work points in a similar direction.
10. The distribution of marks (usually in
the form of percentages) varies between subject disciplines in
terms of both mean mark and spread (Yorke et al, 1996;
Yorke et al, 2002). Subjects in which student performances
are more likely to be adjudged right or wrong (as is the case
with science-based studies) tend to have wider, flatter distributions
of marks than do subjects in which discursiveness predominates.
Some subjects tend to have high mean marks (eg subjects allied
to Medicine) whereas others tend to have low means (eg Law). The
honours degree award data published by the Higher Education Statistics
Agency [HESA] clearly show these effects (see Yorke, 2008, p.118).
One cannot therefore with confidence interpret classifications
without an appreciation of the norms pertaining to the particular
subject(s) involved.
11. A minority of institutions use a grade-point
system instead of percentages. Whereas this appears to mitigate
the disparity in mark profiles at the level of the module, the
mitigation appears not to extend to the level of the honours degree
classification (Yorke et al, 2002).
12. The "subject benchmark statements"
produced under the auspices of the Quality Assurance Agency for
Higher Education [QAA] were intended to act as reference-points
for curricula, and thereby give employers (particularly) a common
frame of reference when considering graduate applications. In
practice, the emphasis given to different kinds of learning outcomes
varies between these statements (Yorke, 2002). SACWG showed that,
despite the existence of the subject benchmark statement for history,
learning outcomes and assessment criteria in the subject were
suffused with fuzziness and that, as a consequence, assessment
was dependent upon the exercise of a considerable degree of professional
judgement (SACWG, 2005; Woolf, 2004).
13. A study of assessment regulations across
35 varied institutions in the UK showed that there were considerable
variations between them (Yorke et al, 2008). Amongst the variations
were the following:
— The weightings in the award algorithm
ranging between 1:1 and 1:4 for penultimate:final year
— The treatment of "borderline"
performances as regards classification
— The adoption (or not) of "compensation"
(ie allowing weakness in one aspect to be offset against strength
in another) and "condonement" (ie not requiring a relatively
minor failure to be redeemed).
— The "capping" of marks for
re-taken assessments (at the level of a bare pass).
THE VIEW
FROM THE
QUALITY ASSURANCE
AGENCY FOR
HIGHER EDUCATION
[QAA]
14. In its Quality Matters series
the QAA (2007) published a Briefing Note on the classification
of degree awards, based on its experience of institutional audits.
The following observation chimes with the evidence from research:
The class of an honours degree awarded to
a graduating student by an institution does not only reflect the
academic achievements of that student. It also reflects the marking
practices inherent in the subject or subjects studied, and the
rule or rules authorised by that institution for determining the
classification of an honours degree. This is based on the marks
obtained in the components of the study programme followed by
the student. The implications of the role these different factors
play in determining the class of an honours degree are that it
cannot be assumed students graduating with the same classified
degree from different institutions having studied different subjects,
will have achieved similar academic standards; it cannot be assumed
students graduating with the same classified degree from a particular
institution, having studied different subjects, will have achieved
similar academic standards; and it cannot be assumed that students
graduating with the same classified degree from different institutions,
having studied the same subject, will have achieved similar academic
standards.—QAA (2007, para 2)
REFERENCES
Bridges, P., Cooper, A., Evanson, P., Haines,
C., Jenkins, D., Scurry, D., Woolf, H. and Yorke, M. (2002) Coursework
marks high, examination marks low: discuss. Assessment and
Evaluation in Higher Education, 27 (1), pp. 35-48.
QAA (2007) The classification of degree awards.
Gloucester: QAA. At www.qaa.ac.uk/enhancement/qualityMatters/QMApril07.pdf
(accessed 28 November 2008).
SACWG (2005) Developing assessment criteria.
In G. Timmins, K. Vernon & C. Kinealy (eds), Teaching and
learning history. London: Sage, pp. 185-191.
Simonite, V. (2003) The impact of coursework
on degree classifications and the performance of individual students.
Assessment and Evaluation in Higher Education, 28 (3),
pp.459-470.
Woolf, H. (2004) Assessment criteria: reflections
on current practices, Assessment and Evaluation in Higher Education,
29 (4), pp.479-493.
Woolf, H. and Turner, D. (1997) Honours classifications:the
need for transparency. The New Academic, (Autumn), pp.
10-12.
Yorke, M. (2002) Subject benchmarking and the
assessment of student learning. Quality Assurance in Education
10 (3), pp.155-171.
Yorke, M. (2008) Grading student achievement:
signals and shortcomings. Abingdon: RoutledgeFalmer.
Yorke, M., Barnett,G., Bridges, P., Evanson,
P.,Haines, C., Jenkins,D., Knight, P., Scurry, D., Stowell, M.
and Woolf, H. (2002) Does grading method influence honours degree
classification? Assessment and Evaluation in Higher Education,
27 (3), pp. 269-279.
Yorke, M., Barnett,G., Evanson, P., Haines,
C., Jenkins,D., Knight, P., Scurry,D., Stowell, M. and Woolf,
H. (2004) Some effects of the award algorithm on honours degree
classifications in UK higher education. Assessment and Evaluation
in Higher Education, 29 (4), pp. 401-413.
Yorke, M., Woolf, H., Stowell, M., Allen, R.,
Haines, C., Redding, M., Scurry, D., Taylor-Russell, G., Turnbull,
W., & Walker, L. (2008) Enigmatic variations: honours degree
assessment regulations in the UK. Higher Education Quarterly
62 (3), pp.157-180.
RECOMMENDATION
The evidence from SACWG's research and elsewhere
indicates that there is considerable variation across the higher
education sector in assessment practices. Whilst this can be seen
as a consequence of institutional autonomy, the rationales for
the various institutional choices that have been made are unclear.
During the Burgess Group's deliberations, suggestions were made
that the sector would benefit from development work which would
explore and evaluate the rationales for assessment regulations,
with a view to providing a basis from which the sector could—more
collectively than hitherto—advance its assessment practices.
SACWG recommends the commissioning, at an early
date and probably through the Higher Education Academy, of a study
of the rationales for assessment regulations in higher education
institutions, and of their associated rules and conventions. This
is seen as an essential precursor to the advancement of assessment
practices across the sector.
January 2009
|
