Memorandum submitted by City & Guilds
1. City & Guilds approaches the discussion
from a vocational qualification standpoint.
2. Assessment should be seen as a rare event
that put demands and responsibilities on both the designer and
the learner.
3. The use of awarding bodies within the English/UK
system is untypical in comparison with European countires. However,
they provide a valued assurance of independence and professional
expertise to the consumer.
4. Through a pair of professional associations
awarding bodies are successfully reducing examination bureaucracy.
5. The twin requirements of validity and reliability
should always govern the choice of assessment methods.
6. While formative and summative assessments
have different purposes there is potentially valuable feedback
available from both which can aid learning for all parties.
7. Coursework should not be abandoned in favour
of examination. The design of coursework should be improved.
8. To reduce the examination burden between
16 and 18 teachers should be given a greater role in summative
assessment of performance.
9. Using multiple approaches to assessment increases
the reliability and accuracy of the assessment of the learner's
knowledge and skills.
10. There is too little time for there to be
the development of innovative approaches to assessment for the
new aspects of the Diploma.
11. While employers should be encouraged to
train to awards or units on the national qualifications framework,
involving awarding bodies can ensure that bespoke qualifications
are of a high standard and are portable.
12. INTRODUCTION
12.1 It must be made clear at the outset that
City & Guilds is a vocational awarding body that has as its
primary focus the assessment and certification of vocational knowledge
and skills. Our market is generally 16+ and the average age of
our candidature is around 30.
12.2 We have over 500 qualifications on offer
and deliver to around 6500 centres in the UK. A centre can be
anything from a FTSE 100 employer, to a College of Further Education,
a Sixth Form college, a private training provider, to small employers.
We issue around 1.5 million certificates a year. We believe that
about 1 in 5 adults within the UK hold a City & Guilds certificate.
12.3 While our history would associate us strongly
with traditional craft skills like agriculture and horticulture
or construction and building services our broad portfolio of products
reaches to retail, care, IT and ESOL (English for Speakers of
Other Languages) and much beyond. The City & Guilds Group
also includes the Institute of Leadership and Management offering
awards in over 2,200 centres. Across the Group the range of awards
extends from Entry Level to the equivalent of Level 8 of the QCA
National Qualification Framework. Our portfolio is also delivered
in about 100 countries internationally through some 3500 centres
worldwide.
12.4 City & Guilds has been an examining
body since 1878 and was awarded a Royal Charter in 1900. It has
necessarily acquired considerable skills in curriculum and assessment
design and delivery.
12.5 With regard to the interests of the Select
Committee we seek to offer some general observations on the principles
and purposes of testing and assessment, and on the role of awarding
bodies.
13. GENERAL ISSUES
13.1 The UK stands out in comparison with both
its local and more distant neighbours in that independent awarding
bodies carry out the process of examining and certification for
national awards. These bodies have become centres of considerable
expertise in these processes and carry a major financial and moral
burden of expectation in terms of accuracy and prompt delivery.
This situation is a consequence of historical decisions and a
presumed wish by past governments not to take on the administration
and associated significant costs of running this national system.
13.2 The management of this system through regulation
provides a reassuring degree of independence to the system that
we believe is valued by the consumer. There is more accountability
in that the awarding body's reputation depends upon their ability
to deliver and market forces exert continued pressure on the system
to ensure high levels of quality assurance. As opposed to being
run by what could so easily be perceived as a bureaucratic government
department.
13.3 Recent initiatives by awarding body consortia,
the Joint Council for Qualifications (JCQ) and the Federation
of Awarding Bodies (FAB), are making progress in the fight against
bureaucracy. In particular the JCQ (comprising Edexcel, AQA, OCR,
City and Guilds, WJEC, CCEA and SQA) supports an initiative it
has called The Eight Pledges. These are principally eight areas
where the awarding bodies will collaborate to reduce the complexity
in the qualifications system by reducing the administrative burden
caused by assessment and quality assurance with the purpose of
simplifying the relationships within and between awarding bodies
and centres. FAB is a consortium of over 30 awarding bodies concerned
with vocational qualifications.
13.4 In the awarding body system we have collaboration
and competition both have roles to play. In that much of the marketplace
is buying national awards then collaboration is critically important
to retain customer confidence and economy within the system. The
opportunity for competition enables awarding bodies to fine-tune
their approaches to satisfy a wide marketplace and provide specific
and differentiating customer benefits.
13.5 The English regulator, QCA has contributed
to this situation. On the one hand when discussing major initiatives
it ensures wide representation by awarding bodies but its supervision
of the awarding process remains specific to each awarding body.
Where there has been confusion in recent times it has been when
QCA has adopted roles already performed by awarding bodies. For
example, in the development of curriculum for say GNVQ or more
recently the management of question banks for Key Skills.
13.6 Awarding bodies are centres of expertise.
The larger awarding bodies, in particular those in membership
to the JCQ, carry specific technical expertise in curriculum design
and assessment practice. The JCQ itself through its committee
structure also promotes the further development of these skills
and addresses the technical issues of assessment and standards
setting for general qualifications.
14. TESTING AND
ASSESSMENT AT
16 AND AFTER
14.1 Assessment is about the collection and
validation of specific evidence from or about a learner. Assessment
in any of its forms is intrusive and for many unwelcome, un-looked
for and unpleasant. Consequently, there is considerable obligation
on the designer of tests or assessments to make them as efficient
and meaningful as possible. Assessment opportunities should be
seen as rare events during which the assessment tool must be finely
tuned, accurate and incisive. To conduct a test that is inaccurate,
excessive, unreliable or inappropriate is unpardonable. Moreover,
it is an insult to the hard work and anxiety of the learner to
waste their time or be needlessly over demanding. Economy it time,
effort and cost is imperative.
14.2 Assessment can be put to two principal
purposes; namely formative and summative. These have different
roles in the learning process and in the ultimate recognition
of achievement. Put briefly, formative assessment provides feedback
to the learning process by identifying both progress and gaps
in learning. Used in this sense it is often regarded as diagnostic.
Summative assessment is usually conducted at the end of a learning
process and is focussed on assessing learning against a known
standard for the purposes of certification.
14.3 There is a strong argument to suggest that
all assessment should support or promote learning. This will depend
upon the opportunity for feedback post assessment. Currently,
this is the strongest division between the two types of assessment.
Formative assessment generally operates at a local level and is
built into a learning programme. The outcomes of assessment are
not high stakes but provide staging posts to further or remedial
learning. It is assumed that summative assessment only has value
in terms of the final result (pass, merit, distinction or grades
A-E etc). Given suitable analysis or interpretation much value
can be extracted from a candidate's examination paper or final
practical assessment. Considering the significant effort that
goes into the final examining process by all parties the current
under-use of this data is a travesty. Some awarding bodies are
now developing analytical software associated with on-line access
to enable some benefit to be drawn from this available data. This
information can be of use to the awarding body, the examiner,
the centre as well as the candidate.
14.4 For summative assessments like GCSE, AS
or A2 enabling the outcome of assessment to support learning would
make demands on both the examination structure and the marking
process. However, the potential benefits for the candidate are
significant. It would also exert additional rigour on the assessment
process to achieve greater detail and accuracy. Some attempt has
been made to compensate for the scant regard given final assessment
through the provision of the opportunity for centres and learners
to review examination scripts for general qualifications.
14.5 The discussion of examination and coursework
may benefit from a brief consideration of two of the technical
issues within assessment, namely validity and reliability. Put
simply, the `test' of validity seeks to confirm that the form
of assessment used adequately reflects or accesses the skills
to be measured. You will have a more appropriate assessment of
the skills of baking by setting the task of baking rather than
setting an essay question on how to bake. The "test"
of reliability requires the assessment designer to show that assessment
will repeatedly produce the same outcome, that there is no inherent
bias or variability in the assessment instrument. Examination
and Coursework should be regarded as two separate assessment instruments
that reside in the assessment designer's toolbox which carry different
degrees of validity and reliability.
14.6 In considering the balance between examination
and coursework it may be interesting to reflect upon the current
situation in vocational qualifications. In particular the National
Vocational Qualification (NVQ) as regulated by QCA. The NVQ is
a performance-based qualification underpinned by specific occupational
standards. The assessment is almost totally locally supervised
against assessment schedules prepared by awarding bodies in association
with sector skills councils (SSCs) and accredited by QCA. Some
NVQ do involve additional knowledge tests issued by the awarding
body. The coursework is evidenced through a portfolio, a physical
or electronic document that maps the learner's progress of performance/skills
demonstration through the various units of the award. A locally
based, occupationally competent assessor who has the opportunity
to ask questions, challenge and reconfirm the performance carefully
monitors and confirms the learner's achievements.
14.7 It is interesting to note that were workplace
qualifications are concerned, where the country's economic performance
is essential and of keen government interest the primary assessment
decisions are made at local (supervisor) level. Whereas for school
based qualifications which may lead to initial employment, further
education and training or higher education an elaborate system
of external examinations and near total independence from the
local centre of learning is required. Many other countries invest
considerable importance in the professional judgement of their
trained teachers with regard to summative assessments of achievement.
14.8 It has been noted in some studies of the
vocational education and training settings that a proportion of
trainees experience difficulties with the development of their
NVQ portfolio because of the tight examination based schooling
they have received which has failed to inculcate the independence
of thought and action needed within vocational education and training
and employment.
14.9 The recent anxieties expressed over coursework
and the opportunities for plagiarism are not unknown in vocational
awards (VQs) though the types of incidence are different. The
quality assurance system for VQs depends upon occasional visits
to centres to essentially undertake an audit of local assessment
practice. This process supported through regulation is called
external verification. The process seeks to ensure that centres
follow the required guidance and maintain the performance standards
and criteria.
14.10 One particular issue to be considered
with coursework is the nature of the task being demanded. If the
task is one which can be easily downloaded from the Internet and
passed off as the new learner's work then one can rightly suggest
that the original task was poorly conceived or set out. Assessment
techniques must of necessity move with the times. If new technologies
make accepted assessment practices redundant then new ways of
accessing the required skills must be devised. This takes time
but the solution is not to rely upon one form of assessment, as
this should be regarded as poor practice.
14.11 It is important to recognise that learners
differ from each other and it is likely that acquiring curriculum
content or specific skills demands a range of skills on behalf
of the learner. Consequently using only one or at best two assessment
techniques limits the type and value of the evidence one is collecting.
There is a risk that the process will not do the learner justice.
Over reliance on one or other forms of assessment cannot be regarded
as good practice. However, it must be conceded that getting the
appropriate weighting between assessment methods is also a difficult
process. Dispensing with coursework is not the answer to plagiarism,
as this would over-focus the teaching programme on the final examination
to the diminution of those auxiliary skills the curriculum sought
to develop. This also returns us to the question of the validity
of the chosen assessment tool.
14.12 The weighting issue has been of particular
concern in the development of the new Diploma, in that the final
award has to be derived from performance in a range of specified
elements. What has been more problematic has been the difficulty
in securing clarity over the purpose of the qualification and
its underpinning curriculum. The tension between the general and
vocational themes will also be played out in terms of assessment
regimes. The vocational trend would be for more emphasis on performance
evidence (ie `can do') whereas the general trend goes for knowledge-based
evidence (ie `knows and understands that'). The vocational aim
for the Diploma was to get learners out of the classroom or at
least to make learning relevant beyond the classroom. It is still
too early to say how the Diploma will work out in practice. A
full and energetic evaluation of the pilot programmes is essential.
14.13 It is difficult to say whether or not
Diploma assessment methods will have a consequential effect on
GCSE. This will in part depend upon the innovation awarding bodies
are able to bring to the assessment of the new elements in the
Diploma. Given current timescales there has been insufficient
time for Component Awarding Bodies to research and develop new
assessment approaches. It can easily be argued that for a new
qualification which is to be the standard bearer for a subtle
blend of general and vocational skills that new and innovative
approaches would be needed in both curriculum and assessment delivery.
Accepting some of the earlier points in this submission some of
this innovation should pay attention to improving feedback after
assessment following the principles of assessment for learning.
14.14 There can be little doubt that the years
16,17 and 18 are a great challenge and pose a significant demand
on all learners as they face an intense 3 year examination period.
It is a shame that the regulated examining systems are unable
to make greater use of the expert judgement of teachers and tutors
and that increasing reliance is put upon single shot examinations.
One of the benefits of the NVQ assessment system is the requirement
to observe performance over time. Consequently random poor performances
can be weighed against evidence of more consistent performance.
Repeated demonstrations of good performance are also required
rather than a single inspired one.
14.15 It must be recognised that the general
qualifications process remains a highly competitive one and a
learner's success depends heavily upon the school or college they
attend and the resources that school or college is able to secure.
In this sense it is not a `fair' system. It will not be `fair'
until all learners are able to access equivalent resources delivered
to a common standard. While the same criticism can be made of
vocational qualifications the system has embedded procedures to
reduce the variability. All occupationally competent assessors
must also hold a nationally approved assessors qualification.
Every centre delivering an NVQ must meet regulated approval criteria.
Each centre is regularly visited by a representative of the awarding
body to ensure that the centre is maintaining the occupational
competence standards in its assessment practices.
14.16 The same is true even where employers
choose to deliver NVQ within their staff training programme. The
benefit being that staff trainees will ultimately receive a nationally
recognised qualification which will be truly portable rather than
a training package locally conceived by the employer which may
have no relevance to any other (future) employer. A training programme
endorsed by the employer alone, except where that employer has
achieved considerable brand credibility, will have only limited
value.
14.17 Where awarding bodies collaborate with
major employers to produce bespoke qualifications the staff trainees
have the benefit of knowing that they will receive a properly
validated qualification based on the experience, expertise and
professionalism of the awarding body. The portability is based
upon the recognised professionalism of the awarding body. It is
also most likely that the awarding body will retain, in a lasting
archive, full records of past achievements. City & Guilds,
for example, goes back 100 years. Few commercial businesses last
that long or would wish to retain records of long departed employees.
May 2007
| 
