Education CommitteeWritten evidence submitted by Mark Hobson

1. I was pleased to see in the 28 October edition of TES a call for examiners to contribute to a House of Commons Education Committee inquiry into the administration of examinations for 15 to 19 year olds. I note that written evidence is being submitted and the call is for a contribution to a seminar to be held in December. The overview set out below will, hopefully, inform the committee about an area of genuine concern such that questions will be asked of those responsible for the quality of administration of qualifications, all the way from awarding body to lecturer.

2. I work at a FE college, part of a team delivering BTEC engineering qualifications across the range level-1 to level-5. Consulting the specification for any certificate or diploma in any engineering discipline will reveal guidance that allows “appropriate assessment methods, suitable to the needs of the learner”. This sounds fantastic in terms of the indefinable mantra of individualised learning: unfortunately, it results in a distortion of assessment quality across the country and an abuse of achievement at a local level.

3. I realise the words used are strong: I shall give reasons for the inconsistency of assessment first. I had the good fortune to assist in research as to why students from a BTEC qualification were less successful when studying for an engineering degree; indeed, they were likely to fail in the first year. This gave me a chance to visit other colleges and explore student performance, how they were assessed and their preparation for university. I also interviewed and reviewed existing university students with the required BTEC qualification.

4. Following visits to a range of local colleges, it could be seen that the diversity of assessment instruments for the mathematics units varied significantly in quality, with some questions aimed at A-level standard but many are simplistic, though all can be interpreted to meet the demands of the unit criteria. All assessments were via assignments – no exams. This method is repeated across the country, as (ex-BTEC) university students admitted as much. This situation of variability will not be overcome until some aspects of all qualifications are subject to externally set exams. This will not be a popular proposal, but do we have truly academic, vocationally related qualifications, a good preparation for university or study at the next level, or just have a non-academic, easy to pass, truly vocational qualifications.

5. The end result is that has decided that the (3 A-level) equivalence given by UCAS to the BTEC qualification is not adequate and A-level maths must be included to access an honours degree programme. This similar approach is taken by other, especially Russell Group, universities; indeed some dismiss the BTEC qualification out of hand. This belittles the grand ambition for the vocational alternative to access top universities.

6. But the reasons are clear. A-level students are subjected to unseen exams, an excellent preparation for higher education. BTEC students very rarely, if at all, take exams; the nature of assessment through assignments with a two or three week deadline is open to abuse; if part of the submitted work is not quite right the assignment is referred, sometimes more than once. BTEC students are, quite simply, ill-equipped to be immediately successful at university compared to students from traditional, exam-based backgrounds.

7. The paragraphs above make a point, but it is the referral of assignments that requires serious review. Senior officials from BTEC will tell you that recent quality improvement measures will ensure such problems will be overcome. Be assured this is misleading. It is possible for a student to have more than one referral, and be given “close supervision” if necessary in more than one unit, let alone just one assignment, to ensure they pass. This situation has developed because staff are required to ensure evidence of achievement is present if asked for – perfectly reasonable. We ensure this happens with students who should perhaps fail because we are told by our managers that if our success levels are low then jobs are under threat. The principle of education is immaterial; if, in our genuine opinion, a student is undeserving of a qualification, we have to check if the data allows us to fail the student; if it doesn’t, they do some more work (you can guess who does this work) to pass. This leads to weak students devaluing qualifications at all levels, yet they are now qualified to access the next level. This results in a compounding situation of raised mediocrity, because those who should fail may well be working at the next level, where they will pass anyway; this is proving an embarrassing situation for experienced staff as it undermines the value of the qualification for hard-working and deserving students.

8. To support the success argument, you only have to look at reports in the TES about colleges receiving outstanding Ofsted grades because of their 98% success rates; why some colleges have stopped delivering exam-based qualifications in favour of vocational qualifications; how some colleges have discovered that units can be delivered in 45-hours or less rather than the recommended 60-hours – “staff have really engaged with our new strategy”. To belittle true achievement is cheap, but anyone can pass any student because the system is open to abuse. Senior managers have recognised that vocational qualifications where no exams are required will result in improved success rates; it is also possible to achieve such in reduced student contact time in some colleges. This is called efficiency, and I understand why it is done – what would you do if the system could be played to make you look good in terms of success at lowest cost? It is called efficiency, but is it education?

9. By abuse, I mean the following:

(a)It is possible for a member of staff to claim a certificate even if a student has failed to meet the necessary requirements for success. This is less than professional, but it is possible. No improvement to quality systems can stop this happening; though having some aspect of assessment by external examination might make it a less likely occurrence.

(b)Certificates can be requested to be given to the course leader (the member of staff responsible for the course) so they can award the certificate to the student. Unfortunately, despite a student failing, the certificate can still be claimed and kept in a locked drawer: the student has failed, does not get the certificate, but the college still has a success. This abuse can be stopped by a directive that college exam departments are not allowed to divert certificates.

10. The inquiry will focus, as it often does, on traditional areas such as GCSEs and A-levels; this is well understood as it is popular with politicians and media. I write the above to ask that you address an area under-represented (FE BTEC qualifications) or not considered really worthwhile by comparison with traditional educational concerns. With the drive for STEM subjects to be given priority, our duty is to assure the quality of all engineering qualifications – especially as apprentices often have BTEC technical qualifications as part of their framework.

11. I am always inspired by the determination of colleagues, in all colleges, who stay positive and continually desire to want to make a difference to a student’s opportunity to make their way in the world. What frustrates most lecturers is the view that everyone passes no matter what and the drive of management to threaten staff with redundancy if success rates do not improve. What do you do – consistently stand on principle and lose your job, or find a way to keep your job, only possible because the administration of BTEC qualifications, awarding body to lecturer, is open to abuse?

12. The advert posted in the TES has bullet points indicating topics to be looked at as part of the seminar. May I ask that the bullet points are amended as below (my italics) so that questions prompted by the above are included as part of your inquiry. (… looking at the following topics):

(a)The strengths and weaknesses of the UK examination and vocational qualification assessment system.

(b)Accuracy and standards in examinations and vocational qualification assessments and how these relate to school and college accountability measures.

(c)How well the examination and vocational qualification assessment system prepares young people for further education and employment.

13. I am sorry to have to bring these concerns to your attention, but I feel duty bound to reveal practices that are questionable. For as long as you are unaware of such ill-equipped quality systems, the administration of such will never be subject to review as it is believed they are adequate. I might add that, if asked, anyone in the system can deny the suggested abuse happens and evidence would be difficult to establish. The critical question is whether such malpractice is possible and, if so, what changes to the administration of such qualifications must be made to deter these practices.

14. I hope you appreciate the spirit which prompted this communication and the confidential nature of its content. I note from the guidance for the inquiry that it will not examine the design or merits of particular qualifications, or scope for change to the range of qualifications offered. However, I believe there is merit in the content of this overview being considered in the context of scope of the inquiry. I respectfully ask that you recognise that the sensitive nature of my comments could well put me at risk in respect of job security and that appropriate words and sentences are deleted to ensure anonymity, should you publish the content.

November 2011