House of Lords - Higher Education in Science, Technology, Engineering and Mathematics (STEM) subjects

Higher Education in Science, Technology, Engineering and Mathematics (STEM) subjects - Science and Technology Committee Contents

CHAPTER 5: Quality, standards and benchmarks

108. In 2009, the Quality Assurance Agency (QAA) told the House of Commons BIS Committee: "the quality of the education offered by UK institutions is its strength and the basis of its strong reputation".[143] Many others have made the same point to us.[144] However, in 2002 the Roberts Review looked at the question of supply and demand and concluded that shortages in some areas related to quality, not quantity, of graduates, stressing that there was a lack of graduates with appropriate general and transferrable skills (otherwise known as "employability skills"—see paragraphs 114-116), or the required breadth of knowledge in their technical or scientific field.[145] These conclusions are mirrored in more recent reports from BIS and others, and also in the evidence we received.[146] Given that the mismatch in supply and demand for STEM graduates relates in part to a lack of high quality graduates in many sectors, not necessarily in the overall number, we decided to look at how quality is assessed and at the mechanisms for improving quality. These issues are particularly important given proposals to open up degree-awarding powers and allowing the title of "university" to be used by more providers.

109. At present, quality assurance of degrees is the responsibility of a number of bodies including HEIs, HEFCE, QAA, and Government. However, individual institutions are autonomous organisations and, as such, have primary responsibility for academic standards and quality.[147] QAA provides a means of external assurance to this process.

110. Under the HE reforms, it is envisaged that students—making decisions based on the course information provided by HEIs—will play a greater role in driving up quality. According to the Government, "the future of higher education is moving to a model where supply and demand will operate in a more transparent market. Students will demand better information about the quality of their degrees, and how these will lead to their chosen careers. Universities will need to respond, particularly in STEM where there is greater correlation between specific industry needs and the content and skills taught".[148] Employers and Government also have a role to play.

111. Although we consider both undergraduate and postgraduate provision in this chapter, we also look specifically at the role that the Research Councils and HEFCE play in providing quality assurance at the postgraduate level. We also look at the move to doctoral provision through Doctoral Training Centres (DTCs) and away from the supply of doctoral studentships related to single research grants.

Definitions of quality and how it is measured

DEFINITIONS

112. Quality can often mean different things to different people.[149] Defining "quality" and its measurement is, therefore, an important starting point to the debate. The QAA makes a distinction between "standards" and "quality":

Standard—the level of achievement that a student has to reach to gain an academic award (for example, a degree).
Quality—a way of describing how well the learning opportunities available to students are managed to help them to achieve their award. It is about making sure that appropriate and effective teaching, support, assessment and learning opportunities are provided for students.

113. In 1993, John Biggs defined the three dimensions of quality in the '3P' model as "presage", "process" and "product" (see Box 2). "Presage" defines the context that is set before students start learning, such as the quality of the student or teacher; "process" defines variables that affect the student's learning experience; and "products" relate to the outcomes of that learning.[150] In our view, the quality of a degree is dependent on all three factors to ensure that high quality graduates leave HE with the right skills and knowledge to prepare them for work..

BOX 2

Measurements of quality

In 2010, the HEA published a report entitled Dimensions of Quality which critiqued different quality measures used within HE. They grouped them, in accordance with John Biggs' '3P' model, into "presage", "process" and "products", although they are related to each other.

"Presage" variables are described as those that exist within a university context before a student starts learning including resources or the quality of student or teacher.
"Process" variables are described as those that characterise what is going on in teaching and learning and include class size, the amount of class contact and the extent of feedback to students.
"Product" variables concern the outcomes of the educational process, such as student performance, retention and employability.

The report concluded that measures of educational process are good predictors of educational gain. Such measures are also considered to be more comparable than product or presage variables across HEIs which have very different mission statements.

Good process measures of quality include class size, level of student engagement, who undertakes teaching and the quantity and quality of student feedback. The best variables concern pedagogical practices that engender student engagement.

The report also concluded that teachers who have teaching qualifications have been found to be rated more highly by their students than teachers who have no such qualifications.

In terms of outcomes it was noted that there was considerable scope to assess work submitted by students as more direct indicators of educational quality than proxy measures such as thos derived from National Student Survey scores.

EMPLOYABILITY SKILLS

114. The terms "quality" and "employability skills" are often confused. "Employability skills" are defined by the Higher Education Academy (HEA) as "a set of achievements, (skills, understandings and personal attributes) that make graduates more likely to gain employment and be successful in their chosen occupations, which benefits themselves, the workforce, the community and the economy".[151] Although an HE system should not concentrate solely on training students for the employment market—it is just one of the objectives of the sector—as we said in Chapter 4, a high quality education and training for employment are not incompatible and graduates should be leaving HE with the right skills for employment. Imperial College London told us, for example, that:

"A high quality research-led and laboratory-based or maths education, equips students to think critically and independently, and to foster the analytical skills necessary to provide solutions to economic, social and industrial problems. Such skills are not only necessary for careers in traditional STEM industries, but are requirements for a variety of business sectors."[152]

115. Employability skills should also include other attributes, or "softer skills", within the outcome measures for a course, such as awareness of the business environment and communication skills.[153]

116. One question we encountered was: how accurately can supply be matched to demand in a diverse economy? The Institution of Engineering and Technology (IET), for example, told us that "while large companies have the capacity to recruit 'raw' graduates and provide training this is generally not the case for SMEs [Small and Medium-sized Enterprises] who need skilled graduates who are equipped to 'hit the ground running' and contribute immediately in the workplace".[154] However, as Vectura observed, there is a danger of going too far in trying to meet all the employability needs identified by employers—"since each company will have different requirements it is not particularly practical to try and match supply and demand in this way".[155] Employers have a role in continuing to develop the skills of graduates in employment, and HEIs have a role to help them do this. But it is not the role of HEIs to tailor degree courses specifically to individual companies. Rather, it is to ensure that graduates emerge with the right skill set to adapt and grow. The role of employers is also to define the basic skill set of an employable graduate or postgraduate. We consider how employers should be involved in setting the skills outcomes for courses in paragraphs 160-174.

THE MEASUREMENT OF QUALITY

117. The quality of a degree course can be measured through a number of variables including direct learning outcomes or looking at the processes which have been linked with high quality provision. Measures can also be direct or proxies of quality.

QUALITY ASSURANCE

118. The Government's Higher Education White Paper, Students at the Heart of the System, published in 2011, proposed that HEFCE should become a lead regulator to promote and protect the interests of students and the wider public, taking over from the Government the function of granting degree- awarding powers. In evidence, Government said: "the UK approach to quality assurance is highly regarded and the revised arrangements for Institutional Review introduced in September 2011 by the QAA have a stronger focus on quality enhancement and on involving students".[156] The HE reforms will not involve a change to the review process but propose a move to a more risk-based approach to quality assurance, building on the current system but focusing QAA effort on areas of most impact.

119. BIS has consulted on these proposed changes, which include proposals to introduce a sanction to suspend, or remove, degree-awarding powers where quality or academic standards fall below acceptable thresholds. Additionally, HEFCE are currently consulting on the triggers which would prompt QAA to carry out an out-of-cycle investigation, as well as the frequency of review, for implementation in 2013-14. The proposals include greater powers for students to instigate out-of-cycle reviews if they have concerns about quality. Under the HE reforms, QAA will also look at the provision of information about the quality and standards of academic programmes. A QAA consultation will follow the HEFCE consultation later in the year.

120. The Government's response to the Higher Education White Paper consultation stated that they will "not at this stage be seeking to introduce changes to primary legislation" but they would move their reform agenda forward "primarily through non-legislative means".[157] It is not clear to us, therefore, if Parliament will be given the opportunity to scrutinise the proposed changes to quality assurance and HEFCE's power. We recommend that the Government explain in their response to this report what opportunity Parliament will be given to scrutinise further the proposed changes to quality assurance, as set out in the Higher Education White Paper. The Government should also set out a timetable for when the changes will take place and outline the form they will take.

The role of the QAA and of HEIs in driving up quality

121. HEFCE has a statutory responsibility for quality assurance under the Further and Higher Education Act 1992. They do this, by contract, through the QAA, which reviews HEIs in England, Wales and Northern Ireland.[158] The QAA is independent of both Government and HEIs. It is funded by subscriptions from universities and colleges and through contracts with the HE funding bodies. They report on quality assurance by visiting universities and colleges to review how well they are fulfilling their responsibilities. They also offer guidance on maintaining and improving quality assurance processes and developing course delivery through the Academic Infrastructure.[159]

122. QAA HEI reviews focus on the standards of HE awards (set out in the Quality Code which covers undergraduate and postgraduate courses), the quality of the learning experience (for example, the provision of learning opportunities), the provision of information about the quality and standards of academic programmes, and the commitment to quality enhancement. For example, QAA assesses whether universities set and maintain UK-agreed benchmark statements or threshold standards for HE awards as set out in the Framework for Higher Education Qualifications in England, Wales and Northern Ireland (FHEQ).[160] These are reviewed, as required, in collaboration with institutions, and professional, statutory and regulatory bodies. The FHEQ also contains a doctoral qualification descriptor which summarises "the research-specific and personal attributes agreed by the higher education sector as a minimum level of achievement for any doctoral graduate" to "achieve equivalence of academic standards across doctoral awards by summarising the key attributes expected of a doctoral graduate." There is a similar descriptor for Masters degrees.[161]

123. In 2009, in evidence to the House of Commons Innovation, Universities, Science and Skills (IUSS) Committee, the QAA emphasised their process-based approach to quality assurance: "we visit institutions to conduct our audits, make judgements and publish reports, but we are not an inspectorate or a regulator and do not have statutory powers. We aim to ensure that institutions have effective processes in place to secure their academic standards, but we do not judge the standards themselves".[162] When questioned on why they looked at processes not outcomes, the QAA said that quality outcomes were included in the Quality Code, which was a "framework of benchmarks, which sets standards and provide programme specifications for individual courses … together with codes of practice about key areas of university activity". They were "responsible for maintaining that", which they did "in partnership not only with institutions but also with relevant professional bodies, with expertise sometimes from employers and, increasingly, with input from students as well".[163]

124. The QAA is responsible for measuring the processes by which quality is assured and for facilitating the development of outcome measures through the development of the standards and benchmarks. In our view, however, this is not enough to drive up quality when the standards and benchmarks are based on attaining a threshold level and allow no assessment of quality provision above the threshold. When asked if degrees from two universities were comparable in terms of the standards required, Anthony McClaran, Chief Executive of the QAA, said: "we are talking about the setting of threshold standards. We are not talking about the distance beyond the threshold that any particular qualification may go".[164] HEA, in their report Dimensions of Quality, describe the absence of comparability of degree standards as a major "obstacle to the interpretation of student performance data".[165]

125. The HEA also noted in their report that "national agencies have a valuable role to be fulfilled in supporting the use of valid measures of quality", in effect seeking to make sure that HEIs are using the best measures of quality. In our view, they are not fulfilling this need. The QAA gives little incentive for HEIs to go beyond that threshold of assessment to drive up quality. The Quality Code sets minimum standards and benchmark statements, and the QAA then judge HEIs on the processes they have in place to enable them to meet them, rather than influencing HEIs to raise standards, improve benchmark statements and the quality of provision.

126. When challenged on this issue, the QAA said: "we have another role … which is also about supporting institutions in the continuous improvement of their quality and standards".[166] This is achieved through offering suggestions of best practice, and through themed reports on specific issues. However, we found little evidence of how such efforts have resulted in an increase in quality of provision and, in any event, would question whether it would be possible to do so given that they are only assuring a minimum threshold standard for courses across the UK with little incentive to revise benchmark statements.

ENSURING THAT STANDARDS AND BENCHMARKS ADDRESS SKILLS GAPS IN THE ECONOMY

127. Many undergraduates and postgraduates reportedly lack the skills and competencies required for employment. The ABPI told us, for example, that "although the numbers of STEM students and graduates have been increasing in recent years … many choose to study subjects which do not provide the appropriate skills for roles in academic or industrial research and development or for other jobs in industry .... Many will not have studied the topics which provide essential skills for bioscience research".[167]We also learned, to our astonishment, that graduates from biochemistry can leave HEIs with limited experience of practical laboratory work.[168]

128. All this suggests that there may be scope for improvement of the quality of graduates through more employer-HE engagement on course content and other activities (see paragraphs 160-174). We note that a CBI survey in 2011 reported that only 17% of responding employers were engaged with universities to develop business-relevant STEM courses (albeit higher for some sectors such as energy and water (53%), construction (30%), science/engineering/IT (38%)).[169] There is clearly room for improvement.

129. When we asked whether standards and benchmark statements took sufficient account of employability skills, Mr McClaran of the QAA assured us that "in terms of the skills that are built into both the framework of higher education qualifications, …but also, in terms of the subject benchmarks, there certainly is a description of skills, including the skills that might be applied by the graduate once they have gone into employment".[170] This led to us to ask why graduates are failing to acquire basic employability skills.

130. With regard to employers involvement in setting standards and benchmarks, we were concerned to see that, according to the QAA benchmark statements, few industry representatives are currently involved in setting some benchmarks (for example, those for engineering). Cogent were also critical of the QAA benchmarks. Dr Caroline Sudworth, Higher Education Manager from Cogent, described her experience: "from the work that I have done with the National HE STEM Programme, we have looked at the CBI benchmark and employability, mapped it to the QAA benchmark statement, and there is a lot missing from those particular STEM degrees. We have tried to discuss this with the QAA but to no avail at this moment in time, so we would like the QAA to look at how employers interact with that quality assurance in STEM degrees".[171] Mr McClaran conceded that he "would be the first to acknowledge that involvement of employers should expand". He went on: "That is one of the issues, as we move into the new review method that we have just begun in England, where we want to expand the involvement, both of students on the one hand and employers on the other". He also said that QAA was having discussions with CHIE about "ways in which we can get expert input from employers to particular areas".[172]

131. Given the skills gaps that exist in key areas across the graduate pool, the QAA has a long way to go in ensuring that employers are sufficiently involved in setting standards and benchmarks. We recommend that the QAA should do more to recruit employers, SMEs in particular, to engage with HEIs and take part in setting QAA standards and benchmark statements. The QAA should be in a position to report back on how it plans take this recommendation forward by July 2013.

132. We further recommend that the remit of the QAA should be reviewed with a view to introducing a system to assure quality, standards and benchmarks in HEIs that is fit for purpose. This should include the development (and achievement) of objectives for the inclusion of employers in the setting of standards and benchmarks, and a yearly list of thematic problem areas, accompanied by an action plan, where consistent skills gaps occur.

Funding to develop the employability skills of postgraduates

133. We recognise that efforts have been made by HEIs and others to improve the employability skills of undergraduates and postgraduates. In 2002, the Roberts Review highlighted the need for the development of research careers and to improve the transferable skills (employability skills) of postgraduate students. As a result, the Research Councils created a specific funding stream (Roberts' Money) of over £120 million between 2003-11[173] to address these issues in all research disciplines. The Roberts' Money was used to fund new training schemes or activities aimed at improving the employability skills of postgraduate students. A number of submissions were supportive of the scheme and the impact it has had on improving the employability skills of postgraduates.[174]

134. In 2010, the Research Councils commissioned a review of the Roberts scheme. The review found that there have been major improvements facilitated by the Roberts' Money such as "improved understanding of the importance of more formalised training and career development for all researchers; and improvement in the way career development and transferable skills training is provided for researchers." But, the review concluded that "there was little interaction of employers and other stakeholders in setting up skills development programmes which could potentially diminish the effectiveness of the programmes". Our recommendation for improved engagement between HEIs and employers, therefore, applies equally to the postgraduate level. As a result, we were pleased to hear that the Researcher Development Framework (RDF), developed by Vitae in consultation with employers, has gone some way to improve the employability skills of postgraduates and guide the knowledge, behaviour and attributes of a successful researcher.[175]

135. Given its findings, the Research Council review recommended that "funding should ... continue to be made available specifically for the development of transferrable skills of researchers and their careers".[176] However, in 2011, the Roberts' Money funding stream was closed. Research Councils UK (RCUK) told us that this was because funding was now being embedded in the standard funding mechanisms. However, we share the concerns of the University of Manchester and the University of Oxford that embedding this type of funding may dilute the message that transferrable skills in postgraduates should form a fundamental part of their training.[177]

136. We recommend that the Research Councils monitor the impact of embedding Roberts' Money into the standard funding mechanisms.

Quality of teaching

137. Several witnesses expressed concern about a lack of incentives for HEIs to improve the quality of teaching within their institutions. The QAA measures a number of process factors that are proxy measures for the quality of teaching. But, as with the standards, they are testing to a threshold level. They are not assessing whether teachers are of a high quality.

138. The HEA, which has 165 HEI subscribers,[178]has an important role to play in promoting good practice in teaching in HEIs. The HEA is owned by Universities UK and GuildHE "which supports the higher education sector in providing the best possible learning experience for all students".[179] It promotes professional development and accreditation of teaching through its re-launched UK Professional Standards Framework (see Box 3).[180]

BOX 3

The priorities of the Higher Education Academy

Helping to improve the quality of learning and teaching practice by providing a structured framework and resources to underpin professional development and by supporting a vibrant and professional learning culture across the sector;

Supporting leaders and managers to develop an organisational culture and infrastructure within which student and staff learning can thrive, and in which change is managed confidently and creatively;

Responding quickly and intelligently to the most urgent and significant strategic issues and contemporary challenges that the sector is facing, supporting the sector to react wisely and decisively during times of unprecedented change and acting as a national voice to positively influence change;

Underpinning all of the above with high quality and rigorous research and evidence and applying this insight to enhance policy and practice.[181]

139. The recent HEA report, Dimensions of Quality, concluded that teachers who have teaching qualifications are rated more highly by their students than teachers that do not. Professor Craig Mahoney, Chief Executive of the HEA, told us: "out of the 180,000 academics working in UK higher education, there are currently 30,000 academics registered on our books as being qualified to teach, having completed a postgraduate certificate in higher education, or having completed a recognised programme through our organisations, which aligns at the same standard against a professional standards framework".[182]He argued that the actual number of qualified teachers was probably higher than this because HEIs were not required to report to the HEA if they had provided training through other means.

140. This evidence suggests, however, that there is a significant shortage of academics "trained" to teach to a high standard. ABPI said that one reason for this shortfall was that the Research Excellence Framework (REF) did not offer sufficient incentive for HEIs to provide top quality teaching.[183] According to Professor Mahoney, longitudinal studies have indicated that "promotions criteria, more commonly … recognise research characteristics and research achievements than teaching achievements".[184]

141. The HEA suggested that, if teaching is to be valued in the same way as research, the career structures within universities needed to change so that more promotion opportunities were made available to good teachers.[185] The AMS made a similar point.[186] The AMS also suggested that guidelines on best practice for improving teaching were integral to improving the value and recognition of teaching. They argued that Government and professional bodies should be proactive in orchestrating the spread of good practice in the management of the teaching load. The Society of Biology suggested putting in place measures to highlight the importance of teaching at HEIs such as HE teaching awards, CPD in teaching and clear routes to promotion that recognised the importance of teaching.[187]

142. There is considerable debate about the relationship between teaching and research.[188] We recognise that research is an important factor in determining the quality of provision within STEM when students wish to go on to conduct their own research, but we agree with HEA that it should not be the only factor. HEA said that "we are seeing a rebalancing" of promotions criteria away from research and towards recognition of good teaching. However, we received little evidence of this rebalancing. From 2014, it is proposed that data on the professional accreditation of teaching staff, which is collected by HESA, could be used to trigger an out-of-cycle review by the QAA if the numbers were considered to be lower than the average.[189] Whilst welcoming this development, it is not, in our view, enough.

143. The Higher Education White Paper, under the HE reforms, stated that "well-informed students will drive teaching excellence". Our panel of students valued an emphasis on teaching but thought teaching was not a high priority within all HEIs. Fabio Fiorelli, a fourth-year MEng chemical engineering student at University College London, for example, told us: "I believe that what could be improved is the interaction with the students. It would be nice to be able to reward those teachers who make the best of their time with the students".[190]

144. In 2010, the Browne Review recommended that all HE staff should receive teacher training.[191] We agree that this should be the aspiration of all HEIs, and that students, if they are to drive quality in the system, should be able to find out how many of their teachers have had accredited teacher training. The Government response to the 2012 Review of Business-University Collaboration (the Wilson Review) stated that they were exploring whether they could usefully provide additional information for students including "encouraging HEIs to publish anonymised information about the qualifications and expertise of their teaching staff and to publish summary reports of their student evaluation surveys of teaching on their websites". They said that HEFCE was developing options for such provision.[192]

145. We considered whether the Government or HEFCE should play a greater role in improving the quality of teaching in HEIs. We concluded that they should not on the grounds that HEIs were primarily responsible for the quality of teaching. However, we look to HEFCE to take steps to ensure that the REF does not act as a disincentive to HEIs to promote quality in teaching.

146. We recommend that the number of lecturers that have received teacher training during the course of their careers should be set out in the Key Information Set (KIS) (see paragraphs 151 to 159), along with information about the training received, and we urge HEIs to offer an accredited course on teaching which all academic staff would be required to complete.

147. Student assessment of staff performance and teaching quality should be applied across all HEIs. We recommend that HEIs should have a robust system in place for assessing the quality of teaching including an anonymised and standardised assessment by students. The anonymised results of such assessments should be published in the KIS at a departmental level. QAA should be charged with reviewing whether HEIs have appropriate systems in place to achieve this and that the assessment of teaching quality is fit for purpose.

The role of students in driving up quality of provision

148. The HE reforms seek to give students the power "to prompt quality investigations where there are grounds for concern". The proposed changes to quality assurance would continue to allow students the ability to trigger out-of-cycle investigations under the QAA "concerns" scheme so as to provide an early warning sign that quality and standards might be at risk. The changes seek to raise awareness of this scheme. Students are also able to raise concerns through the Office of the Independent Adjudicator (OIA).

149. HEFCE's consultation also proposes the establishment of an annual process for scrutinising key data and information which could prompt an investigation. Such data could include feedback from students through the National Student Survey (NSS) data.[193] In addition, the proposals seek to ensure that there is continued student engagement in quality assurance and enhancement processes more generally. The recently introduced Institutional Review was designed to "embed the principle of full student engagement in quality assurance", outlining ways in which students could engage in the process.[194]

150. We asked our panel of students about how they judged quality within their institutions and if they were aware of how they could influence quality of provision. Their views on quality varied. Preferred measures for quality included access to facilities,[195] and how well their studies prepared them for work[196]. Anecdotal evidence from the student panel suggested that many do not know how quality is measured in their HEIs. Will Evans, for example, a third-year biochemistry student from Imperial College London, told us: "I did not know what the QAA was until I was invited to this meeting".[197]

THE KEY INFORMATION SET (KIS)

151. Part of the move to a stronger focus on involving students and aiding student choice to drive up quality is through the provision of more information about courses. This has been done through the development of the KIS, which HEIs must provide for prospective students.

152. The KIS will outline information that students have identified as useful to them in choosing a course. These areas are:

student satisfaction
course information
employment and salary data
accommodation costs
financial information, such as fees
students' union information.[198]

153. We note that the QAA's proposals for the use of data to trigger out-of-cycle reviews do not include referring to wider data held in the KIS, other than the NSS. This is because, they say, they "do not consider ... [the KIS] to offer comparable, well-understood, established, valid or reliable proxies for the quality of teaching and learning".[199] The HEA also concluded in their Dimensions of Quality report that it seemed unlikely that comparative indicators of quality currently available could provide prospective students with a valid basis to distinguish between individual courses with regards to quality.[200]

154. Concerns have also been expressed that focusing on the NSS and, therefore, on student evaluation of their "experience" might be at the expense of taking into account learning outcomes, and that students may not always be the best judge of their own educational or employment interests.[201] Furthermore, the student experience is "difficult to quantify". As Will Evans told us: "I have not been to any other universities so I have nothing to compare it with".[202]

155. The Imperial College Student Union told us that:

"the KIS is a welcome innovation which, if it contains the right information, will drive up quality and improve the decision-making process of prospective students. However, most current students are not aware of the proposals. It will be a major challenge to encourage prospective students to use the objective information offered in the KIS over the subjective information they are bombarded with from friends, family, peers and prospectuses."[203]

156. They proposed that "more fields be introduced to the KIS to allow prospective students to compare teaching quality, such as: a percentage breakdown of teaching mode (one-to-one, tutorials, lectures, self-directed study, fieldwork)", amongst other measures.[204]

157. The KIS will require HEIs to focus effort in key areas and inform student choice. However, according to the National HE STEM programme:

"initial 'mock-ups' show only an initial analysis of perceived quality and short-term measures; the longer-term benefits to the learner of higher education study are not included. For example, only six-month post-graduation employment data is currently shown, but the full benefits of STEM study may not be evident until several years after graduation. The provision of such longer-term careers information for graduates would allow prospective students to assess and compare the value of studying STEM programmes."[205]

158. Amran Hussain, a biomedical sciences graduate, told us that other outcome factors should also be included in the KIS, such as which skills would be gained from which courses, potential career paths, as well as more information on the destination of graduates.[206] The Higher Education White Paper recognised that, because of the lack of data currently available, this is an area of weakness. We are not convinced that HEIs are yet in a position to provide students with the information they need. We agree also that the KIS should be extended to the postgraduate level.

159. The KIS is a good starting point to help to ensure that students have the information they need to make an informed decision about their courses. However, the value of some of the information offered is not clear or sufficient to enable a student to make an informed choice about the quality of provision delivered by their course. The Government should ensure that the information provided in the KIS gives students the information they need to make an informed choice about the quality of their course. We recommend that the KIS should contain more detailed information on destination data beyond six months, as well as career paths; other measures of quality (including teaching); and more information on outcomes (that is, the skills that students will acquire during their studies). A similar KIS should also be available to postgraduate students with equivalent information on postgraduate provision.

Increasing employer involvement to ensure that graduates leave HEIs with the right employability skills

160. We have already said (in paragraph 128) that employers need to be more involved in setting of standards to ensure that graduates leave HEIs with the right employability skills. This is just one aspect, however, of the wider need for engagement. The recent Employability Skills Review, published by the National HE STEM Programme, recommended that the following steps should be taken:

"Encourage HEIs to explore ways of engaging with employers to develop employability support plans that will help ensure their graduates have the relevant practical skills that are required for the workplace; deliver an enhanced capacity for employer engagement supported by training and a commitment by employers to financially support programmes which provide clear benefit; encourage HEIs to utilise 'in-house' careers advice and guidance support resources; and increase HEI awareness of the developing methods of providing both direct and indirect experience of employers, and support their wider adoption across STEM."[207]

161. We received evidence of several examples of successful employer engagement with HEIs, such as industry representatives sitting on advisory boards within HEIs, and the Wilson Review also identified a number of examples of good practice. However, two areas in particular were drawn to our attention as being especially important for effective engagement. The first concerns the involvement of employers in accreditation by professional bodies; and, the second concerns the number and quality of work placements and sandwich courses. We discuss each of these below.

ACCREDITATION

162. The Government said in their Plan for Growth, published in March 2011, that accreditation schemes will give employers the confidence that graduates have the necessary skills.[208] A key benefit of accreditation is that it is informed by the needs of employers but is independent of any individual employer.Accreditation for a professional qualification (such as a Chartered Chemist, Physicist, Chartered Scientist or Registered Scientist) indicates the acceptability of a degree as part of the qualification route to professional status, which has built-in transferability.

163. Accreditation is seen to be a useful medium through which to engage industry in setting employability outcomes. For example, the Society of Biology said:

"when fully rolled-out across all the biosciences, accreditation will recognise outstanding biosciences courses across the UK that focus not only on core knowledge but also on experimental and analytical skills. It is our hope that Degree Accreditation will provide employers with assurance over the levels of laboratory and fieldwork experience provided by a degree, and the coverage of key areas of expertise required for further employment in specialist scientific careers. Accreditation will also make it easier for students to choose degrees which will equip them for future scientific careers."[209]

164. Mark Down, Chief Executive of the Society of Biology, told us: "professional bodies are well placed to facilitate ... dialogue [between universities and employers] because we have people who sit in both camps".[210] Dr Rob Best of the Institution of Chemical Engineers and the Engineering Council supported this view: "the Register Standards Committee of the Engineering Council is made up of about 50% of people from industry and 50% academics".[211]

165. The Engineering Council commented: "there is a very good match between the Engineering Council's published general learning outcomes for accredited engineering degrees and the employability skills cited by the CBI".[212]

166. The British Computer Society (BCS) told us that "computer science students do not always have the range of transferable skills required by industry, which includes entrepreneurial skills". The BCS, therefore, encouraged the "inclusion of transferable skills training in Computer Science degrees via its accreditation process".[213] There are clearly concerns in this area for forensic science students too. LGC, for example, commented:

"due to their broad and necessarily superficial coverage of multiple disciplines the majority of … courses do not equip students with the right fundamental technical skill-set for employment in our laboratories; it is therefore an issue of quality rather than number of graduates … The broadened interdisciplinary nature of many UK degrees means that the new recruits with these degrees rarely have sufficient practical laboratory skills or in-depth knowledge of fundamental science concepts to deliver the required quality of service, and need additional training in-house … It would appear that this position has arisen due to academic institutions interpreting what is required by industry without sufficient proactive industry involvement in stating requirements for recruits and the opportunities available to them."[214]

We therefore welcome the fact that the Forensic Science Society has a programme of accreditation of forensic science courses, currently accrediting courses in 23 HEIs[215] out of 51 HEIs that offer courses in that subject.[216]

167. Employers also supported accreditation as a mechanism through which industry could influence the outcomes of courses to ensure that graduates had the required skills for employment.[217]In addition, many commented on the importance of accreditation in driving up quality of provision more generally, above that provided by the QAA.[218] The Wellcome Trust, for example, said that "the introduction of accreditation should help improve the quality of STEM graduates, promote best practice in STEM teaching and harmonise course content across institutions".[219] Dr Sudworth from Cogent also noted that the professional bodies had "a role in ensuring greater quality than in the QAA benchmark statements".[220] However, accreditation is not currently available for all courses or subject areas.

168. Efforts have also been made to improve engagement with employers at the postgraduate level through such accreditation schemes and through Vitae. The Research Councils fund Vitae to support the professional and career development of postgraduate researchers and research staff in HEIs. Vitae produce the RDF which describes the knowledge, behaviours and attributes of successful researchers.[221] The RDF was developed after consultation with HEIs and employers. It is endorsed by over 30 stakeholders including the Research Councils, UK funding bodies and Universities UK. Several witnesses praised the RDF and use it when mapping or reviewing training and courses in order to provide the skills in the RDF.[222]

169. Given the limitations on the role that the QAA plays in sign posting high quality provision, we believe that accreditation of courses by professional bodies would be a sensible way forward. Accreditation may not be possible for courses in areas where there are no professional bodies. However, for those that have professional bodies and do not already have an accreditation scheme, we would urge them to consider setting up such a scheme.

KITE-MARKING

170. In their 2011 Autumn Statement, the Government expressed their support for "the kite-marking of courses that employers value by science, technology, engineering and maths Sector Skills Councils supported by the Confederation of British Industry".[223]

171. The Science Council, however, argued that kite-marking of degrees for particular employers or sectors "could become bureaucratic, costly and fragmented and thereby fail to respond to the needs of either students and a very broad range of science employers". They suggested that kite-marking might be appropriate for vocational degrees but in reality very few STEM degrees fitted such criteria.[224] The Engineering Council also expressed considerable concern over kite-marking:

"The creation of a kite-marking scheme alongside the well-established arrangements for accreditation by professional bodies could well cause confusion, not least to potential students and their advisers. If the proposal is to be implemented then it is important that there is clarity about what it is intended to do and how it differs from professional body accreditation."[225]

172. When we asked the Rt Hon David Willetts MP for his views on the potential overlap in the aims of accreditation and kite-marking, he said: "we are moving to a much more open environment. ... I can imagine a university that was very keen to be able to say, 'Rolls-Royce approves of our engineering course' ... A lot more of that will be going on, and a good thing, too".[226] Rolls Royce, however, disagreed: "we are really not convinced that [kite-marking] will significantly enhance the quality or visibility of good courses, and would be concerned if this became either a bureaucratic exercise, or a distraction to universities".[227]

173. In our view, it would be overly burdensome for employers to kite-mark individually hundreds of courses in the UK. A better approach would be to involve industry through the accrediting bodies and for companies to state whether they supported the accreditation. Given the tension between accreditation and kite-marking, we invite the Government to explain the aim of kite-marking and what it is expected to achieve beyond that which accreditation by professional bodies already provides.

174. We recommend that professional bodies, such as the Institute of Physics or the Institute of Mechanical Engineers, should make further efforts to provide accreditation of different STEM subject areas to ensure that students have confidence in the quality of their chosen course and that they will achieve high quality outcomes in terms of skills and knowledge. For those courses where there is less of a clear link with a profession, we recommend that the Science Council consider whether it would be possible to develop a broader system of accreditation to ensure that graduates have the core skill set required of a STEM graduate. We further recommend that the Government should provide support for such activities in the early stages of development until they are fully established.

PLACEMENTS AND INTERNSHIPS FOR UNDERGRADUATES AND POSTGRADUATES

175. Many of the submissions we received commented on work placements as a key factor in ensuring that graduates acquired suitable employability skills.[228] The Wilson Review also noted that placements, internships and other forms of work experience were extremely valuable to students in terms of academic performance and employability skills, and that they improved employability opportunities. A report commissioned by HEFCE and published in 2011 found that "there appears to be evidence, clearest for sandwich placements, that a benefit of structured work experience is improved employment outcomes after graduation", and argued that "the priority for activity/interventions by the HE sector should therefore be to support work experience placements for students during their period of HE study so that they develop the employability skills employers require and begin to build a body of work experience in advance of entering the employment market proper".[229] Both the Dearing Report in 1997 and the Roberts Review in 2002 also recommended that more undergraduate courses should offer student placements.

176. The Russell Group, and many others, agreed: "a key aspect of developing employability skills for many students is the opportunity to gain first-hand experience of the workplace during their studies".[230] Southampton University told us that work-related experience "can be highly relevant and beneficial … for the student and a significant factor to bring them into STEM-related employment. It is also beneficial for the employer to identify new talent".[231]

177. Our student panel told us that carrying out a placement helped them to decide which career route to follow, as well as preparing them for the world of work.[232]Imperial College Union raised a similar point with regard to placements for postgraduate students. They told us that the quality of a placement can have a major impact on a student's decision about whether to pursue a career in research: "if the placement is poorly designed or uninteresting ... their perceptions of a career in research or industry can be changed permanently."[233]

178. The findings from a recent Science Council report support these views:

"research identified that for graduate internships, the number of vacancies in STEM industries seems to be much lower than in many other sectors, including finance and business and that it is easier for a STEM graduate to find an internship in a business-oriented environment than in a scientific or technical one. The research also identified that ... STEM graduates appear to be less likely than other graduates to pursue internships. Given the call from employers for graduates with higher levels of practical and technical skills, it was surprising therefore that there are very few genuinely scientific or technical internships for graduates".[234]

179. A number of submissions suggested that employers should be encouraged to offer internships.[235] The Wilson Review said that "to enhance graduate skills levels and ensure a smooth and effective transition between university and business environments, there is a need to increase opportunities for students to acquire relevant work experience during their studies", and that "ideally, every full-time undergraduate student should have the opportunity to experience a structured, university-approved undergraduate internship" and made a number of recommendations about how this might be achieved.[236] Placements should also be available for postgraduates.[237] The Wilson Review further suggested that the Government should support companies that host students on full sandwich placement years, or provide internships, through a tax credit or grant mechanism. For unpaid internships it recommended that HEIs use their Office for Fair Access (OFFA) funds to support students.

180. When we spoke to the Rt Hon David Willetts MP about the Wilson Review recommendations, he said that "there is striking evidence that industrial placements ... are concentrated in a very small number of universities ... [therefore] if they can do it other universities ought to be able to do it as well".[238]He also said that BIS were in the process of assessing the feasibility of the recommendations, but that he was not sure whether they would be sustainable.[239]

181. Alternative mechanisms for exposing students to the work environment need to be explored. The businesses to which we spoke emphasised the benefits of building better relationships and collaborations between HEIs and employers as a way of encouraging employers to take part. The Royal Society and the Wilson Review made similar points.[240] We are aware, however, that this issue is a particular challenging for SMEs.[241] Semta noted that "whilst the majority of large companies in the sector actively engage with universities, provide placements and employ undergraduates, graduate and postgraduates, this is not the case for the majority of SMEs who make up 99% of the sector. The problem for SMEs is one of perception in terms of barriers: e.g. relevance of HE engagement to an SME; value and return on investment by SMEs in recruiting a graduate; resources to support an undergraduate, work placement, internship, graduate training and post-graduate support". They were currently looking at mechanisms to encourage SMEs to engage.[242]

182. Our panel of employers indicated that economic incentives for employers to offer placements would be welcome but that more had to be done in terms of aligning the needs of students with those of the employers. Professor Chris Wise, founder of Expedition Engineering, suggested that what students were learning at universities had to dovetail with the work experience that employers had to offer and this would require a close collaboration between HEIs and employers.[243]

183. It is widely recognised that good quality, well-supervised work placements and internships increase the employability of undergraduates and postgraduates. Their availability is almost non-existent in the case of postgraduate provision and in recent years, placements for undergraduates have been in decline, from 9.5% of the total full-time cohort in 2002-03 to 7.2% in 2009-10.[244] The Wilson Review recommended that all undergraduates and postgraduates should be offered internships and that more sandwich courses should be offered. Whilst right in principle, given the current economic climate, it is unclear how this provision would be funded. In July 2012, the Government's response to the consultation on the Higher Education White Paper stated that fees for a sandwich course should be no more than 15% of the normal fees.[245] This is a welcome development. However, the Government did not specify how they intended to encourage or incentivise employers to offer placements.

184. We recommend that the Government, employers and HEIs find a way to incentivise employers, particularly SMEs, to offer more work placements, and encourage more students to take them up.

185. In order to assist HEIs in engaging with employers and in securing placements for their undergraduate and postgraduate students, we further recommend that a central database should be established to post opportunities for placements for undergraduates and postgraduates. We recommend that the Government extend the remit of the Graduate Talent Pool service to include undergraduates and postgraduate placement opportunities.

The role of the Research Councils and HEFCE in the quality assurance of postgraduate provision

186. The Research Councils fund around 25% of all doctoral students from UK universities of which around 76% are studying STEM subjects.[246] HEFCE also provide funding to HEIs for postgraduate provision through the research degree programme supervision fund (equivalent to £205 million in 2011-12).[247] They have a responsibility, therefore, in conjunction with HEIs, to ensure the quality of the provision they fund. They also have a role in ensuring both that high quality STEM applicants enter STEM doctoral training and that doctoral graduates have the necessary skills that employers demand. (PhD teaching funding is also allocated from several other sources, which have different methods to assess the quality of provision. These are not discussed in this report.)

187. The Research Councils use their own methods to assess an institution's ability to supply high quality of doctoral provision and the quality of the candidates they fund. For example, some Research Councils only consider applications from first degree graduates who have achieved a 2:1 or a first class degree.

188. Different types of doctoral training have distinct proxies of quality measures, for example Doctoral Training Centres (DTCs) have to meet certain criteria in order to receive funding from the Research Councils. They are reported to have improved the quality of doctoral training because of the criteria for setting up a DTC. These include a strong research environment, critical mass of PhD positions, delivery of set learning outcomes, and conditions for learning.[248] There are also other forms of high quality doctoral provision[249] ABPI told us, for example, that DTCs are valued by employers and by the students who undertake PhDs through them because they provide a critical mass of students who learn from each other and benefit from access to different disciplines in a general area of science and technology.[250] The Engineering and Physical Sciences Research Council (EPSRC) mid-term review of their DTCs concluded that the DTC approach was an effective way of training a cohort of students, and leveraging substantial industrial funding. RCUK noted that the Economic and Social Research Council (ESRC) centres have provided clear evidence that they can deliver the highest quality training provision.[251]

189. On the other hand, from 2012-13, HEFCE will use the Research Assessment Exercise (RAE) rating of HEIs, a process carried out every five years in order to assess the quality of research undertaken by HEIs to allocate a block grant to fund postgraduate provision according to an institution's research quality star rating.[252] The use of the RAE rating to allocate funding for doctoral teaching provision, as opposed to the number of students and the cost of course provision, was supported by RCUK, because it links quality of research to doctoral provision more explicitly.[253] However, it is considered by many to be controversial because it does not ensure that the teaching provision is of a high quality. Oxford Brookes University, for example, told us that "we are unaware of any evidence that demonstrates a correlation between quality outputs and the quality of postgraduate training", indicating that the existence of internationally recognised research at an HEI does not necessarily mean a high quality postgraduate learning environment.[254] Other universities and the QAA agree that research output should not be the only quality measure and that other factors, such as completion rates and quality of teaching, should also be taken into account.[255]

190. Based on the evidence we have received, we find it difficult to judge the processes used for the assessment of quality in postgraduate provision. Our impression is, however, that the quality of postgraduate provision is measured in an inconsistent way across funding bodies and warrants further scrutiny.

191. We recommend that the expert group proposed to be established to look at postgraduate provision should examine how the quality of postgraduate teaching provision is assessed to ensure quality and consistency of approach across funding bodies, and consider how measures of quality of postgraduate education that go beyond research excellence might be developed. In particular, we would urge the Research Councils and other postgraduate funding bodies to expand the quality principles that underpin the DTC model to other types of postgraduate funding provision.

DOCTORAL PROVISION MODELS

192. Research Councils split their provision into the following broad categories:

Collaborative doctoral studentships allocated through a variety of routes, such as for example Collaborative Awards in Science and Engineering (CASE) whereby an industrial sponsor collaborates through the co-supervision and sponsorship of a postgraduate researcher.
Doctoral training grants which are allocated according to evidence about the excellence of the research environment or in response to peer-reviewed proposals including a business case.
Research council proposals for a small number of projects in particular subject areas (otherwise known as project studentships).
Doctoral Training Grants or industrial doctorate centres.

193. Recently, some Research Councils have chosen to increase their provision through DTCs and industrial doctorate centres. DTCs enable concentration of effort within centres of research excellence with the flexibility to allow HEIs to offer four year postgraduate training to cohorts of students—thereby creating critical mass and the ability for students, as Professor Sir Adrian Smith noted, to "develop the advanced skills and knowledge they need to be successful in their career" over a longer period.[256]

194. There is considerable support for the concentration of postgraduate research provision within centres of excellence through the development of DTCs. Arguably postgraduate provision through DTCs has improved the standards of PhD training for students.[257] However, there is concern that PhD funding by the Research Councils is being squeezed as a result of the move to the provision of PhD positions through DTCs, with fewer grants available for PhD studentships.[258] The provision for each individual student through DTCs is more expensive than other forms of provision due, in part, to the flexibility and option to offer four year courses. For example, it has reportedly reduced the overall number of STEM PhD students funded by EPSRC.[259] In addition, EPSRC plans to discontinue the provision of project studentships on research grants and fellowships as a result of efforts to concentrate research funding in centres of excellence with a critical mass of research. Many believe that the removal of project studentships is a backwards step.[260]

195. This shift towards DTCs and away from project studentships has raised a number of concerns because it will be difficult to allocate students to small-scale projects which often lead to research breakthroughs. This will impact disproportionally on UK and EU students which make up the majority of project studentship placements.[261] There are also concerns that DTCs are too narrowly focused and skewed in terms of areas of science and geographical location which could have a negative impact on specific areas of research. For example, there is only one DTC for synthetic organic chemistry[262] and there are no DTCs for physics in the South East of England.[263] We note, however, that the other HEIs are able to collaborate with the DTCs.

196. There is concern, therefore, that DTCs will have a negative impact on the breadth of research that takes place outside of the centres, given that PhD students are often involved in new areas of research through project studentships on grants.[264]

197. It has been suggested that DTCs should be just one element of PhD provision, because there are high quality PhDs in universities that do not have DTCs (those, for example, funded through CASE Studentships).[265] Other proposals have been put forward. They include increasing industry collaborations;[266] the use of regional alliances to offer joint training to postgraduates;[267] the use of cohort-based training and four year funding but applied to wider subject areas;[268] and, the maintenance of an element of project-based funding for PhDs to provide additional opportunities in emerging research areas.[269]

198. RCUK assured us, however, that "research council funding for postgraduate research is not restricted to DTCs", and that "a range of modes of delivery of postgraduate research training are likely to continue to be needed to address research capacity, the requirements of specific disciplines, or the needs of users and beneficiaries of research".[270] Over the last ten years, the Research Councils have increased their overall spend on doctoral provision, and the number of students that they fund from 4,243 in 2000-01 to 5,430 in 2010-11, although there has been a decrease from a peak of 6,065 in 2007-08.[271] The DTC model for delivering postgraduate provision is a welcome development and we understand the rationale to focus on centres of excellence. But the DTC model should not be the only model if we are to retain a breadth of research excellence—not only within our centres of excellence focused on strategic research objectives—but also excellence in smaller research projects which often lead to important scientific breakthroughs. We question, therefore, why the EPSRC has removed the small, but vital, 2.4 %[272] of its doctoral funding from project studentships, given the important role they play in maintaining the breadth of excellence in the UK. It is not possible to tell from the data available whether other sources of funding give HEIs the flexibility to fund such studentships by other means.

199. We recommend that the Government encourage the Research Councils to preserve a variety of PhD delivery models to ensure that the UK's current breadth of expertise in science is maintained and that new areas of science are able to grow. We also recommend that the proposed expert group set up to consider the supply and demand for STEM postgraduate provision considers whether the current provision for funding doctoral study across funding bodies is sufficient to cover the breadth of excellent research across the UK.

143 BIS Committee, 12th Report (2010-12): Government reform of Higher Education (HC 885). Back

144 University of Manchester, University of Oxford, Council for the Mathematical Sciences, the Wellcome Trust. Back

145 Op. cit., the Roberts Review. Back

146 Op. cit., STEM graduates in non-STEM jobs; Op. cit., The demand for STEM skills; CBI, CRAC, National Higher Education STEM programme, Syngenta, ABPI. Back

147 QAA. Back

148 The Government. Back

149 Royal Academy of Engineering, the Science Council. Back

150 HEA, Graham Gibbs, Dimensions of Quality, September 2010. Back

151 HEA, Employability in higher education: what it is-what it is not, April 2006. Back

152 Imperial College London. Back

153 1994 Group, ABPI, LGC. Back

154 Institute of Engineering and Technology. Back