|
UNCORRECTED TRANSCRIPT OF ORAL EVIDENCE To be published as HC 1405-i House of COMMONS MINUTES OF EVIDENCE TAKEN BEFORE Education AND SKILLS COMMITTEE
RESEARCH FUNDING
monday 10 jULY 2006 SIR ALAN WILSON and PROFESSOR DAVID EASTWOOD
Evidence heard in Public Questions 1 - 93
USE OF THE TRANSCRIPT
Oral Evidence Taken before the Education and Skills Committee on Monday 10 July 2006 Members present Mr Barry Sheerman, in the Chair Mr Douglas Carswell Mr David Chaytor Jeff Ennis Paul Holmes Helen Jones Mr Gordon Marsden Stephen Williams Mr Rob Wilson ________________
Witnesses: Sir Alan Wilson, Director-General for Higher Education, DfES, and Professor David Eastwood, Vice-Chancellor, University of East Anglia and Chief Executive designate of the Higher Education Funding Council for England, gave evidence. Q1 Chairman: I think you are both aware of the great interest there has been since the Chancellor of the Exchequer made a reference to the winding up of the RAE exercise and its replacement. There was some surprise that the Chancellor had made that announcement rather than anybody else, but we will come to that later. Looking at this Committee's work on the funding of higher education and who should pay for it two years ago, it did play some part in the resolution of our discussions over variable fees and all that, which many people think was a very important milestone in the development of higher education in our country. Equally, if we do not get the research side and the funding of research right in this country, again that has serious implications for our university system. All this seems to be a bit of a rush. Somebody said to me at a recent conference that I chaired at the Royal Society - we seem to have most of the Vice Chancellors of the country at it - that this all stems from prior to the Chancellor's statement that three leading vice-chancellors went to see either the Chancellor or other people in the Treasury and seemed to persuade them that we needed some changes fast. Sir Alan, I cannot believe that that was the case, but you know how these stories emerge. Sir Alan, why is there haste about all this? It seems all to be in a bit of a hurry. Sir Alan Wilson: In terms of stories about three vice-chancellors, that is something I know nothing of, if it ever took place, so I start from that position. Q2 Chairman: So the Vice-Chancellors of Imperial, University College London, and Bristol did not go and see the Chancellor! Sir Alan Wilson: Well, nothing is impossible, Chairman, but I have no knowledge of it. That is all I am saying. In terms of haste, the notion that it was all very fast for that kind of reason we would say was not the case, partly because the document that was eventually produced is a follow-up to a document that was published in 2004, the original 10-year framework for investment in science. The policies that were further developed in the budget science paper were really a continuation of the policies that were announced in July 2004. From our perspective it has been continuing work. In terms of the Next Steps paper that was published with the budget - and in a sense this almost answers the question, "why the Chancellor?" - it goes back to the 10-year science framework. I think the Chancellor is anxious, as part of the budget, as I understand it, to have a comprehensive review of progress since the 2004 paper, and research was part of that. From our point of view it is an ongoing process, and we have worked with HEFCE all the way through that period in terms of looking at possible metrics and performance indicators. We have talked to Treasury officials and DTI officials. Much of what was presented in the press about the rush, and certainly the story about when DfES officials told them are simply not true. Q3 Chairman: So it is not true that you were surprised in the DfES! Professor Eastwood, would you know whether this was greeted with surprise in HEFCE? Professor Eastwood: I think there is a parallel story to the one that Sir Alan has just sketched. After the RAE 2001 the funding councils jointly set up a review of the RAE methodology under Sir Gareth Roberts; and on the basis of the Roberts recommendations, the funding councils agreed substantial changes to the methodology for 2008, including a substantial reliance on metrics in the 2008 exercise. At nearly the same time the decision was taken alongside the RAE in 2008 to run a shadow metrics exercise; that is to say to test in real time an alternative lighter touch methodology for research assessment. Indeed, work was in hand within the funding council, and between the funding council and other bodies, to build that alternative model. So there was a direction of travel here towards a robust RAE in 2008, on the basis of what we might broadly call the Roberts methodology; but alongside that to test and charter a new future for research assessment in the world beyond 2008. To that extent, what was announced at the budget and the announcements around the budget were constant in that direction of travel. Q4 Chairman: Professor Eastwood, was it not the case that at the time of the Gareth Roberts report something like 80 per cent of institutions expressed approval and satisfaction with the peer review and the RAE exercise generally. Was that not the case at the time? Professor Eastwood: It is certainly the case that in response to the consultation around Roberts there was a strong preference within the sector to retain a significant element of peer review, and that Roberts did; but alongside that there was the move towards a greater reliance on metrics, and a sense too that the available metrics would continue to develop both in terms of range and in terms of reliability as time moved on. I do not think that the position we are now in is anything more than an evolution - a substantial evolution perhaps but nevertheless an evolution of the position we were in around 2002/2003. That is reflected in the kinds of responses that are beginning to emerge from the sector. Of course, there was a flurry of excitement when the consultation document was published, and one would expect that; but I think there was a serious engagement with the issues raised within the consultation document, and something close to a settled view in the sector that the RAE 2008 is very important - it is very important that we get it right for a whole series of reasons - but that this would be the last RAE "in the current form". Q5 Chairman: To be honest about this, it is all about who gets the money, is it not? Whatever system you use, it is about who gets the money to conduct the research. Is it not the fact that the sensitivity is that if you change the rules you may be taking money away from one set of institutions or departments and giving them to others? At the heart of this is there someone in the Treasury or someone in the higher education world or someone in the Department for Education and Skills saying, "the money is going to the wrong people"? Are they saying that? Have we got to change it? Professor Eastwood: From the perspective of the Funding Council, a research assessment does three things. First, it identifies and assesses research quality, which is central to the Funding Council's commitment to fund excellence where it finds it. On the basis of that, and importantly, it has constituted a very important benchmark for the quality of research within UK HE. That matters not just in terms of research and research performance internationally, but also in relation to the branding of UK higher education. Thirdly, as you say, Chairman, it is an exercise which underpins the funding allocations that the Funding Council makes to institutions. We could have a long discussion about what might constitute the right kind of distribution funding, but I have heard nothing in the current debate that suggests that the broad allocation of funding is inappropriate. It needs to be dynamic and it will shift over time. There are a number of rather important debates around the funding of interdisciplinary and multidisciplinary research for example, and also about the resourcing of applied research. There are some areas where some concerns have been expressed, and expressed with some force. It is a large step from that to saying that there is profound dissatisfaction at the funding allocations that are emerging; on the contrary, the Next Step document was rather presuming that the funding allocations would remain broadly as they were. Q6 Chairman: Sir Alan, is it not the fact that if there was a discontent about the way research funding was being allocated - we have seen a dramatic change over the years in the number of 5-star and 5 departments; the number has increased very rapidly, and that may because research so much better or because academics have learnt to play the game rather better - or more academics have learnt to play the game rather better. What is at the heart of this haste? Everybody knew that there was a change in the system going on. Everybody knew that in Australia they were moving from a metrics system and planning to move to a peer-reviewed RAE system - Hong Kong similarly. There seems to be a movement in the other direction. We seem to be going into this metrics area against the flow - is that right? Sir Alan Wilson: I think it is against the flow in the sense that we have had twenty years' experience of the research assessment exercise. Q7 Chairman: Which everybody thinks is wonderful, and they are copying us! Sir Alan Wilson: They think that what has been achieved in this country since 1986 is very impressive. It may be that they have to proceed from the equivalent of a time base that we may have achieved ten years ago or something of that kind. Not uncontroversially - and in that sense you must be right - there has been a reasonable assumption that a new method of both assessment and allocated funds would be appropriate. In terms of your original question, there has not been a position in DfES or as far as I know anywhere else saying that certain kinds of universities should get more money and some should get less, because at the government level it has always been about policy. Professor Eastwood has indicated some of the current issues about funding in applied research, funding of interdisciplinary research; but the principle that the best research will be funded wherever it is, is something that underlines all of this. The scale of the exercise - which is why we look for a system that is less bureaucratic - is considerable. It is not just simply that the cost is measured - which is substantial but reasonable in relation to what has been allocated - it is the time and, in a sense, the way that it dominates the development of policies in particular institutions. If there is a simpler way of doing it, there is the possibility, as we said in the consultation paper, that we might be able to move from what is a fantastic platform that has been established to being ever more ambitious in the future. Q8 Chairman: So you do not think there is any truth in the assertion that the Chancellor and the Treasury might have been saying, "we want more applied research, more technology transfer, more team working across departments and across universities; we want to see research much more shaped towards what increases the wealth of the United Kingdom and much more practical outcomes". Do you think that has not been a Treasury view? Sir Alan Wilson: It may be a Treasury view in the sense that those concerns have been shared right across the sector and right across different funding agencies. Even the research councils, which you might say are primarily there to support basic research, have had an increasing concern with the mechanisms with which the results of that basic research are applied. I think that many would argue that the distance in time in terms of what used to be called a linear model between basic research and becoming useful is shrinking. I think it is a policy question not just for the Treasury but for all of us - how come the funding needs of research in the economy as well as blue-skies research and research in all kinds of public interests are balanced. At the end of the day it may go back to funding in another sense; that generous though the research budget is in this country relatively, there is never enough to sustain what everybody would like to sustain. Q9 Mr Wilson: This is not about a new way of handling research; this is about the Chancellor saving money, is it not? This is about the Treasury making a grab for £45 million in savings. Sir Alan Wilson: I think that all the evidence in terms of the Chancellor's support of science is that he has been committed to increasing budgets rather than saving money on research. In fact in the last two spending reviews he has actually ring-fenced greater than average increases for research funding, and certainly there is no evidence in any discussions I have had with officials that that situation is changing. Q10 Mr Wilson: In 2008 we are all going to see tighter times ahead in education. The Treasury needs to save money. They are looking forward to where they are going to save it, and this is just one of a number of areas that they are targeting, is it not? Sir Alan Wilson: I do not want to anticipate discussions on the comprehensive spending review that will take place in all departments, and certainly within DfES; and it is a matter of political judgment for our ministers at the end of the day to decide on these relative priorities. I would say again, Chairman, that I have no evidence that in any of the Government departments that are party to the Next Steps chapter, DfES, Treasury, Health and DTI, that anybody wants to do anything but sustain a strong research base and provide the funding for it. Q11 Mr Wilson: As the Chairman earlier indicated, there are high levels of satisfaction with the current system, so why does it need to be replaced? Sir Alan Wilson: In a sense, as I think I indicated earlier, the big savings for the community are less in terms of money, wherever the money savings are channelled - and they could be channelled into further research - it is the particular way in which it has dominated the time of many academics, and there is a good possibility, I would judge - and I am perhaps making a personal comment, Chairman - that it could increase research productivity. Professor Eastwood: The direct costs of RAE 2008 will be of the order of £8 million, the direct costs to the Funding Council. The £45 million is a calculation of indirect costs incurred in institutions in preparing for the RAE. Some of those costs are constant costs, costs associated with research management and performance management and so forth. I think there are almost certainly savings to be made here. The system has, as Sir Alan said, matured over twenty years, and some of those embedded costs can probably be stripped out. I would echo what Sir Alan said: that would be a saving that I see being redirected into the research effort, rather a saving that was stripped out of the HE budget. Q12 Mr Wilson: The consultation paper assumes that the RAE should be discontinued, and it seems to be on the basis of widely held views, or what people say. Where is the substance? You yourself said it is time taken by academics. Where is the substance? Where is the evidence for the supposition that you are making? Sir Alan Wilson: There is evidence that it is possible in principle to run a simpler system because of the correlations between the data reported in the consultation paper. Q13 Mr Wilson: Where do I actually see that evidence? Sir Alan Wilson: These are the models that are on the website, where the reference was given in the consultation paper. Indeed, in the Next Steps paper there are two graphs on pages 20 and 21 of chapter 4, which show in aggregate levels certainly some very high levels of correlation. In terms of the feasibility of using something like research income and then other indicators as measures of quality that can then be used for funding allocations, the evidence is there, and quite a number of people have believed for a long time that this would produce a simpler method that would free people for research, rather than run the process as it has been run. As we are all agreeing, none of this is without controversy. Any proposals will be controversial, and in any consultation there will be people who say, "Keep the RAE; it has worked very well"; and there will be others who will say, equally strongly, "the RAE is now in a diminishing returns phase; there is a simpler way of doing it; please let us do that". Q14 Mr Wilson: You are right that it is extremely controversial, and for that reason do you not think the Government should have made the case for change a lot more strongly than it has? Professor Eastwood: Some of the case was made in Next Steps chapter 4. There is an analysis there, and the consultation document rather presumed on that and did not wish to replicate it. The other thing I would say about the document that was published at the beginning of June is that it is genuinely a consultation document. It offers some illustrations in terms of types of models and in terms of funding outcomes of those models. It is a consultation with the sector, asking the sector to engage with the issues engaged with the analysis. I think that those involved in this process are fully expecting that the sector will come back, obviously with the critical engagement of the kind that you are suggesting; but also will come back with proposals that will take us forward in maintaining the capacity to assess research quality, to have a sensible framework for distribution of research funding, but to do so in a lighter-touch way than has been possible in the last few years. Q15 Mr Wilson: But it is not a consultation that keeps the RAE system on the table; it is a consultation on a metrics basis, is it not? Therefore it is not a consultation at all. Professor Eastwood: It is a consultation that asks a certain set of questions around the STEM subjects on the one hand, and the arts and humanities and cognate disciplines on the other; so it does recognise that methodologies for different disciplines may vary. To that extent it recognises that there is a varied disciplinary landscape with which research assessment has to engage. It is also asking whether or not we have the right sorts of metrics, or whether the sorts of methodologies we are envisaging will enable us to make appropriate assessments. You are right that it is not saying, "Do you wish to maintain the RAE in its current form?" However, it is I think inviting serious engagement, and it is also inviting respondents to think about proposals which might further elaborate the sorts of methodologies that are implicit in that document. A number of institutions, not least the research-intensive institutions, are beginning to engage with the consultation in precisely that spirit. Q16 Mr Wilson: What are you actually trying to achieve? What is the end game for this metrics-based system? What is it that you want it to do that the system is not doing already? Sir Alan Wilson: It will be less bureaucratic; it will cost less; it will take less staff time, but still support excellence; and the correlations that I have talked about suggest that it can still support excellence. It encourages ambition, or facilitates ambition, by taking away people from the formalities of the RAE. It responds to what is becoming an increasingly important disciplinary research agenda. It connects to the subject that Professor Eastwood talked about, which is the relative amount of support for applied research or user-led research. That is very important for all of us. The extent to which metrics relate to that should, at the end of the day, be part of the funding formula. That is a policy decision for our ministers at the end of the day. This approach will facilitate the examination of alternatives. Q17 Mr Wilson: I understand it is less bureaucratic, it costs less and will support quality, but, as the Chairman said, in Australia that is not what it achieved. Lots of research papers have been produced, but the overall quality diminished, which is why they are switching systems. Professor Eastwood: We have made very clear in the documents that we need to have a methodology that is capable of robust assessment in the manner of the RAE. We also commit ourselves in the consultation document to model the likely effects of any change in the system; and indeed that is one of the things we are consulting on as well. We are aware that changes in the assessment methodology and changes in the funding methodology will lead to changes in behaviour. Some of those we might wish to drive; some of them we might wish to inhibit. We are aware of the issues that you raise. In modelling the shadow exercise for 2008, and then in evaluating its likely effect, those kinds of questions will be to the fore. Q18 Mr Wilson: I have two questions in one, coming back to the evidence: what evidence do you have that the metrics basis is more efficient than the RAE system; and what evidence do you have that the metrics basis is less expensive? Professor Eastwood: The evidence for the latter, that it is less expensive, is that it will enable us to run the RAE without the current elaborate infrastructure of panels. We think it will also mean that the amount of preparations that institutions do for an RAE will be diminished. It will not be eliminated but ----- Q19 Mr Wilson: Do you have any estimates of the savings? Professor Eastwood: No, we have not, but then until we have devised the model we will not be in a position to ----- Q20 Mr Wilson: It is purely speculation that it will be less expensive because you have cut some areas! Sir Alan Wilson: Intuition takes us quite a long way. I agree with Professor Eastwood that calculations will eventually be done, but the present RAE in place involves something like 82 different panels or committees, with quite substantial numbers of members on each, and in a sense there are their own administrative demands to go with it. That, alone, is taking quite a lot of academic staff time because it is the academics that are staffing these 82 panels. We are confident that there are gains. What I still feel is just as important as any monetary gains which might be reinvested are the gains in time. The academics that are on these panels are often at the height of their research careers, and it will simply free up some of their time to go back to the fundamentals of research. It is very difficult to measure what the value of that would be. Q21 Mr Marsden: Sir Alan, you know very well that there have been searching criticisms of the perverse incentives that the RAE has produced, not least by our predecessor committee in 2001 and by the Science and Technology Committee, and indeed across the system. Given that this is the case, why have you not taken the opportunity of this revaluation to do a thorough analysis of the existing behavioural impact of the RAE to inform any successor system? Sir Alan Wilson: I think the behavioural impacts have been reasonably well understood. The analysis of this kind of thing is never perfect, but it is about the transfer market. Q22 Mr Marsden: Would you like to tell us what they are? Sir Alan Wilson: The main one which is cited is, as it were, the transfer market in academics; the possibility that researchers who might have put in a certain amount of time in teaching are actually negotiating jobs for themselves that are wholly research jobs; and that may or may not be a good thing. It is very difficult to judge in general whether some of the consequences of the existing system are good or bad. You might argue that increased mobility between universities is a good thing in many ways. I think the general feeling is that overall, whatever the positive sides, the transfer market is too intense at the early stages of an RAE period. To my mind, one of the most important things, which, again, is terribly difficult to measure, is that the RAEs almost have to be organised in terms of traditional disciplines. The Funding Council has put an enormous amount of effort in trying to add a structure which deals with interdisciplinary research; but we are almost certainly moving into an era where in science subjects a large proportion of the research undertaken will be interdisciplinary. Q23 Mr Marsden: I would agree, certainly from my own soundings of academics and my own knowledge of the area, with all the criticisms of the existing RAE that you have enunciated. The point about the transfer issue is particularly relevant in humanities and particularly with younger academics. However, none of that answers the question as to why in that case is not made systematically here, and how a metrics system will improve on that. The criticisms that have so far surfaced have not been so much that the RAE is absolutely wonderful and should not be touched; it is that the RAE has significant flaws, which you yourself have underlined today; but there is nothing in what is on the table at the moment from you to suggest that those significant flaws will be addressed by a metrics system, is there? Sir Alan Wilson: What we have said in the consultation paper is that we are certainly aware of the consequence of the RAE as it exists in terms of perverse incentives, and any system - because as the Chairman said at the outset, it is connected to funding = is bound to have incentives relating to funding. We have identified in our own minds some of the dangers of the new system, an obvious one being that if Research Council income plays a major part in the funding formula, then research councils could be deluged with a much larger number of applications for research. Q24 Mr Marsden: The fact still remains that what is on the table at the moment - which I accept is consultation - is a leap from a system which has fundamental flaws, and criticisms have been made of it, to a system that has not been tested, where we have no evidence - this is all intuition or assumption - that it will not make matters worse. Sir Alan Wilson: In terms of the untested nature of it, quite a lot of work has been done on correlations and so on, as I mentioned earlier. One of the reasons why it is very important, and something that has been a plank of policy for two years or more - there should be a shadow exercise alongside the 2008 RAE - planned in the light of a consultation which is now on the table, will produce the tests, in a sense. Q25 Mr Marsden: Are you saying it is a "suck it and see" exercise? Sir Alan Wilson: I hope the consultation and the models that have been published with it make clear that while there is a reasonably general view among many people that a metrics system can replace the existing RAE, the tests that have been carried out so far show that it is a non-trivial exercise. There are a lot of these correlations and a lot of finer points that have to be dealt with. What we are trying to do in the consultation paper is to expose these so that the community can help us make progress. Part of the argument about haste, which understandably we keep coming back to, Chairman, is that to get this apparatus in place following a consultation, to have a good shadow exercise, which will be Mr Marsden's test in 2008, means that we are already on a relatively tight timetable. I would be reasonably confident that the consultation would certainly throw up something that we have not thought of at this stage, but will get us in a position by around the turn of the year where the Funding Council and Professor Eastwood can start to plan for the shadow exercise in 2008. Professor Eastwood: If we had been proceeding in a different sort of way, then I think it would have been a "suck it and see" approach, and of course that would have been wholly inappropriate; but to elaborate what Sir Alan has been saying, 2008 will run. It will run as a robust exercise, and it will run on the basis of what I only describe as the Roberts revision, and it will be taken forward by the funding councils. Alongside that we will then run the shadow exercise. We will evaluate that shadow exercise in the context of RAE 2008. That will give us the opportunity, on the basis of that real-time test of the alternative, to make some modifications and do some re-engineering, if that is appropriate. That is before it is run in earnest. That is an orderly process. At the moment, to construct the model that will be run in shadow form, then to run that shadow form to obtain an appropriate evaluation; then to make the amendments which seem appropriate prior to running the new model - as Sir Alan says, that process, while it might look as if we are moving swiftly now, is a process that we were embarked upon already in the ways I was suggesting earlier on, with work going on behind the scenes. It is coming into the full glare of publicity, but it was actually going on behind the scenes. Q26 Mr Marsden: Given all of that, how is this going to address the issue, which is often raised, that the RAE exercise, and for that matter the metrics exercise, will merely entrench rewarding research in universities that have excellence in them at the moment, and not encourage it in others; that you are going to create a two-tier university research system? Professor Eastwood: That is where the capacity of the existing methodology and perhaps an enhanced capacity in the future - a methodology to identify excellence is important. The distribution of QR, the form of funding that comes from the RAE, is broader across the sector and broader across institutions than the distribution of any other form of research funding; and there is nothing in what we are proposing which would disturb that. Stephen Williams: We have heard the Roberts review mentioned a few times. That was in 2003, and all universities are currently, so they have told me, in advanced stages of preparation for the 2008 RAE. Sir Alan, why has the Government decided that now is the right time to drop the existing RAE system? Q27 Chairman: Stephen is a Bristol Member! Sir Alan Wilson: The Vice-Chancellor of Bristol University has had well-known views on the RAE, much quoted! Having said that, I have now lost track of the question! Q28 Stephen Williams: Why now? Sir Alan Wilson: I was going to say that, as I indicated earlier, it is not so much now. It goes back to the 2004 10-year science investment paper that was published under the logos of three Government departments, and it was assumed then that there was a strong presumption that something like a metrics basis would take over after the 2008 RAE. It is not that this has just been thought of in a very short space of time; it goes back at least two years. In terms of earlier reviews, they are ideas that have been around for a long time. The particular issue now is really what I indicated earlier about trying to get a good shadow exercise in place by the time of the 2008 RAE. Q29 Stephen Williams: So when all these reports that we would have read after the budget speech back in March, after the Chancellor's announcement about people expressing surprise - all those people should have seen it coming in 2004. Sir Alan Wilson: I am sure they did see it coming in 2004, Chairman; it was very clearly there. I think inevitably when there is a publication it brings it to the top of people's minds and potentially affects funding, and people start to think about their future strategies. I think that that is very understandable. Q30 Stephen Williams: Sir Alan, did your department request this review, or was it acceded to a request from another department? Sir Alan Wilson: I do not think anybody has been acceded to anybody else; it has been a process that has been in place since the 2004 10-year framework. We jointly publish annual reviews in that. We published one, and I think the second one is about to be published. I think it was convenient, and important indeed to use the framework in the budget to draw a number of threads from that 10-year framework together in 2006, and research clearly had to be part of that. Q31 Stephen Williams: Chairman, there is some suspicion, as both witnesses will have heard from earlier questions, about which part of the Government is driving the process; whether it is the Treasury, which has the primary role, or the DfES. If you look at the timetable, it is all built in from the stop point, which appears to be the Chancellor's pre-budget report, which traditionally is in November in this country, feeding back to the end of consultation at the end of October. The consultation has just started. Has this whole process not been truncated just so that the Chancellor can have the answer he wants by the time he comes to his pre-budget report in November? Sir Alan Wilson: I do not think it is about the Chancellor's timing, Chairman. The fact that Professor Eastwood and I are sitting here before you now as co-chairs of the group that produced the consultation paper is a demonstration that DfES - and in its relationship with the Funding Council - is one of the lead partners in this exercise. We have certainly had representatives of the Treasury and the DTI on our consultation group, and indeed the other funding councils. What has come out has been agreed by all parties. Q32 Stephen Williams: Is it the DfES's consultation rather than jointly with the Higher Education Funding Council? Sir Alan Wilson: It is essentially joint. Everyone has an interest in it. All the ministers of the relevant departments have an interest in the policy developments. We have indicated areas like applied research and so on, which will be major policy questions in years to come. The implementation of the outcome of this consultation, the shadow exercise, will be the responsibility of the Funding Council. I think that is why certainly from the DfES point of view it is very important that Professor Eastwood is closely associated with this. Q33 Stephen Williams: Professor Eastwood, your predecessor from your new forthcoming role, Sir Howard Newby, has said that policy goals should come first rather than the detailed consultation on the types of metrics; that we need a review looking at what the UK needs from higher education research. He said that running a metrics review is no substitute for a higher education policy review. Do you agree with that? Professor Eastwood: I think the expectations of higher education in this sphere are well known and well articulated. The 10-year framework is one articulation of that; there was also an articulation of that in the HE White Paper back in 2004. I think that higher education is well aware of its responsibility to maintain the UK's position as number two in the international league table for research performance. I think the emphasis on translation of research is one that universities are well aware of, and indeed sympathetic to. The importance of maintaining funding for blue-skies research is right at the heart of the dual-support system. In that sense, the key policy drivers are very clearly articulated. Q34 Chairman: But are they, Professor Eastwood? We have still been trying to get to the heart of the policy drivers here. Is it saving money; is it better allocation of research resources across our country; is it international competitiveness? I am still not sure what is at the heart of the policy drivers. Professor Eastwood: I think it is all of those, and it is also having an assessment methodology and a funding methodology that is fit for purpose for the next ten or fifteen years. Q35 Chairman: It takes senior academics out of the loop! Professor Eastwood: I do not think, with respect, that that is what is being proposed. Q36 Chairman: Sir Alan was looking quite pleased; that all these academics are going to be released to do more research and not going to be involved in the loop. Professor Eastwood: Sir Alan can speak for himself of course, but there is a distinction between diminishing the weight of peer review, which is there in the current methodology, which is what Sir Alan was talking about I think, and maintaining the confidence of the sector in the system of assessment of funding that we evolve. That confidence will be there partly because the methodology will have been demonstrated to be fit for purpose, and partly because peer review does sit behind a number of the income metrics that we will be using. Research councils use very heavy peer-review for example. Here is a suggestion in the consultation document that there be an overall assessment of institutions' research plans and research directions. I think that in different ways there will be those kinds of inputs. The issue around metrics is not about a move towards an utterly desiccated system, but a move towards using some proxy indicators for quality. Q37 Stephen Williams: Amongst Professor Eastwood's list of reasons for doing it was to maintain the UK's competitive position as number two in the world after the US in world research rankings. Is not one of the advantages of the current system that a vice-chancellor, whether from UEA or Bristol, can go to China or somewhere else in the world and say, "I have got this 5-star department" in whatever it is, and everywhere in the world people will recognise that? They will also the system by which you have arrived at that assessment. Are you not at risk of jeopardising that world-wide confidence in what we have at the moment? Professor Eastwood: I absolutely share with you a sense of the importance of that ability to badge quality in the UK. There is no doubt about that, and I tried to refer to that in some of my earlier remarks. I would just make two supplementary points. One is that RAE 2008 will articulate quality in a different way, and so will not have 5 and 5-stars; it will have a graded profile. That was one of the Roberts responses to a previous criticism of the RAE and in particular the funding that flowed from it. That is to say it has a kind of cliff edge, and if you were 5, 5-star and then 4, there was a big fall; so the new methodology is designed to smooth that. There will be a different language of excellence emerging from RAE 2008. I do think it is vital, and the consultation believes it is vital that the new system should be able to identify, to describe and to badge research quality, for precisely the reasons you have given. Sir Alan Wilson: It is interesting in the context of this discussion that the analyses that establish this country as number two in the league table and second only to the United States is essentially a bibliometric analysis, an analysis of papers published in different kinds of journals, and then the number of citations - in other words, the extent to which this research has actually been used. That has been one of the main measures that has established the league table in which the communities have some confidence. The problem with metrics is that they do not cover the full range of subjects; but it is a field that is developing very rapidly, and this kind of international benchmarking - I think we are beginning to be reasonably confident - can be addressed through the use of those kinds of metrics. We are not just talking about research income; we are talking about available measures on a global basis of the value of research, as perceived through citations of published work. Chairman: Do you want to drill down on this metrics system - David? Q38 Mr Chaytor: Why are the Australians abandoning the system that we are now moving towards adopting? Sir Alan Wilson: In trying to answer the question, I cannot claim familiarity with the Australian metrics system - perhaps I ought to be able to! My guess is that the position that we have reached in this country has needed 20 years of the research assessment exercise to produce a foundation and move the university system in this country from where it was in 1986 in research terms to where it is now. I have no doubt that in that sense it has been very valuable. However, if I can make another personal statement, I believe that diminishing returns have set in, and there is a danger that particularly the failure to address the disciplinary agenda and to take on the biggest challenges because there is risk - and if you want to make sure you have your four publications for an RAE panel, you perhaps take less risk than at least we would want some of the community to take on. Q39 Mr Chaytor: What happens in the United States, or is the system so different that comparisons cannot be drawn? Sir Alan Wilson: It is essentially very different. You have a variety of funders. You have a different system for contributing overheads from government funding in particular. By that I mean the research funding through National Science Foundation and National Institute of Health and so on. It is not easy to draw comparisons. Q40 Mr Chaytor: On the arguments in favour of the metrics approach, you have cited costs, saving of time and diminishing returns and the changing nature of research and the growth of interdisciplinary research - but how exactly will some of the metrics criteria listed in the consultation document respond too the interdisciplinary issue more effectively? For example, how will the bibliometric approach reward interdisciplinary studies more effectively? Sir Alan Wilson: Can I add to that research income of various kinds, and take the two kinds of indicators? In relation to research income, research councils are increasingly funding interdisciplinary projects and those defined on an interdisciplinary basis rather than on a traditional subject basis. That will be reflected in research income so that universities that are more successful in winning funding for interdisciplinary projects for research councils ----- Q41 Mr Chaytor: Why can that not be adequately reflected by a process of peer review? Sir Alan Wilson: My own instinct about this is that there is a massive combinatorial problem. There are too many combinations. If you look within the sciences particularly at the combinations of disciplines that produce an interdisciplinary research project, if you wanted to find a panel for something like bioinformatics, which should be a combination of chemistry, biology and computer science, you would need a very large number of those panels; whereas experts in a field within a research council can find ways of doing that. If we look at the aggregate of all those ----- Q42 Mr Chaytor: At the moment presumably there are a large number of single discipline panels, but how many panels are there at the moment? Sir Alan Wilson: There are 67 subject panels and 15 so-called super panels. Q43 Mr Chaytor: Is it not a question of reducing the number of 67 down to a smaller number of broader interdisciplinary panels? Why would that not work? Sir Alan Wilson: I think what happens when you try to do that is that you always have a good number of cross-boundary flows, however you try to define that smaller number of panels. What we also have to bear in mind, Chairman, is that in the consultation document we have kept open the question of the possibility of having exactly those kinds of broader panels to take an overview of the metrics system that is developed, because obviously something which is entirely mechanism certainly in its early days could have dangers. I have not responded to Mr Chaytor's question, Chairman, on bibliometrics, which is another side of the metrics analysis. It is very interesting on a global basis: there are an increasing number of interdisciplinary journals, so if you are looking at successive citations from journals, the publishing world has responded to the interdisciplinary agenda very rapidly, and so it gives us a base which reflects the research frontier rather more accurately than traditional subject panels would. Q44 Mr Chaytor: Would that argument apply equally to the other criteria that are listed on the consultation documents, simply the raw numbers of research active staff or the number of some PhD students? You are arguing that increasingly research active staff would be engaged on interdisciplinary work, the numbers of research students would be engaged on interdisciplinary work and ---- Sir Alan Wilson: I think, Chairman, it is Mr Chaytor's question about panels turned into the way the metrics are used in a formula, how broad are the groupings and that is something that has to be explored, so in the models that we presented and put on the web site for illustrations we have actually looked at different kinds of groupings, but I cannot say at this stage that we know what the final recommended answer will be and that is why we have actually put alternatives on the web site to inform the consultation. Q45 Mr Chaytor: In terms of the broader issue between the role of metrics and the role of peer review, do you think it is absolutely an either/or choice to be made? Sir Alan Wilson: It is not an either/or, firstly in the sense that Professor Eastwood referred to in the context of another question and that is lying behind metrics, whether it is bibliometrics or research council income, there are different peer review processes, so it is not as though these numbers are not, as it were, products of peer review, but it is a different kind of peer review, so in that sense we are still connected and, as I said earlier, it is possible that certainly in some areas particularly arts and humanities and perhaps in social sciences we still need panels to work with the metrics base. Q46 Mr Chaytor: What would be some of the disadvantages of moving into largely a bibliometric based system or a largely research income based system, what would you identify as the biggest problems? Sir Alan Wilson: I think at the moment I do not see huge problems. I mean I think it has to be put to the test in 2008 and I think we will learn a lot from that exercise. I mean all the signs are, because the correlations are so high and if I am right about the fact that many of these metrics are closer to the research frontier than traditional subject panels might be, then I think on balance it is more likely to be beneficial than not, but I think it is very important, for instance in terms of perverse incentives, we have recommended, or we have put in a question since we are not convinced that we will have seen at this stage all the possible perverse incentives that this has to be monitored very carefully. We know what has happened in the past, if something goes wrong, as it were, with this one in terms of incentives ---- Q47 Mr Chaytor: In terms of incentivising behaviour and rewarding behaviour or discouraging behaviour, there must be some underlying assumption about the kind of behaviour the Government wants to encourage and discourage, so what do you imagine the outcome will be if post-2008 there is a shift to a system that is significantly based on metrics and what kind of behaviours will be encouraged and discouraged? Sir Alan Wilson: I mean in one sense I do not think there will be a huge change in that, I mean it is not to say that it will not be possible for new entrants to emerge more easily than it may be the case in the present system, but what we have succeeded in doing in this country is generating a community of top class researchers and I think those top class researchers will still present themselves through whatever kind of metrics are used. Now many of those are now working in interdisciplinary teams, but I suspect if you went back ten years they were still working, they were beginning to work in interdisciplinary teams, but they were presenting themselves to subject panels and I am sure subject panels have done their best in assessing the quality of interdisciplinary work, but I think those are the kinds of shifts that we will find. Q48 Mr Chaytor: In terms of the distribution of research funding this must inevitably lead to a further concentration? Sir Alan Wilson: I think that is actually certainly not the case in terms of the word "inevitably". Q49 Mr Chaytor: Is it likely to lead to further concentration? Sir Alan Wilson: I think if you actually had the patience, and it probably does need patience to look at the large numbers of tables on the models that we have used for illustrations on the web site ---- Q50 Mr Chaytor: I think I am actually going to give that a miss actually. Sir Alan Wilson: Probably more of them have moved away from increased concentration and in fact there has probably been slightly less concentration, if my memory serves me correctly, so it is certainly not inevitable. Q51 Mr Chaytor: Slightly less concentration, because we still have the RAE system and a peer review system. What I am trying to get at, is there not a correlation between those who are in favour of greater emphasis on a metric based system and those who are in favour of a greater concentration of research funding? Sir Alan Wilson: Is there a correlation? Q52 Mr Chaytor: Is there a correlation between the two? Sir Alan Wilson: Yes, I mean I am sure there is a correlation between them but, at the end of the day, it is a policy question for Professor Eastwood and his council colleagues because it is the weightings and the funding formula as much as the assessment and the metrics that will actually determine the degree of concentration. Whether it is metrics or RAE neither of the systems determines the degree of concentration, it is the weightings that go into the funding forum. Q53 Chairman: Sir Alan, you know a famous scientist, Joe Schumpeter said "politics about who gets what, when and how" and two and a half years ago when we looked at higher education one of the things that really worried us was a very well known vice chancellor came into this room and said he believed that a much greater concentration of research in a handful of research, which at university is a handful, and I said, "Do you actually mean five only?" and he said, "Yes, five", and our report said that would mean that most of the research in research funding universities would be in London and the South East, it would cut out Leeds, your old university, it would cut out your university, the University of East of England and we would have that concentration and then what we said is if there was not a research rich science base in each of the regions of our country, it would be a very retrograde step. Are you telling me that there is not something going on here, whether you call it metrics or the changing system, that actually is not the agenda of a small group who still want - they are very articulate, they are not secret about this, Sir Alan - who want research based to be based on a handful of universities; come on? Sir Alan Wilson: I fully understand the question, Chairman, and I would say very clearly that there is no hidden agenda of that kind. Q54 Chairman: Not from you, no. Sir Alan Wilson: The policy of the Government is to fund the best research wherever it is found. Q55 Chairman: Even if they ended up in five universities? Sir Alan Wilson: I think that the system from where we now stand, you know, it would take a very different kind of policy change to concentrate research funding in five universities. No RAE or metrics based system of any kind that we have thought about would bring about that kind of concentration, it would need a very different kind of policy directive and that, to the best of my knowledge, is not under consideration by any of our ministers. Q56 Chairman: It is worth us bearing in mind what major players are saying? Sir Alan Wilson: Perhaps I could add, Chairman, that what I am arguing is in part connected to what has been a core principle for a very long time which is the best research should be funded wherever it is. You can then apply different levels of some selectivity, as it were, and still maintain that principle, but I think any level of selectivity, and in a sense I am repeating myself, you know would not generate research in five universities, it would need somebody to say, you know, "we will just fund research in five universities". I think the other thing, Chairman, is what you have said yourself about the regional dimension and as a government department speaking with someone from DfES I mean we are very well aware of PSA targets for economic development across the country and we are very well aware of the contribution that universities make to that agenda and so we actually have discussions, whether it is with the treasury or with COG or DTI about how HEs contribute to that and particularly the research agenda, so we would want to support, I am sure Chairman, research across the country. Professor Eastwood: If I could just add that the concentration of the kind that you were sketching would run counter to the considered position of the funding council, would run counter to the view, I think, of the research councils, both of whom see distributed excellence, if I can use that term, as fundamental to maintain the supply of outstanding researchers as well as meeting the regional case that you have articulated. I think it is also worth noting that alongside the counter investments we have been talking about when we have been talking about QR or the research councils, there has been a big commitment to invest in the research infrastructure in general and the science research infrastructure in particular and if you look at the pattern of that investment through initiatives such as SHRIF, that investment too in pretty heavy kit, again is distributed over a substantial number of institutions, so it seems to me that there is, as it were, a broad consensus around the way in which we are investing both in capital terms and in recurrent terms in the research base which reflects the funding of excellence, selectivity, but appropriate distribution. Q57 Chairman: I still doubt whether out of this Committee. and I have got a restricted group of people in this country, they really understand; a lot of people out there understand the research assessment exercise, peer reviewed the system committees, however many there are, and they understand that system works. Now this system called metrics people out there I do not think, and I bounced this question over the weekend, "What do you mean, what are you measuring? Mathematicians - what are they going to measure then and how do they know those measurements are fair? What is this metrics?". Professor Eastwood: It is certainly not a good dinner party discussion topic. Q58 Chairman: No it is not. Ordinary people in this country ought to be able to understand, it is assumably camouflaged by some sort of nomenclature that is impenetrable. What are you measuring? Professor Eastwood: What we are measuring is we are measuring the investment that is going into research through income measures, we are measuring the outputs in terms of the growing use of bibliometrics and the impact that they have and we are measuring volume in terms of the number of researchers, the number of PhD students and so forth. A number of those things have been there in the RAE before, they are not new, and a number of the peer review panels, particularly in the subject areas which were the first to develop bibliometrics, used those, and the economists would be a good example, used those as an aid to make their judgments about the quality of published outputs in 2001 and, to some extent even in 1996, and will do so again with the Funding Council's blessing in 2008, so metrics have been there and I think if you look at some of the indicators, notably bibliometrics, what they are telling us is now much more sophisticated than it was even five years ago, they are telling us very interesting things about the nature of impact of published research, when it has an impact, where it has an impact and it seems to me that what we are offering in 2008 is the possibility of a very serious evaluation of the sophistication of those sorts of metrics. Q59 Chairman: So all this, Professor Eastwood, much more sophisticated measurement, all these measures go into, and who makes the decision about allocation, because that is not neutral, is it? Does it go into a wonderful computer that says, "This is the distribution of resources for research", or does it go to human beings in committees? Professor Eastwood: Ultimately the responsibility for the distribution of research funding to institutions is statutorily the responsibility of the Funding Council. In coming to that judgment the Funding Council obviously is aware of the policy framework and is responsive to the policy framework and, as Sir Alan was saying earlier, and it is clear in the published consultation, that at an appropriate level there will be a number of panels to advise on the appropriateness of the metrics. Q60 Chairman: It is a big shift away from decision-making from academics to bureaucrats and quangos like yourself. That is the shift of power, is it not, you are taking away from academics, you are giving them much more time to play with their research, but the decisions will be made by people like you who really will jump to what the Government wants? Professor Eastwood: I think that is a very stark characterisation and not one I would necessarily agree with. I think the first thing to say is ---- Q61 Chairman: Professor Eastwood, I have been in this chair for quite some time and I am used to dealing with people in your job and I have to tell you that it is very different, if HEFCE does something, a whole bunch of academics who are independent ... Once you become a paid employee of HEFCE, you do become a very different kind of person, do you not, you must realise that? Professor Eastwood: Perhaps about next year I will come and comment further on that. Q62 Chairman: It is interesting, is it not? On a serious point, Professor Eastwood, the sort of thing you have been telling this Committee today is not what Howard Newby was telling the Royal Society only ten days or two weeks ago when I chaired the meeting over there and it is certainly not what Gareth Roberts said either, so the view there is some kind of unanimity, your predecessor at HEFCE is not singing from your hymn sheet today and Gareth Roberts is not singing from yours, Sir Alan, that is the truth of it, is it not? You have come here, all agreed, everyone out there, all reasonable men and women agree with us today; you know that is not true? Sir Alan Wilson: Chairman, as I said to Mr Wilson, it is a highly controversial business and I think we can all agree about that, but that goes both ways round, it is not the successive proposals against traditional RAE, it is very complicated, but I think what I would argue, Chairman, and it goes back to your comment again about taking academics out of the loop, I mean I think what in a sense we are trying to do is to put the academics back into another and more productive loop in terms of research. There will still be bidding for funding from research councils, there will still be peer review there and indeed some of them, Chairman, will still be on Professor Eastwood's council so it is not simply, as it were, officials deciding what happens to the funding. I think what is important, and I go back to your earlier point on this, is that whatever we do and whatever the system produces in the end, is transparent and understandable and I do not think this has to go into obscure mathematics and in fact if you actually take the RAE in its present form, I think Professor Eastwood has been hinting at this, there is some subjective academic judgment in actually producing research ratings, getting from those research ratings and then sent off to use metrics, getting from those research ratings to a funding formula involves the same kind of complications, whether it is in the RAE framework or whether it is in the metrics framework and I think we do have a responsibility between us to explain. Q63 Chairman: Sir Alan, you know I am a great admirer of yours and of Professor Eastwood, but it is my job as Chairman to this Committee to tease out some of these commonsense things that people will ask and they do ask the thing, "Is this movement", like the learning and skills council in further education, "becoming the all powerful body?" Here in higher education are we seeing a transfer from a whole group of committees and academics making the decisions to a bureaucracy, wherever nicely described, the civil service, in your case Sir Alan, and a quango in your case Professor Eastwood, is that a shift of power and should people be worried about it? Perhaps they should not be worried about it. Professor Eastwood: I think in terms of the process that we have set out, the first serious commentary on the alternative will come from the shadow metrics exercise in 2008, it will come from our panels, so the first group to offer substantial commentary will be academics. The consultation suggests that there might be a different approach in STEM subjects from the arts and humanities and some of the non-quantitative social sciences and in the areas where we think metrics might be more robust and that will be tested in 2008, there will be panels to advise, so I think it is a process which might dial out some of the complexity and we hope might dial in some transparency, but I do not think it is a process which can be described, as it were, simply leaving the funding to upper ratchet. On the contrary, the judgments which underpin the metrics are all peer review judgments, whether it is the distribution of funding on the one hand or the decision to cite an article or research output on the other, so academics are absolutely at the heart of this and what we are trying to do is to find the most efficient way of deriving quality judgments from those data, ensuring that they are valid and validated and then the Funding Council, in a way which is appropriate, will translate that into funding outcomes as it has done since the 1980s. Q64 Mr Marsden: What I would like to do now, if I may Chairman, is to move on to some of the specific impacts of these changes both in the so-called STEM science technology, engineering and maths subjects and also in arts and humanities. Just to make the observation in passing to you, Professor Eastwood, and you talked about disputed excellence, but there is also of course the law of unintended consequence and I do not think anyone on this Committee doubts your bona fides in that respect, but we are trying to examine what some of those unintended consequences might be. The first question I have got is for you and it is very much focussed on the issue of arts and humanities. How can a metric system in any shape or form, however modified, replace the RAE when a metrics based system cannot deal with arts and humanities subjects which account for about half of the existing research activity? Professor Eastwood: The first thing to say in response to that is that HEFCE and the AHRC - Arts and Humanities Research Council - have set up a group to advise precisely on that and the view interestingly of the AHRC is that it is possible with or without some elements of traditional peer review ---- Q65 Mr Marsden: It is possible? Professor Eastwood: With or without some, and this will be determined, some elements of -- Q66 Mr Marsden: They are not ruling out right from the beginning metrics as an ingredient? Professor Eastwood: No, indeed they are not, and indeed the humanities' panels in 2008 will be using some metrics because all panels in 2008 use at least 15 per cent, or at least 15 per cent of their judgment is metrics based, so metrics are there and what this group chaired by Professor Michael Wharton of UCL will do is evaluate the way in which metrics, particularly bibliometrics, have developed in the humanities and feed in to the consultation in the autumn its judgments. Q67 Mr Marsden: Can I just ask you, I mean in terms of we toss this phrase "bibliometrics" around, but I know from a previous existence that journal articles carry different weights depending on the journals in which they appear; a very specialist journal article might attract a particular weighting, a more generalist journal article would attract a broader rating. Is that going to be spelt out, made more transparent under a metric system, a bibliometric system, any more than it is under the RAE system? Professor Eastwood: You mean the weightiness that particular journals might carry? Q68 Mr Marsden: Yes. Professor Eastwood: I think we can already see it in some areas of the social sciences and actually in some areas of the humanities that there are particular journals which carry particular weight, particular journals that academics are desirous of publishing in. Q69 Mr Marsden: And is that, to come back to the original point that I raised, is that transparent in the same way as we have league tables for schools, are you envisaging a situation where anyone could go along and say, "Professor X has contributed to that department because he got three points from", I do not know, "his New Scientist article as opposed to two points from a more specialist article on biochemistry", for example? Professor Eastwood: I do not think the one precludes the other because I think it is in the end about quality. I think the issue for the arts and humanities in particular is how far they are down that particular road and the kind of bibliometrics that are now increasingly available. For example, there are metrics around monographs becoming available for the first time which is obviously of a very considerable importance to the humanities. Having set up this group to advise, I do not want to pre-judge the outcomes. Q70 Mr Marsden: I am not asking you to pre-judge it, I am asking you to try and elucidate what is quite an opaque subject, even the term "metrics" is an opaque one and to try and translate what it would mean for your average academic or your average university department in, for instance, arts and humanities under a new system. I would like to ask you another question in that context and that is to say, supposing I am a young academic in my mid to late thirties and I am trying to decide, I want to do a big book, it might be a big history book or it might be a big book on physics or whatever, you know as well as I do that one of the criticisms of the present system is that the RAE forces that young academic to do chopped-up little articles in rather narrow journals rather than having the time to pursue bigger issues in a bigger book. Is a metric based system likely to, or a system that takes in metrics, likely to make that easier or more difficult for someone in the arts and humanities? Professor Eastwood: If the conclusion was that metrics can measure the impact and quality of monographs, which is an acclaim which is now being made, then if there was a problem there it will start to address that problem. My own view is that some of the effects of the RAE in this regard have been exaggerated, particularly in a seven year cycle it is perfectly possible to produce a monograph to produce major articles and even to produce articles for noble publications. Q71 Mr Marsden: This is not what many of your historian colleagues say, people in the Royal Historical Society, the Institute of Historical Research and in other organisations who are concerned that the current system in RAE is stopping some of those big books being produced by younger academics, let alone a more metrics based one? Professor Eastwood: I think I would say with the greatest of respect to my colleagues that --- Q72 Mr Marsden: You acknowledge people are saying this? Professor Eastwood: I acknowledge people are saying this, but I think it is a misdiagnosis, I think there are a number of wider trends which have little to do with the RAE and that are attributed to the RAE and what we are seeing, for example, in monograph publications, something which is happening in publishing, and the difficulties of getting monographs published, is much less an artefact of the RAE than an artefact of the publishing industry and the information revolution. Q73 Mr Marsden: Let me turn to the STEM subjects then. Do you accept, Sir Alan, that it is going to be a lot easier to produce a metrics system based on STEM subjects than arts and humanities? Sir Alan Wilson: It is certainly true at the present time, partly so because of the volume of research council and other research funding, so you potentially get more sensitive indicators and hence the enacted out consultation paper where we explore a wider range of indicators and input on arts and humanities. Q74 Mr Marsden: You said yourself earlier on that you had been influenced by the concerns that had been expressed about the current RAE system perhaps inhibiting collaborative and disciplinary research costs and STEM subjects on a number of different departments. Are you convinced that that particular issue, which is a real issue, is one that would be addressed by a predominantly metrics approach in those subjects? Sir Alan Wilson: I think it makes it a lot easier. I think research councils and other research funders are putting an increasing proportion of their funding into interdisciplinary projects and that would be represented in a funding formula. Q75 Mr Marsden: What would you say to those people who say that these proposals have essentially been drafted, or this consultation has essentially been drafted, with a focus on STEM subjects and that the arts and humanities have really been a tagged-on after-thought? Sir Alan Wilson: Well it is certainly not an after-thought, Chairman, and again hence the annex. I mean I think we wanted to produce a comprehensive response. I think perhaps I could make one related comment that builds on the big book question or the individual academic question. I mean one of the things that we have said is I think there is a danger in thinking that we are producing a system that focuses on individuals. It will actually operate at a much more aggregate level and what we have said in the consultation paper is that we believe that universities are now increasingly in a position where, because of past RAEs, they now have to manage their research internally and I think the young historian who wants to produce a big book, in that case I would be disappointed if the university management in either the department or the university was not, in effect, supporting that and I think that is one of the challenges. Q76 Mr Marsden: I would love to believe that and anecdotal evidence suggests in a sense that it does not always happen. Can I ask you, Professor Eastwood, a final question? When you were being pressed earlier about the possibilities of measurement you referred to "input" and "output", you said, "We want to be able to measure input, we also want to measure output". I mean output surely in terms of research also involves the impact of that research; I mean obviously it may involve something very directly like a scientific application, it also is an impact on various groups of people. Does not the impact of research also involve the impact on teaching and dissemination? Professor Eastwood: Yes, I think it does. Q77 Mr Marsden: In which case why are you not, or maybe you are going to, why are you not going to use the opportunity of this consultation to look at the impact of research on teaching and how you would measure that either within the existing RAE context or the metrics context? Professor Eastwood: I think I would make two comments there: one is that through a different initiative there is now a funding stream available to ensure that those institutions which have less QR, nevertheless ---- Q78 Mr Marsden: But a much smaller amount of money? Professor Eastwood: It is a smaller scheme, but nevertheless it goes in the direction of the point that you are making which is that higher education, the teaching that comes out of the research an informed environment, so that is the first point I would make. The second point I would make is in the current RAE, particularly in the humanities where what one has are, if you like, publications which represent a synthesis and the creativity is at the level of that synthesis, then they are publications which, quite appropriately, are assessed within the RAE. I think there is a deeper issue which we may return to on another occasion no doubt which is given that we might shift the focus to teaching, what is the appropriate funding for teaching which will facilitate that kind of research and teach, but that is a related, but I think a distinct question. Jeff Ennis: My first question, Professor Eastwood, is really a follow-on to the line of questioning that both the Chairman and Gordon has just been pursuing recently in terms of the utterances from Sir Howard Newby in terms of the way we are approaching this issue, because he said on RAE at the HEPI conference on 21 June that the order of any discussion must be to clarify policy goals first, then consider the RAE and then consider metrics - not to begin with a consultation on metrics which will impact on the RAE which will impact on research policy and we have obviously had organisations such as Universities UK, to name but one, who have been calling for a full debate on this issue. How would you answer those particular points? Q79 Chairman: When do you take over Professor Eastwood? Professor Eastwood: I take over in September. Q80 Chairman: Beginning of September? Professor Eastwood: September 1, yes. Q81 Chairman: Sir Alan, are you moving out of your present job? Sir Alan Wilson: I am afraid Professor Eastwood and I only overlap by one month because I will leave on 30 September. Q82 Chairman: And you are going to become Master of Corpus Christi in Cambridge? Sir Alan Wilson: Indeed. Q83 Chairman: Congratulations on that. Do we know who your successor will be? Sir Alan Wilson: Not yet. Professor Eastwood: I think it is wrong to see the consultation document in isolation, I think the consultation document has to be contextualised. "Next step" is a part of the immediate context for that, but the ten year strategy is there, as is HEFCE's own current corporate plan, so I think that the issue that Sir Howard rightly pointed to, is there a policy analysis, are the principles articulated? Yes, they are, they are in the ten year framework, they are in our current corporate plan. Is what is being consulted on running counter to those? No, I do not think it is. Q84 Jeff Ennis: I am trying to work out why an organisation like Universities UK would say something like that or Sir Howard Newby. Might they think that the Government is working to a hidden agenda here do you think or ---- Sir Alan Wilson: Can I comment on that and it connects to one of your earlier comments, Chairman, about academics being taken out of the loop which resonates very strongly with me? I mean I am rather surprised that UUK would, as it were, call for a full debate, because there is a sense in which the availability of the full debate is there almost continually. I would say the department through its ministers and through its officials and indeed the Funding Council have a pretty good record of, as it were, continual engagement with the sector and continuing consultation as is clear in the central discussion we have been having and very fortunately academics do not hold back, so if they actually have comments to make about government policy then we hear them. I actually do not feel we are in a vacuum. I agree with Professor Eastwood that there is a policy framework within which this is set and I do not think from the nature of our normal engagement with the sector that we are cutting off any debate and there is certainly no hidden agenda that we are trying to avoid. Q85 Chairman: The only slight worry that this Committee would have is we know that the ministers that we know in the Department for Education and Skills are more worried about those in other departments? Sir Alan Wilson: I can only say, Chairman, what we said at the very beginning in discussing this, that the departments are actually working well together. Q86 Jeff Ennis: Will metrics be consistent with the Government's policy of enhancing the capacity of the UK's research base in your opinion? Sir Alan Wilson: Again if I could start, Chairman. I think that the size of the research pot is, as it were, a massive argument in the next comprehensive spending review and I do not see any sense in which a change of methodology affects that decision, so I think decisions on support for research will be made in the usual way and, as I said in another context earlier, I think all the evidence is that there will be continuing support. Q87 Chairman: Do you agree with that, Professor Eastwood? Professor Eastwood: Yes, I do and I think one of the crucial outputs of the RAE and any successor to the RAE, is that within the framework of dual support institutions have resource that they themselves can direct, that is to say that they can invest strategically and I think it is getting that balance right which is critical and I think had we been doing this from scratch and been doing this in an environment where institutions were not yet used to research management, then I think we would all rightly be nervous, but I think what crucially this will result in as far as institutions are concerned, is QR as a block ground which will have a number of purposes. Underwriting the research base is one crucial purpose, but also enabling institutions to speculate, speculate in terms of blue-skies research is another and I think we need to preserve that and that is why I strongly welcome throughout all this the commitment on all sides to the maintenance of dual support which I think is absolutely critical. Q88 Jeff Ennis: Final question, Chairman. How would a metrics system achieve the high level of national and international buy-in that is required to maintain the reputation of UK research, the reputation that we currently enjoy? Sir Alan Wilson: I think in the end, as I said earlier in relation to our second place in the league table, it actually turns on bibliometrics and while in the past departments have promoted themselves in terms of their five or five star rating, that is not actually an international comparison because other countries, including the United States, are actually not doing that and there is no common base. If you actually look at bibliometric analyses of citations there is actually a common base, so I think as the expertise in that area develops it will actually improve our capability for international bench-marking. Q89 Stephen Williams: Can I just ask, Chairman, on a subject we have not really touched upon so far, you mentioned the fear that there may be five centres of excellence, if some people have their way perhaps a few more, perhaps two handfuls centres of excellence, whereas we have over a hundred higher education institutions in this country. I helped launch a report commissioned by 35 universities largely from the CMU group, but some other universities outside that group as well, where they demonstrated they get research income from the private sector, the NHS, the EU, but very little indeed from the Funding Councils, partly because their research often cannot be cited because it is contract based, therefore it is not in the public domain. Is anything we are doing in this review going to look at how research funding can reach the post-92 universities or are we going to continue to have research funding concentrated in Russell Group, the 94 group universities rather than spread out to other places that are still developing their research base? Sir Alan Wilson: I think it depends, Chairman, on the weighting of applied research by research matrix in a funding formula and we have been very open in the consultation paper that this is a possibility and because we value that side of the research spectrum, very much value the contribution of the universities that Mr Williams is talking about and I think that weighting is a policy question for ministers in the future. If you actually look at the illustrative models on the web site there certainly are model runs which distribute more research funding to CMU universities because client research funding is being so appropriately weighted in that illustrative funding formula, but at this stage it is not, as it were, for us to choose in policy terms, it is for our ministers down the line in institutional terms, it is for HEFCE down the line. Q90 Mr Chaytor: Sir Alan, one of the prominent arguments for a move to a metric system is the growth of interdisciplinary research. Why is there no reference to that in the consultation document? Sir Alan Wilson: I am sure there is, Chairman, when we re-stated the principles from the "next steps" paper and it is missing, then I am sorry, but ---- Mr Chaytor: I read it on the train coming down and I listened to what you said about it, and I skimmed through it again because I have got it here, and as far as I can see the word "interdisciplinary" does not appear once. Q91 Chairman: Sir Alan, I do not think you have got time to go right through it. Sir Alan, if you could drop us a line about that? Sir Alan Wilson: Yes, I certainly will. Q92 Chairman: Can I say that this has been a good session. Is there anything you want to say before we close this session? Sir Alan Wilson: No, I think I appreciate the questions from you and your colleagues, Chairman, it has been an interesting session, I agree with you on that. It is always valuable for us to be challenged and I appreciate that and we will learn things which we will take away and build into our thinking. Q93 Chairman: Professor Eastwood? Professor Eastwood: Can I thank you for the session and say I look forward to continuing the dialogue on this and other subjects. Chairman: We look forward to a long and happy relationship with you and you will have to live with us. Sir Alan, I have known you a long time in many roles and you have added lustre to all of them and can I wish you well in the new role and I think I said I will be at your new college only on Thursday night, so I shall suss out the catering for you. Good luck. |