Examination of Witnesses (Questions 60
- 79)
WEDNESDAY 20 OCTOBER 2004
PROFESSOR IAN
DIAMOND, MR
GLYN DAVIES,
MR ADRIAN
ALSOP AND
PROFESSOR PATRICIA
BROADFOOT
Q60 Chairman: Alpha ratings? What
percentage do you get there?
Mr Davies: Obviously it varies
from year to year but around 80% of applications are alpha rated.
Q61 Chairman: Are your plans anything
beyond asking for more money? Have you any other ways of getting
alpha rated project percentages up?
Professor Diamond: In many ways,
the only thing one can do is to try to find partnership funding.
The only other way you could do it would be simply to get fewer
alpha rated projects submitted to you. That would, I submit, simply
cut them off for somewhere else and frankly we would rather have
the opportunity to peer review the proposals that come to us rather
than not have that opportunity.
Mr Davies: In the late 1990s,
we had great concern about the number of applications we were
receiving which were not alpha rated. At that time, the split
was roughly about 60:40. We engaged in discussions with universities.
We changed our system of application to an open date system so
that the reader had a chance to look at the application before
they submitted it to us. That did work and it did reduce the number
of applications over a period of about 18 months to this 80:20
ratio rather than the 60:40 ratio which was, in our view, a worthwhile
development, because we were primarily cutting off applications
which were not going to be successful. If you are going to have
a healthy application rate, you are bound to have some applications
which are not of the best and we suspect that if we try to get
to 95% alpha we may be losing a few of the good applications as
well.
Q62 Chairman: You are getting more
applications coming in because more people want to do research
now.
Mr Davies: Yes.
Professor Diamond: We have a huge
community.
Q63 Chairman: Is the evidence that
after you rebuffed them once they would still come back again?
Professor Diamond: To be honest,
the ESRC funding is something that people really, really want.
If you go back into the universities often people will look very
carefully at the referees' responses, ask themselves what they
should have done and they will come back not with that same proposal
but with a new proposal which addresses a new area. People will
come back to us knowing that they have a chance. Perhaps people
who get responses which say, "This is a long way away from
the standard we require" may make a difficult judgment of
themselves but we have a community which does get funded sometimes
and not others.
Professor Broadfoot: I can confirm
what you have said, Ian. There is very significant status attached
to getting an ESRC grant. It is in itself a performance indicator
of your academic quality. It is also a driver, both directly and
indirectly, historically of RAE performance. I think people like
myself who put in for grants hopefully get them but even if you
do not get them it is something that you learn from and you develop.
It is part of life really that you accept a certain amount of
failure.
Professor Diamond: The important
thing is the mentoring of junior researchers because when you
are a junior researcher and you apply for your first grant and
perhaps it gets turned down this can seem like the end of the
earth. The ESRC is able to mentor a little in the universities.
Q64 Chairman: One of the factors
must be their academic record and they do not have one because
they are juniors. How do you compensate?
Professor Diamond: We are very
keen to encourage junior researchers to come for small grants
initially. We run a small grants programme which encourages junior
researchers. We are also very clear in our minds that, as part
of the discussions we are having at the moment, we may well have
a ring fenced pot of money for junior researchers just starting
out on their careers to apply on a level playing field with other
junior researchers for their first grant. If we did that, I would
like to include in that the opportunity to get networks together
and for some training courses, for example, in how to manage your
first research assistant. A person walks through the door for
the first time and says, "Hello. I am David. I am your research
assistant." What do you do? Bringing that kind of thing on
we see as very important.
Q65 Dr Turner: There seems to be
considerable variation between disciplines and their success rate
in applications: 54.5 for linguistics applications down to 16.7
for interdisciplinary and 15.3 for education. Can you account
for this bias and is there a bias against funding interdisciplinary
research, which is something we have suspected across other research
councils?
Professor Diamond: There is absolutely
no bias whatsoever. Often, the figures which are presented, which
are those for straight response mode, reflect the alternative
funds that exist. For example, we are putting £28 million
into education research,[3]
largely response mode. Those figures would not be included in
those data so take out the enormous number of education researchers
who have funding from a different source and this is the success
rate for another group. In linguistics, a pretty small community,
a small number of successes can influence those rates over time.
We have also identified examples in management research where
the quality of the proposals over a number of years was not great
and that meant that we moved towards an initiative, the Advanced
Institute of Management, to try and increase the quality of UK
management research.
Mr Davies: It has been a concern
of ours. We commissioned some research last year to look at three
subjects. We took sociology, linguistics and management, very
low relative success rate, sociology, medium, and linguistics,
very high. We were particularly concerned about learning whether
there was anything in our system which was biasing the outcome.
It is important to note from the figures that within subjects
they change quite a bit radically from year to year which it itself
important to show that there is no fix. There is no given rate
which applies. The research project came to four broad conclusions.
One, they found nothing in our system which was exceptionally
biasing one way or another, although there was some advice on
trying to ensure we had a mix of referees who were both focused
on the very specific subject and had a broader view. The major
determinants were the extent to which there was other money available
for funding outside ESRC in a particular research subject. In
subjects like management and education, we are a relatively small
part of the total funding available. You raise the question what
happens if somebody fails with ESRC. That means they have to go
somewhere else to get their funding, to a government department,
to industry, to somewhere else. That was a factor because the
reapplication rate in the lower success rates was less than in
those where they were higher. The second major impact was alternative
funding within ESRC. For example, we had the Advanced Institute
of Management initiative. We have the Teaching and Learning Research
Programme in education , so there are other pots of funding. The
final one of the three is about research culture in the departments
and there is some correlation between subjects which do less well
with us and, for example, do less well in getting high graded
departments in the RAE. There is something there which we are
addressing further because we need to think about it, but things
like the AIM initiative and the Teaching and Learning Research
Programme are working on the other side of ESCR in how we improve
the quality of applications coming from, for example, management
and education . We are certainly seized of the need to do something
further in the area of social work which is coming up very much
in our agenda as an area where we need to do something about improving
the quality of research and research applications.
Q66 Dr Turner: You have not really
addressed the point of the low success rate in interdisciplinary
research applications apart from to deny bias. You would say that,
wouldn't you? Is it the fact that you make sure that given that
the research is alpha rated and you have to make sure the peer
review system does not have any bias there, you then have mechanisms
for seeing that there are other sources which can fund this work?
What steps do you take to make sure that worthwhile things do
not fall down the cracks?
Mr Davies: The first thing is
that the number of applications which are classified using
interdisciplinary is very small. The figures arise because we
asked the applicants to advise us on what they considered to be
the primary discipline, the primary subject, and then several
others. We are merely taking what they say is the primary subject.
There are many applications which will say, "Primary subject
is sociology but this also involves psychology and linguistics."
That will come out as a sociology application for these figures.
Where they are interdisciplinary, it means that the applicant
has said, "I cannot give a single discipline or a mix of
disciplines. I am merely saying this is wholly interdisciplinary."
They are an odd collection of a fairly small number of applications.
We do look at them individually to see if there is something biased
in the system but I think it is about the nature of how they are
classified, which is a very small number. In the upper sixties%
of our applications have involved more than one discipline, including
disciplines sometimes outside the social sciences.[4]
I think it is around 45% of applications to us which involve three
or more disciplines. Most of the applications coming in to us
involve a mix of disciplines. This particular set is a small group.
Q67 Dr Turner: They are disadvantaged
because they feel unable to identify a lead discipline?
Mr Davies: I do not think they
are disadvantaged by that because we choose the referees on the
basis of those who know about the subject in which they are doing
the research. We do not have a disciplinary grouping. We merely
say, "This is research on rural studies." We choose
people who know about rural studies, about the subject, whether
they come from economics, from geography or sociology, just as
we would in any other area.
Mr Alsop: Where interdisciplinarity
is a question not within the social sciences but between the social
sciences, natural sciences and engineering, we are in regular
dialogue with our colleagues in the other research councils and
the Arts and Humanities Research Board to make sure things do
not fall down the gaps and neither do the applications subject
to double jeopardy. We have quite sophisticated systems, we like
to think, to make sure that that does not happen.
Professor Diamond: RCUK were invited
by the Council for Science and Technology over the summer to consider
across the research councils interdisciplinarity and how we funded
it. In producing a report which I am very happy to let you have,
we came to two conclusions. Firstly, that we did not have any
bias across the research councils in the way we dealt with interdisciplinarity.
Secondly, perhaps we had not publicised as effectively as we might
the way in which we dealt with interdisciplinarity. A challenge
for us over the next few months is to do that publicity. We feel
very comfortable that we do not in any way bias interdisciplinarity
and we would be very happy if you would like to come to Polaris
House and we could take this in greater detail.
Q68 Dr Turner: How does the move
towards full economic cost of grant awards impact on you?
Professor Diamond: I lead for
the research council as a champion for economic costing and Glyn
has been doing an enormous amount of work on this. We see it as
being a wholly desirable activity but one which is clearly a challenge
and one which is a partnership between universities. We are encouraged
by the extent to which this partnership is going forward and we
believe that when we start to receive grants under full economic
costing early in September 2005 we will be ready, the universities
will be ready and we will move forward in a very positive way.
How might it affect our applications? It may increase the number
of applications if universities decide to ask colleagues to put
in for grants which effectively pay for their salaries in a huge
amount. We have been honest about that with the universities to
say there is no extra money for volume of research. The extra
money is for sustainability of research and increased volume of
applications will lead to decreased success rates. The universities
have been quite clear in accepting that. I think the universities
will work very positively towards that. One slight area for us
which is important is the small number of grants we give which
are almost exclusively data collection. They are often quite large.
In that area, we are having discussions at the moment as to whether
we will try to fund the entire amount of the data collection because
it would be quite unreasonable for us to give a grant of 95% data
collection to a university and expect them to put a lot of money
into it themselves.
Q69 Mr McWalter: The notion of identifying
a lead discipline for genuine interdisciplinary subjects is nefarious
and I think it is really awful. I speak as someone involved in
the interface between computing and philosophy. I went to the
computer scientists and they said, "If you think that is
computing, you do not understand computing"; I went to the
philosophers and they said, "If you think that is philosophy,
you do not understand philosophy." The truth is that if you
have a really interdisciplinary project you are stretching the
bounds of what it is to do sociology or whatever and you need
somebody who is pretty good to understand that that is what you
are doing and nevertheless integrate it in so I would just urge
you, please, not to identify a lead discipline and look at these
projects in their own right as doing what we should be doing which
is stretching the frontiers of knowledge.
Professor Diamond: We believe
we do stretch the frontiers of knowledge, but we hear what you
are saying.
Q70 Chairman: I am trying to work
out whether you live in fear of getting too many grants or you
go out and encourage people to apply for grants. For example,
if you are at a conference and some bright young or old thing
gets up and talks about something, do you go up and say, "That
sounds really interesting. Why not put an application in?"?
Are you living in that kind of world, encouraging people to apply
or do you just suck it and see and hope?
Professor Diamond: In the last
six months, I personally have been in over 20 institutions as
part of the consultation meeting with social scientists and I
say that we want to receive their grants in a smooth and effective
way for them. "Tell us about any barriers that you perceive
in applying to us so that we can remove those barriers."
Our job very simply—we believe this fundamentally in ESRC—is
to encourage great research and for us to have the opportunity
to fund great research. We only do that by encouraging people.
Q71 Chairman: I am surprised you
did not say that when I asked you if the number of grants was
increasing. You did not say, "The reason for that is because
I am working my socks off trying to get people to apply."
Now you are saying that.
Professor Diamond: I am sorry
for not giving you the answer you particularly wanted but I have
done it.
Q72 Chairman: Is the stereotype of
the young PhD student in the 20s and 30s, single and so on, which
some people have spoken about, true or do you have a different
type of PhD student across the board with regard to gender, balance
and so on?
Professor Diamond: In social science?
Q73 Chairman: Yes. How do you see
your PhD students?
Professor Diamond: There are as
many types of PhD students in social sciences as there are people
out there on the street. There are certainly people, as I was
some years ago, straight from an undergraduate degree and a master's
into a PhD. There are other people often in some of the subjects
rated as professional practice who have been out pursuing that
professional practice and now see their careers moving into research.
There are other people who have been in a completely different
world who see an opportunity to come in. That is brilliant and
they bring those experiences to social science. What we must do
is have strategies which allow them to do that and to bring everything
to bear on social science over the rest of their career. If you
take one of the greatest social scientists ever, someone like
Titmuss, he came to academia after a long career elsewhere and
that is something we must encourage.
Q74 Chairman: That is music to my
ears because we have talked to all the other research councils.
Do you think you are unique in this aspect of your work in terms
of the recruitment of people who come to do the research?
Professor Broadfoot: It is a joint
effort between the Research Council and the community because
what universities have done in social science which they have
not done in other disciplines particularly is to provide opportunities
for part time studies. Social scientists have pioneered part time
PhDs, for example, and there are huge numbers of those.
Q75 Chairman: That is unique, in
a way.
Professor Broadfoot: It is less
unique than it was.
Professor Diamond: We have always
been very keen on part time PhD applications and fund them. They
come a lot in places like social work.
Q76 Chairman: Their success rate
of achievement is as high as others who are there full time?
Professor Diamond: Yes.
Professor Broadfoot: It is harder
to do.
Professor Diamond: We have an
assessment of PhD success rates and we sanction universities that
do not reach a threshold and part time PhDs, given a little extra
time obviously, are included in that section.
Q77 Dr Iddon: You have identified
four challenges for the next five or ten years. Could we ask first
how those challenges have been identified? Did you go out to the
research community?
Professor Diamond: Very much.
I have probably mentioned nine or ten times this morning this
consultation. It is not the case that we sit in Polaris House
saying, "What is the future of social sciences?" We
have gone out to the community and said, "What have the major
achievements been recently and what are the future challenges?"
The whole agenda around succeeding in this changing economy is
something that is coming to us from all over the place. The same
with population, the same with lifestyle and health.
Q78 Mr McWalter: I like what you
said about the first one. "The continuing potential disjunction
between the requirements of the research assessment exercise and
those of the research councils in areas such as interdisciplinarity,
applied research and research leading to professional practice
and engagement with users." You recognise this tension. Which
areas of social science have been particularly affected by problems
with the RAE? To what extent have these problems been resolved
by the plans for the 2008 RAE? What further improvements need
to be made and how effectively do you think the research councils
are in working together to address those issues?
Professor Diamond: Certainly we
would argue that some areas of social science have been affected
by the RAE, where people have felt the need to publish in professional,
academic journals rather than put things into professional practice.
Some disciplines have been affected more than others. I have already
mentioned development studies and we supported very much the Development
Studies Association bid for its own panel which would be able
properly to reflect the special issues there. On professional
practice, education is a classic example. Research related to
professional practice has been said not to be properly reflected
in the research assessment exercise which looks at academia. What
we have done to try to address this is we have asked John Furlong,
who is professor of education at Oxford and the immediate past
president of the British Educational Research Association, to
lead for us a small piece of work which is identifying quality
thresholds in research related to professional practice and in
applied research. John's report, which is in draft form and which
I have seen, is excellent. This provides a framework for a panel
assessing quality to be able to say, "This particular piece
of research which might not have appeared in a peer review journal
is a really high quality piece of work." We would argue that
in this area—we hope the framework will cascade into other
areas—the research assessment exercise panels should make
these guidelines based on things such as this widely available
as the guidelines which they will use to judge research which
may not be in the normal academic sphere, so that the very best
researchers can see themselves being properly rewarded by undertaking
research which may end up as a government report or a non-standard
piece of output. We have to do that or people will see their careers
needing to go in particular directions. We are very keen at ESRC
to be part of making that happen.
Q79 Mr McWalter: With the 2008 research
assessment exercise, you have made a contribution to how that
policy has evolved. Is that going to stop research being twisted
away from stuff that is important to stuff that is immediately
publishable?
Professor Diamond: At the moment,
we support the plans as they are. We are very encouraged by the
ideas of the big panels and the smaller panels and we look forward
to the next few weeks and months with the guidelines coming out
and hope that some of the things I have just said to you are reflected.
We did approach HEFCE about some of these issues earlier this
year and the welcome we received was warm and friendly so we feel
that we have a positive dialogue on this agenda.
3 Note by the witness: This is through the
Teaching and Learning Research Programme. Back
4
Note by the witness: In an examination of a large number
of applications are the last 18 months, 18% of applications had
only one discipline listed; 35% had two disciplines listed; 47%
had three disciplines listed. Back
| 
