Examination of Witnesses (Questions 122
- 139)
MONDAY 19 FEBRUARY 2007
MS GEMMA
TUMELTY, MR
WES STREETING,
PROFESSOR MICHAEL
ARTHUR AND
MS TABITHA
ALDRICH-SMITH
Q122 Chairman: The Committee welcomes
the witnesses to our inquiry. Some of them have been before the
Committee before and others are here for the first time. This
is not a Committee that tries to be unpleasant to witnesses; it
seeks to extract information from them and it always does it in
the nicest possible way. This is a very important inquiry into
higher education. If you remember, 10 years ago at the publication
of the Dearing report which had all-party support Lord Dearing
painted a 20-year scenario and said that after 10 years he wanted
a thorough inquiry to see how it was going. We have not heard
that the DfES or anybody else is doing that, so we are undertaking
it. This is a thoroughgoing, across-the-piece inquiry. We are
now writing up our views on Bologna and getting on with the main
part of the inquiry. There is no better way to start than with
student satisfaction and experience, and that is what we want
to get from you this morning. Perhaps I may ask whether one from
each side would like to make a two-minute introduction; otherwise,
we can go straight to questions.
Ms Tumelty: Thank you for inviting
us to be part of the panel. Obviously, this is something that
matters greatly to our present and future members. When talking
about student experience one tries to look at it in terms of expenditure.
That is not just about money, which I am sure we will come to
shortly, but about time. The expenditure of money and time encompasses
quite a lot of it. In our opinion, what it really boils down to
is that there is not really a homogeneous or single type of student
experience and we need to break it down a little. It depends on
numerous factors: the reason for entering higher education—what
you want to get out of it—who you are, your background and
whether entering higher education is a traditional thing or a
first experience. As you have seen from the National Student Survey
results, the higher education institution that one enters and
the type of qualification, whether it is a foundation degree,
access course or medical degree and so on, impact on the student
experience. It also depends on where one lives, which is a matter
we would like to deal with later, and the accommodation costs,
whether one goes to the private rented sector, private hall sector,
halls of residence or chooses to live at home. That may have an
impact under the new funding regime. We would like to move away
from the idea of students as customers and see them as co-producers
of their education. That is a huge part of the student experience
which inputs into academic representation and student representation
and means the student taking an active partnership role with the
institution or student union in what the educational environment
is like, which is quite empowering. Obviously, the student experience
needs to look at all aspects of student life: before one applies
to university, the academic experience whilst there, teaching
hours, contact time, quality of teaching, the pastoral support
that is received, any extracurricular activities in which the
student takes part, which are hugely beneficial, accommodation
and health, graduation and beyond. One also needs to look at student
experience with the benefit of hindsight. In summary, we want
to see a movement towards an experience which does not depend
on who you are and where you go.
Professor Arthur: I have submitted
a written report[24]
and will not repeat what is in it. From my perspective, the National
Student Survey has been a very significant addition to the sector.
I think it has been pretty successful considering how large and
complex the project is. In particular, I believe it is the first
time that the students of the nation have had a collective national
voice. When the results come out each time it is very important
for each institution to take the results and be seen to be responding
to them. There is evidence of significant enhancement of education
as a consequence of the results of the survey. That is particularly
pleasing because it was designed to act as an opportunity for
prospective students to see what other students on courses in
universities were thinking of those courses, but because it is
public, relates to a single point and is national there is good
evidence that institutions have been responding to the findings
of the survey.
Ms Aldrich-Smith: I noted that
at the beginning the Chairman referred to "each side".
As a representative of UNITE, a student accommodation provider,
rather than having "sides" in the sense of students
and universities—
Q123 Chairman: I meant to refer to
sides of the room. It was not an ideological observation.
Ms Aldrich-Smith: That is good,
because I want to advocate the opening up of the higher education
experience to business and the community as well. We believe that
the higher education experience is about the academic experience.
It is a social experience and independent living away from home
is absolutely part of the whole transforming experience. When
one is living away from home one is also working. Forty per cent
of students work part-time during term time. One is living in
a new community. 37% of graduates indicate a preference to stay
on in that community when they finish their studies. I believe
that the higher education experience needs to be broadened to
encompass those other areas, including businesses like ours which
are committed to supporting the higher education sector. As for
the debate about students as customers, we see them in that light
because they are certainly our customers. Whether universities
see that differently is another matter, but they are consumers
of the education experience and we must think about them in that
way, too.
Q124 Chairman: I should like to open
the questioning by asking Professor Arthur about the shortcomings
of the survey. I looked through it again last night and noted
that it did not cover all universities. It is a bit patchy. We
know that there are over 100 universities and some are not there.
I declare an interest as governor of the London School of Economics.
I was intrigued to see that the LSE is not there. In addition,
I was not quite sure whether this covered all students or only
some.
Professor Arthur: It is a survey
of undergraduate students in their final year, so it does not
cover postgraduate students at this point. To cover the postgraduate
student experience would require a very different survey. It has
been set up to cover the undergraduate student experience. The
shortcomings are fairly obvious and they are the ones you mention.
In order to report in the National Student Survey the current
request is that 50% of students on any one course should respond;
it should be at least 20 students. The reasons for those thresholds
relate to statistical advice when the survey was first set up
about three years ago. I would describe that as a platinum standard
of statistical validity, and inevitably a few institutions drop
below that threshold. You are quite right that there are three
institutions where the student unions have actively boycotted
the national survey: the Universities of Oxford, Cambridge and
Warwick. We have been working hard to try to persuade them to
participate, and I think we have had some success this year—the
proof of the pudding will be in the eating—with the University
of Warwick. I would now describe the University of Oxford as being
at least neutral and only the University of Cambridge has boycotted
the survey this year. I believe it is a great shame that they
have done that. This is an opportunity for those students to say
nationally and in public what they want to say about their institution.
Several issues concerned them originally: first, the invasiveness,
if you like, of the survey on student time. We have listened to
that issue and cut back the number of contacts we make with each
student deliberately to make it more student-friendly. They were
also concerned about reporting outcomes versus the level of expectation.
They had the notion that the latter would be much higher at Oxford
or Cambridge than at other universities. I believe that that is
complete nonsense. The expectation of all students going to all
universities is of a high standard. We have been working away
at that. I am a little concerned about widening participation
issues for those universities if they have students that come
from low income families and go onto the website and cannot get
information. I imagine that that would put them off. I have been
using that line with students and vice chancellors. I have spoken
to all three vice chancellors of those universities and am pleased
to say that they are now much more supportive of the survey than
they were originally. As to other limitations of the survey, certainly
the first incarnation of the website was not good enough. We have
recently recommissioned a new provider and the new website will
be very closely linked to UCAS. The new website will be run by
a combination of UCAS and Hotcourses so that students who go in
through UCAS, select universities, look at entry criteria and
are able to link up to the courses through the UCAS site can quickly
go to the new website, which will probably change its name, and
see what students think of the courses that they are looking at.
That will be a great improvement. It is very much a response to
the review undertaken this year by the quality assurance framework
review group.
Q125 Chairman: Mr Streeting, what
are your views on the fact that some universities do not participate?
Is this a conspiracy between the leadership and student unions
locally? Do you think that all universities ought to co-operate?
Mr Streeting: I am glad you have
asked that question. I think it is important to highlight that
in all three cases, and certainly in the case of the University
of Cambridge, where I was present at the student union in the
first year of the rolling National Student Survey, there was—who
would have thought it?—an enormous degree of complicity
between institutions and student unions in terms of opposition.
Student unions were certainly prodded and pushed towards a state
of active opposition, and it is important that that institutional
perspective is also brought to light. It certainly was not the
case that the student union avoided participating in or boycotted
the survey in direct opposition to their institution. There was
enormous complicity and it was deeply unfortunate. Professor Arthur
has touched on an important point, which I made early in the academic
year when the results were released, that in the case of Oxbridge
they have particular issues to deal with in terms of the ongoing
myths to do with the admissions system—the notion of the
old school tie and secret handshakes. They will do themselves
no favours when in future applicants click onto the UCAS website
and find NSS student satisfaction data for almost every HEI in
the country except Oxford and Cambridge. I believe that that sends
a very worrying signal. NUS very much supports the National Student
Survey and it was something for which we campaigned over a number
of years. When the survey first arrived we still supported the
concept. We were less sure about supporting this particular National
Student Survey, but many of the problems in terms of the intrusive
nature of the phone call and other methodological dimensions of
our concerns have been dealt with. To elaborate some ongoing issues
that need to be dealt with as we look to reviewing the NSS after
three years, one matter the survey does not do is report the results
of joint honours students in a meaningful way. Currently, there
is very little distinction or opportunities to offer students
on joint honours courses. For example, if I was reading history
and politics at the University of Leeds I might be very happy
with the politics component of the course but not necessarily
the history part of it. I cast no aspersions on those particular
departments; it is just an example. It does not help when reviewing
the data to drill down to find out where the problems are and
how the experience can be enhanced. Another area of outstanding
concern that sticks out like a sore thumb is not the methodology
but the level of student dissatisfaction in the areas of assessment
and feedback. That is an area of concern which institutions in
terms of the policy agenda locally as well as the sector nationally
really need to address.
Q126 Chairman: We published the pretty
wide-ranging nature of this inquiry into higher education, but
Ms Tumelty in her introduction and Mr Streeter just now have mentioned
some interesting broader concerns. There is almost an obsession
about student finance and fees in the written submissions. It
is extraordinary that for a union that is supposed to represent
students right across the piece the only thing it wants to give
evidence to us about in its written submission is fees.
Ms Tumelty: The remit of the inquiry
of the Committee is very far-reaching. There are many areas that
we would want to contribute to if we did not have such finite
research resources. To be fair, the issue of fees and funding
is consistently prioritised by our membership through the democratic
processes. I believe that we submitted something on Bologna as
well.
Q127 Chairman: Yes. We asked for
that separately.
Ms Tumelty: We do an extensive
amount of work on this and that is why we are pleased to have
this opportunity to come now. We will be able to follow that up
by backing up all the research we have done on it, for example
through the accommodation cost survey, some bullying research
done recently, work on health and other matters that we do in
the National Student Survey.
Mr Streeting: The context is really
important here. It would be unfortunate to suggest that the NUS
is the only organisation in the sector talking about fees and
funding.
Q128 Chairman: I am just saying that
your evidence concentrated only on that to the exclusion of all
the other areas in which your students might be interested.
Mr Streeting: If I am honest,
I believe we were surprised to see fees and funding included in
the inquiry at this stage, bearing in mind there is a review in
2009. It is absolutely right that we monitor evidence across that
period and see how the system beds in. Without meaning to be too
critical of the inquiry itself, at the beginning you made a comparison
with Dearing. We and many other voices in the sector are concerned
about the expansive nature of the review and short timescale in
which to respond. Look at the volume of work and evidence that
went into Dearing and all the staff support that went into it.
It is almost as if the Select Committee set out to produce a "shearing"
report but with a smaller budget and on a shorter timescale. I
believe that it is totally appropriate for the NUS to focus on
the big ticket issues such as fees and funding.
Q129 Chairman: I quite like the reference
to "shearing report", but the fact of the matter is
that the Select Committee does things differently. It conducts
its inquiries harder and faster, and it has a great deal of experience
in doing them. If before we finish this inquiry you want to add
anything to the evidence we shall welcome it. You have been around
long enough to know that that is not a hard and fast time limit.
Ms Tumelty: We shall definitely
follow up the points raised today, put it all together and submit
it to the Committee.
Q130 Chairman: But you must have
been encouraged by the way student numbers have held up in England
over the past decade.
Ms Tumelty: The NUS has always
campaigned for the expansion and widening participation in higher
education, long before it became politically expedient to do so
by the university sector. Last week we welcomed the increase in
applications. What we did not have at the time—I presume
that they are now up on the UCAS website as of Friday—was
the breakdown of the socio-economic groups, ethnicity, gender,
class and subjects. Obviously, we have been preparing for this
hearing. We shall look very closely at those figures and at where
the impact, if any, will be. Essentially, all of the issues to
do with fees, funding and sustainability of the sector are not
just to do with admissions; it is concerned with a number of factors.
Ms Aldrich-Smith: As to the broader
issues, in a normal distribution curve UNITE tends to look at
the tails to find the trends and what may be different. Certainly,
within the student experience the service that one receives from
university welfare services, availability of paid employment and
careers services are some of the matters that students in the
survey appear to be slightly more dissatisfied about in general.
Overall, satisfaction is really high, but thinking about how we
may be able to improve those matters as part of the student experience
could be areas on which to focus, as well as the big ticket issues.
Q131 Mr Chaytor: Mr Streeter, can
you say a bit more about the complicity between the NUS and individual
institutions in blocking participation in the survey?
Mr Streeting: It was certainly
not NUS complicity. I will hold up my hands and be honest. That
was a decision we took during my year in office as president of
the student union.
Q132 Mr Chaytor: To clarify it, you
advised your members at Cambridge not to participate?
Mr Streeting: Our student union
did so, yes.
Q133 Mr Chaytor: You now regret that
and would not advise any other student union in any other university
to do that?
Mr Streeting: Absolutely—and
I am doing quite the opposite. When the survey first began we
had a number of concerns about the methodology. To the full credit
of HEFCE and Ipsos MORI, the whole way that the survey is approached
has been constantly adapted and changed wherever possible without
affecting the validity of the survey and ability to compare data
over a number of years. Significant change has occurred. I am
far more comfortable with the NSS as it stands now and am incredibly
positive about the discussions that are taking place in the steering
group about the forthcoming review and where we might go in future
with an additional bank of questions. The survey has adapted and
evolved. It is unfortunate that institutions have chosen not to
take part. As far as concerned my university, there was a concern
about the time implications and how useful the data would be,
but one of the best things to come out of the NSS is not just
the usefulness of the data to potential applicants but the fact
it has acted as a catalyst to drive up standards in quality at
an institutional level. To the credit of many HEIs they have seen
the results and areas of concern, drilled down into the data and
worked with the students union. Where that has happened we have
had the most productive results in all sorts of areas, particularly
in assessment and feedback. I hope that that is reflected in the
future, but it has been a driver for improvement.
Q134 Mr Chaytor: My next question
to Professor Arthur is: are there specific examples where individual
universities have taken on board the results of the survey and
started to change their practices? Can one or two examples be
quoted?
Professor Arthur: Certainly. I
am sure that we can also let you have written evidence in the
form of a survey taken by HEFCE last year. Perhaps I may start
with my own university. We have improved our induction process
for students. We have started a new peer mentoring scheme where
existing students help freshers. We have started a new and much
more detailed campus-wide student survey asking questions of all
three years so we have local information and can improve things
before we get to the final year. There is a major review under
way of our internal learning and teaching processes. To pick another
example, the University of East Anglia comes to mind. They have
increased the amount of anonymous marking that has been going
on. It is very commonplace for universities to set up a specific
action plan related to the results of both the 2005 and 2006 surveys.
The results of both surveys are very similar, which is hardly
surprising. We had the results of the 2005 survey only late in
the year and the next survey starts before we can even change
anything. I am expecting most of the improvements to emerge from
this year onwards. I would encourage people to think in a five-year,
not year-by-year, timeframe. A number of the things that universities
are trying to change will take some time. I have here a list of
about 10 universities which responded, and I shall be very happy
to submit them to the Committee.
Q135 Mr Chaytor: But your steering
group has not actively recommended that there should be an action
plan in response to the survey?
Professor Arthur: No. The steering
group is concerned largely with making the survey run and trying
to refine it. It does not really have the power to do that. I
think it is inevitable. I know that QAA will do this during an
institutional review. It will have access to the results of the
National Student Survey: they are in the public domain. I presume
that it can also ask for access to the local results that are
available only through the dissemination website. I anticipate
that QAA will certainly be using the results of student surveys
in their overall assessment of the quality of higher education
institutions. If I was a member of the QAA I think I would be
asking harder questions of the three universities mentioned just
now—Oxford, Cambridge and Warwick—where the data is
not in the public domain. I believe that there will be a consequence
for those institutions that do not actively participate. That
is another matter that hopefully will persuade them they should
do so. I am delighted to hear that Mr Streeter is repentant about
his earlier activities.
Q136 Mr Chaytor: Ms Tumelty, in your
opening remarks you referred to students as customers. How do
you explain that the only university with an overall satisfaction
score of 4.5 is one where all the students are customers, that
is, the Open University?
Ms Tumelty: In that case some
of our feedback is concerned mainly with the style of learning
and flexibility because obviously it is done part-time. The quality
of the teaching materials is obviously one of the key points that
students have made. Other factors are student support and academic
and non-academic feedback, and also whether the provision is flexible
enough to fit in with their lives, which is another key issue.
I do not quite understand what you mean by your question because
obviously all of the students are customers.
Q137 Mr Chaytor: Do you not see a
relationship between these results and the fact that individuals
who follow an OU degree are contributing financially from their
own resources to that extent? They are putting a lot of their
own investment into that degree. Does that have any relationship
to the satisfaction of the student?
Ms Tumelty: I think that financially
all students now contribute significantly to their degrees, further
education course or postgraduate courses, but it would be really
sad if we lost the notion of learning for knowledge and learning
sake as much as for future earnings, experience and everything
else. If we go down the route of talking solely of customers there
is a very different relationship with the institution. The relationship
is not necessarily based on, "You give us this and we give
you that." We would like to see them much more as students
who are co-producers of their education and their education experience.
Q138 Stephen Williams: Professor
Arthur, the National Student Survey has 21 questions grouped around
different themes such as feedback, teaching and learning and personal
development. The outstanding statistic from the biggest grouping—assessment
and feedback—is that 40% of students are unhappy with what
they get, which must be the largest finding in that survey. That
must send shockwaves through the higher education world.
Professor Arthur: It did not send
shockwaves to the extent you might imagine. We already knew that
from subject reviews. This is always the area that causes the
greatest issue. It is very difficult to know precisely what is
going on, but I suppose there is an expectation when students
come from school of a very high level of feedback through the
assessment process. Compared with the national curriculum where
there is very regular feedback and a lot of help during feedback
to get through to the next stage, going to university is rather
different, so there is an adjustment to a different and much more
open learning scheme. This finding just repeated what has already
been found in the subject review. Having said that, one of the
benefits of the national survey is that it really has lifted out
that issue and shown how prominent it is, because in essence it
was the lowest scoring sector of the National Student Survey in
every single institution in the country. The prominence of the
finding is now very apparent. This has led to a number of us looking
at what we do in terms of assessment and feedback within our institutions
and all sorts of new ideas have cropped up. If I think of my own
institution, one school in particular has started a complete feedback
week; it is a kind of open door policy for all the students in
that school to come and get whatever feedback they feel they have
lacked in the past. The view of HEFCE is that probably quite a
lot more work needs to be done on it, and it has asked the Higher
Education Academy to work particularly with HEFCE and institutions
to try to improve assessment and feedback. I believe that if that
starts to happen up and down the land it will be a tremendous
outcome.
Q139 Stephen Williams: In terms of
how the questions are analysed, as I understand it the 21 questions
are given equal weighting when the report is collated, yet research
done by the University of Bristol, an excellent institution, found
that assessment and feedback in students' own minds was not terribly
important. Although it was the worst finding in the survey, in
terms of how students ranked its importance it did not feature
that high, but because all the questions have equal weighting
it distorts the results. Is that something which HEFCE will look
at as well?
Professor Arthur: I believe that
is a valid point. Any survey like this will be a balance between
the simplicity of the survey and, therefore, the willingness of
students to complete it—the survey takes between five and
six minutes to complete—versus the complexity and level
of detail that one might get back from the questions asked. We
thought it particularly important to keep the questions identical
for the first few cycles of the survey. These questions are jealously
guarded, and we are very reluctant to add further ones unless
we can be assured they will be of extra value, but over time the
questions will be refined. As to weighting different outcomes,
of course that would be possible. I think we would need to come
back to what the survey was originally intended for. It was originally
intended to inform prospective students about the quality of the
course they were thinking of taking, or the quality of the institution
that they were thinking of going to. Therefore, as the student
is looking and comparing, say, French at Leeds with French at
Manchester with French at Oxford, Cambridge or whatever, and pulling
up the data there is at least a comparison of like with like across
the various institutions that he or she is thinking of attending.
In terms of the original intention of the survey, I think this
is a reasonably valid methodology. By the way, what it was never
intended to do was create a league table of the quality of institutions.
If one wants to think of creating a league table one might wish
to start weighting many different factors, but that was never
the intention of the National Student Survey. It was intended
to inform students, and it has had the wonderful secondary effect
of creating greater enhancement of the learning, teaching and
other aspects of student experience.
24 Ev 37 Back
| 
