Evidence-based policy
Use of existing evidence
105. The outcome might have been very different if
existing research had been utilised regarding both the approach
to e-learning (pure or blended) and the progress of comparable
ventures in other countries.
106. Dr Howells told us that there should have been
more testing of various models, approaches, and prototypes.
'I am a great believer in testing models. I think
we should be much more evidence based in terms of how we devise
policy and run organisations. We probably could have done with
more time in terms of looking around the world, perhaps not reacting
as quickly as we did to what we perceived to be great threats
coming mainly from America of our own students being captured
to do degrees by universities like Phoenix and so on.[67]
'I think we should pilot and test more often with
these kinds of things, so probably we could have been a little
bit more modest in our aims and also used some of the great expertise
that is out there
I can see that there was a real sense of
adventure at the time that this was a great new future and we
had to be in there right at the very beginning. It is easy for
me to say this with hindsight now.'[68]
107. Robust and reliable market research information
could have guided UKeU in a very different direction. Whilst it
was the case that UKeU inherited an ambitious project, this need
not have had the hold on subsequent strategic decisions that it
did have. The ambitious nature of the original idea for the project
alone cannot be blamed for the failure of UKeU. If UKeU had made
proper use of market research, the initial ambition for the project
would have held less importance.
108. The problem for UKeU was a combination of
the ambitious nature of the original idea, and an over-confidence
about the level of demand for e-learning which led to an approach
which was insufficiently focussed on research and marketing and
which was not learner-centred. To be successful, the project's
main focus should have been on clearly identifying its market
and knowing the demands of its customers.
109. The lesson to be learnt is that such high-risk
ventures entering new and emerging markets must have a focus on
front-line research. They need to have the flexibility to adapt
to changing market trends, and directors/managers must be able
to make strategic and operational decisions, but these decisions
must be evidence-based and rooted in robust and reliable research
information.
The Government's approach to
risk
110. It was HEFCE's responsibility to conduct a Risk
Assessment of the project. HEFCE's risk register for the UKeU
project was shared with the DfES and endorsed by the Minister
in 2001. We were told by Sir Howard Newby that '
the risks
which have come to fruition
were risks that were identified
in the original register'.[69]
It is not obvious what actions were taken to mitigate risks or
whether previously agreed actions were taken where risks did occur.
Was there an agreed action plan for the accounting officer in
light of the risk assessment; and if so were these actions taken?
111. There are lessons to be learnt about the Government's
attitude to risk. UKeU was approached not as an experiment or
pilot but as a fully fledged project and, as a result, its failure
has brought questions of accountability as well as of the lessons
to be learnt. We do not wish to make the Government too
risk-averse. In his evidence to the Committee Dr Howells raised
the issue of risk:
'I certainly think it was a very risky experiment.
I would not go so far as to call it a disaster because I am quite
interested in the lessons that we have learned. In terms of some
of the things that the Americans have tried and failed at, for
example, this is pretty modest stuff. There is a very different
mentality in this country, by the way, about risk and the consequences
of failure...Having said that, the Chairman put his finger on
it, this is not private money; this is taxpayers' money. The Chairman
also said very early on, 'we have also got to be careful that
we do not make the government risk-averse.' I draw from that the
lesson that we have to make sure that those structures are right,
that they are very clear and very open and that people can measure
those yardsticks.'[70]
112. We do not want the Government to become increasingly
risk-averse as a result of the UKeU experience. Instead it should
learn from this experience and, in the future, take a more experimental
approach to such high risk ventures. This would involve focussing
more on testing various models and prototypes; taking an evidence-based
approach; involving the private sector as partners in a more organic
process; undertaking effective risk-assessment procedures; and
setting open and transparent success criteria for such projects.
51 Q 74 Back
52
Q 83 Back
53
Q 23 Back
54
Q 421 Back
55
Q 59 Back
56
Ev 2, para 14 Back
57
Q 615 Back
58
Q 634 Back
59
Q 629 Back
60
Q 622 Back
61
Q 25 Back
62
Q 327 Back
63
Q 28 Back
64
Q 72 Back
65
Q 613 Back
66
Education and Skills Committee, Third Report, Session 2001-02,
Individual Learning Accounts, HC 561-I, and First Report,
Session 2004-05, Public Expenditure on Education and Skills,
HC 168. Back
67
Q 464 Back
68
Q 464 Back
69
Q 102 Back
70
Q 432 Back