Written evidence submitted by the UK Association
for Heterodox Economics (PR 52)|
1. I am replying on behalf of the UK Association
for Heterodox Economics (AHE) to your request for submissions
on the operation and effectiveness of the peer review process.
We address our response specifically to the way this operates
in economics. This profession is in the public eye following errors
of judgement and theory brought to light by the current recession,
which lends particular importance to this aspect of the committee's
2. We urge the committee to study our proposals
and the evidence that supports them. The AHE speaks for a wide
range of economists, particularly those critical of, or dissenting
from, the views of the economics mainstream. Our membership includes
500 practising economists, including editors of a range of peer-reviewed
journals, and our regular annual conference attracts over 100
scholarly papers. The AHE is a forum for international collaboration
and exchange, so that, for example, our 2010 conference in Bordeaux
was hosted by our sister organisation, the French Association
for Pluralist Economics (FAPE/AFEP).
3. A wealth of experience, and a record of high-quality
research, informs our response. We provided input to the international
benchmark review of the Research Assessment Exercise (RAE), and
to the UK Quality Assurance Authority (QAA) consultation on the
revision of the Subject Benchmark Statement for Economics (SBSE).
The AHE designated Andy Denis to act as guest editor for a special
edition of the International Review of Economics Education
dedicated to pluralism in the teaching of economics in Higher
Education Institutions, at the invitation of the Economics Network
of the UK Higher Education Academy. AHE members conduct and publish
systematic scholarly research on the impact of the economics profession's
evaluation, publication, research and teaching procedures. This
submission draws on that research.
4. The recent economic crisis has confirmed widespread
public and professional concern that the process that produces
economic ideas and policies has proved defective. We believe these
concerns are legitimate. The peer review system as it operates
in economics is, we argue, a significant contributory factor to
failures which came to light in the recent recession.
5. Peer review constitutes one aspect of a wider
process that has produced what Colander et al (2009) term
a "Systemic Failure" of the economics profession. The
AHE argues (Freeman 2010) that this arises from the "Regulatory
Capture" of the economics profession, a view we share with
Lord Turner, chair of the Financial Services Authority. This phenomenon,
well-known in economics, occurs when an institution plays the
role of regulating, that is to say, forming judgements over what
is fit for public purpose, and when a particular, private interest,
affected by the outcome of these judgements, arranges systematically
to influence these judgements so as to favour itself as against
other private interests and thereby, ultimately, against the public
6. Regulatory capture is systemic: no "lever"
may be identified that can be pulled to create or dismantle it.
The peer review system is however a fulcrum: it determines what
the profession "judges to be" good economics. The metric
of last resort when deciding if research funding should be allocated,
appointments made, or policy recommendations adopted, is publication
in approved journals.
7. Economics plays a specific role in society;
economic judgements are used by decision-makers to determine which
policies are valid and which are not, whether in government or
in other institutions with a public role such as banks, trade
bodies, regulators of public services, or indeed, credit rating
agencies. The economics profession thus plays a de facto regulatory
role, whilst enjoying almost total freedom from external scrutiny
or even self-regulation. Yet its judgements are frequently unsound,
as is now recognised in the wake of the 2008 financial crash and
8. The process that renders economics vulnerable
to capture was termed "groupthink" by the International
Monetary Fund (IMF)'s Independent Evaluation Office (IEO 2011),
which identified groupthink as a principal cause of serious errors
of judgement. It stated that: "The IMF's ability to correctly
identify the mounting risks was hindered by a high degree of groupthink,
intellectual capture, a general mindset that a major financial
crisis in large advanced economies was unlikely, and incomplete
9. A prominent outcome of groupthink is the so-called
"Efficient Markets Hypothesis" (EMH) - until recently
considered the hallmark of good judgement in economics, but which
led to the false conclusion that there could be no systemic failure
of the financial markets. This doctrine is now rightly discredited.
The problem to be addressed is, however, why the economics profession
discarded this unproven hypothesis after, not before, the crisis,
and why research that dissented from it - now widely accepted
as valid - was equally widely dismissed within the profession
until a year into the crisis. This oversight led to much harm.
Public policy should consider what guarantees the public may be
given against further oversights of this scope and nature.
10. The peer review system in economics contributes
to groupthink in a number of ways, which re-enforce each other.
The effective origin is the imposition of a hierarchy of acceptability
as an outcome of the peer review system. Economics journals are
informally or formally stratified into so-called "Diamond
List" or "Elite" journals defined as those which
secure the greatest degree of peer approbation, at the highest
levels of the profession.
11. Ranking by acceptability narrows the field
of ideas deemed legitimate to a narrow band of received ideas
that constitute themselves as a set of doctrines, stifling critical
judgment and denying the public access to alternative ideas -
notably, ideas that could have foreseen, prepared for, ameliorated
or even helped avert the recent crisis.
12. The peer review system interacts perniciously
with the allocation of research funds (Lee 2007, Lee and Harvey
1998, Gillies 2011), intensifying the tendency to groupthink.
Our research shows that UK Higher Education Institutions responded
to the RAE, and its successor, the Research Evaluation Framework
(REF), by focussing recruitment and research priorities on scholars
publishing in journals that conform to the mainstream and form
part of the "Diamond List". This stifles research which
might draw conclusions counter to those published in the mainstream
13. Higher Education Institutions react logically
and instrumentally to RAE and REF processes in an environment
where publication is the main metric of research excellence. They
determine both research priorities and recruitment or promotion
decisions on the basis of current or expected publication record
in mainstream journals. This practice is not always officially
acknowledged and on occasions denied. However, our studies show
it is very widely practiced. In consequence, University departments
of economics have been stripped of critical, dissenting or even
14. This creates a vicious circle. The scholars
thus selected have been chosen for conformity with current received
doctrine, and make up the "peer group" which judges
what is to be published, with the result that academic economics
is being steadily reduced to a body of people that conform with
a narrow range of dominant - and, as it has turned out, frequently
wrong - theory.
15. Applied and theoretical research deploying
other ways of interpreting economic behaviour is increasingly
confined to departments outside of official economics, for example
in Business Schools and Departments of International Development;
even there, it is under pressure from orthodoxy and, of course,
scope for the publication of dissenting or critical work in economics
journals is just as restricted.
16. Despite a highly influential public role,
economics is uniquely free from external scrutiny and uniquely
oblivious of the need for a public standard, code of conduct,
or indeed, ethical principles. Economics alone amongst the professions,
particularly those with a major influence on public policy affecting
large numbers of people, recognises no process of verification
beyond peer reviewed publication, other than paid appointment
or simple receipt of fees. Its output is not scrutinised for functionality,
by applying the simple test of whether they work, as is the output
of Engineers or Architects. Nor does any professional body enforce,
monitor or even attempt to define good practice, as in Medicine,
Accountancy, or the Legal Profession. Finally, there is no code
of conduct: the profession has evolved no clearly-stated criteria
as to what constitutes good or bad economics, and does not see
the need for them.
17. A linked cause for concern is the disproportionate
impact of private or external funding from bodies with a significant
interest in the outcome of economic research. It is particularly
striking that, unlike medical publications, economics editors
do not request information on sources of funding, do not publish
them, and take no precautions to offset the risk that accompany
conflicts of interest.
18. The recent embarrassing resignation of the
director of the London School of Economics draws attention to
the scale of the potential risk. Substantial concerns have been
voiced about the Icelandic Chamber of Commerce's procurement,
for a considerable fee, of a clean bill of health for Iceland's
financial institutions from the then Secretary of the Royal Economic
Society, months before these institutions collapsed. We believe
it an urgent matter to incorporate procedures into the peer review
system to protect against, and to publicly declare, conflict of
interest - in line with publication standards in other disciplines.
19. The AHE believes that a review of all practices
in the economics profession is an urgent matter. There is no more
important place to start than the process which passes judgement,
and in the absence of all external or independent scrutiny, thereby
defines, what constitutes "good economics" - the peer
review process for economics journals. The restriction of economic
doctrine to a single paradigm, or a best a small range of paradigms,
and to a narrow range of ideas and theories, is in our view the
principal problem to address in a much-needed reform of this process.
20. The conclusion that the AHE draws from its
research, and the experience of its members, is that the solution
is an approach to economic theory which we term "pluralism".
This respected approach is normal in the natural sciences but
foreign to economics, despite a declaration in support of it published
in the American Economic Review in 1992, signed by 44 prominent
economists including four Rijksbank Nobel Prize winners. Pluralism
informs us that the duty of the economist is not simply to study
or present a single answer, certified correct, to those that she
or he advises, but to make available the conclusions which flow
from the range of legitimate available approaches germane to the
problem at hand, to make clear the assumptions upon which each
depends, and to devolve to the decision-maker the ultimate right
and duty to choose between these conclusions.
21. We have also concluded that pluralism should
be assertive; it should be the subject of specific, explicit guidelines,
supported by mechanisms of incentive, scrutiny, and where appropriate,
sanction. In the absence of such prescription, the natural tendency
of economics, arising from its insertion into modern society,
is to revert to the practices which give rise to groupthink.
22. In line with this approach, we recommend
an assertive pluralist reform of the peer review system in economics
with the following objectives:
22.1 to protect against the narrowing of received
economic truth to a single trend or small range of trends of thought,
to the exclusion of legitimate critical and dissenting views.
The public needs specific protection against this distinctive
feature of economics not seen (Freeman 2009) in other academic
22.2 to provide due place for critical and dissenting
judgements, without discriminating so as to define them as de
facto inferior by virtue of the sole fact that the theories
involved, and their conclusions, are either unpopular with their
opponents, or are not widely subscribed to. Specific, prescriptive
measures are required to discourage the narrow selectivity to
which economics is peculiarly prone.
22.3 to guard against the systematic confusion
of quality with conformity. Economics, unusually for science,
exhibits a marked tendency to reject scholarship that does not
conform to current orthodoxy, no matter how well argued, theoretically
coherent, and well-supported by evidence. To guard against the
groupthink this produces, the economics peer review process should
be required to exhibit transparency, objectivity and to pay due
attention to external criteria of judgement that do not arise
from within the profession.
22.4 to take prudent precautions against the
self-referential use of rankings to determine funding allocations
for research whose output is measured by publication in peer-reviewed
journals on the basis of self-declared hierarchies of merit unsupported
by externally-defined criteria and subject to no external scrutiny.
22.5 to take prudent precautions against distortion
arising from conflicts of interest.
23. To this end we propose a number of changes
to the peer review system as it operates in economics:
23.1 A standard of acceptability in the peer
review procedures of journals is required, covering journals in
which publication is taken as a criterion for allocating UK research
funds. The simplest, though not the only way to achieve this,
would be to set up an independent body to draw up standards and
oversee their implementation in consultation with appropriate
public and private bodies representing both the profession, and
those affected by the quality of its output. The aim would not
be to override editorial academic freedom but to provide an independent
standard of good practice in economics publishing, above all to
inform the allocation of funds.
23.2 Attention should be paid to the following:
23.2.1 There should be published guidelines on
acceptable peer review procedures;
23.2.2 Account should be taken of the standard
wherever and whenever reference is made to publication in economics
journals as a metric of research output and quality, notably in
allocating research funding but also in recruitment and promotion.
23.2.3 The standard should include a code of
publication ethics in economics, drawing on the ethical standards
set by professions such as the Medical and Legal professions,
and drawing also on the guidelines offered by the Press Commission
for published magazines, particularly concerning accuracy, fair
representation, and right of reply.
23.2.4 Clear, transparent, and public criteria
are required for the selection of articles for publication. The
aim of such criteria is to provide for objectivity, in the specific
sense that an independent observer can verify whether the criteria
have been applied.
23.2.5 A common core of the criteria should repose
on a code of publication ethics common to the profession, for
example as specified in 23.2.3 above.
23.2.6 Where a contested range of theory or opinion
is relevant to the subject matter of either an individual article,
or a field of research or publication, journal readers need access
to the range of pertinent views involved.
23.2.7 Published articles should be judged, among
other criteria, on whether they take due account of, and recognise
the existence of, alternate, dissenting, or critical views.
23.3 Journals should therefore recognise an editorial
duty to take due account of legitimate views, meeting normal general
academic standards, that dissent from those that they publish,
or for which they exhibit a preference unjustified by explicit
23.3.1 The standard should require journals to
protect against misrepresentation. This includes the right of
reply, and a duty to publish corrections or retractions in the
event that a published article is subsequently proven to contain
significant misrepresentation. This is particularly important,
since the preservation of an orthodox doctrine and the non-recognition
of dissent is, in the main, rendered possible by the systematic
misrepresentation or omission of opposing views.
23.3.2 In cases where paradigmatic conflicts
are known to exist, the standard should require that review panels
are balanced and include reviewers who work within the paradigm
adopted by the author. There should be no presumption that a reviewer
hostile to an author's paradigm can make sound judgements on quality
of the author's submission.
23.3.3 The standard should require that journals
seek out and publish views that significantly dissent from or
criticise views that are heavily represented in their selection
of articles, where a body of dissent of high academic quality
23.3.4 The standard should require a right of
appeal against improper application of selection criteria or non-conformance
For and on behalf of the UK Association
for Heterodox Economics
10 March 2011
Colander, D, Föllmer H, Haas, A, Goldberg, M,
Juselius, K, Kirman, A, Lux, T and Sloth, B (2009). "The
Financial Crisis and the Systemic Failure of Academic Economics",
Kiel working paper 1489, Kiel Institute for the World Economy.
Denis, A (2009) (ed). International Review of
Economics Education, "Pluralism in Economics Education",
Volume 8, number 2. London: Economics Network.
Freeman, A (2009). "The Economists of Tomorrow:
the Case for a Pluralist Subject Benchmark Statement for Economics".
in Denis, A (2009) (ed). International Review of Economics
Education, "Pluralism in Economics Education",
Volume 8, number 2. London: Economics Network.
Freeman, A (2010).
"The Economists of Tomorrow," American
Journal of Economics and Sociology,
Blackwell Publishing, vol. 69(5), pages 1591-1613, November
Gillies, D (2010). Economics and Research Assessment
Systems. Working Paper, University College London.
Hodgson, G, U Mäki and D McCLoskey (originators)
(1992). "A Plea for a Pluralistic and Rigorous Economics".
The American Economic Review, 82(2).
IEO (2011). "IMF Performance in the Run-Up to
the Financial and Economic Crisis: IMF Surveillance in 2004-07".
January 10, 2011.
Lee, F (2007). "The Research Assessment Exercise,
the state and the dominance of mainstream economics in British
universities", Cambridge Journal of Economics, vol.
31(2), pp. 309-25.
Lee, F and Harley, S (1998). "Peer Review,The
Research Assessment Exercise and the Demise of Non-Mainstream
Economics". Capital and Class, (66), pp. 23-53.
The Association for Heterodox Economics is self-funded.
It receives occasional donations from other heterodox or pluralist
organisations including grants of the order of GBP2000 in 2008,
2009 and 2010 from CPEST, the Cambridge Political Economy Society.
Its activities are conducted by volunteer labour and it has no
paid officials or employees.