Peer Review

Written evidence submitted by Veli Albert Kallio (PR 93)

Vice-President, Environmental Affairs – Sea Research Society

Chairman – Frozen Isthmuses’ Protection Campaign of the Arctic and North Atlantic Oceans

President Emeritus – Astronomical Association Mikkelin Ursa r.y. (est. 1923)

Fellow – Royal Geographical Society

Fellow – The Explorers Club

Newton's Principia, bearing the imprimatur of Samuel Pepys, then-President of the Royal Society

English laws of 1586, 1637 and 1662 required an official licence for all book printing. The 1662 act required that, according to their subject, books needed to receive the authorization, known as the imprimatur, of the Lord Chancellor, the Earl Marshall, a principal Secretary of State, the Archbishop of Canterbury, or the Bishop of London. This law was finally abolished in 1695. [1]

What is a good ‘Peer Review’ process ?

"The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability - not the validity - of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong." [2 ]

Richard Horton, editor of The Lancet

"The Brief History of Peer Review "

Peer reviews became commonplace in most sciences only after the Second World War.

For example, Albert Einstein's revolutionary papers in 1905, Annalen der Physik, were never peer-reviewed by anyone else than the journal's editor and co-editor: "in journals in those days, the burden of proof was generally on the opponents rather than the proponents of new ideas." [3 ]

The above editorial statement from Nature suggests that the peer reviews usually side with the proponents of the old ideas and it is the responsibility for the new ideas to prove themselves right.

From the success of Albert Einstein we can certainly conclude that science thrived a long before the peer review; I highlight a selection of examples where "peer review" fails to act as a scientific quality control.

In its broadest sense the ‘peer review’ can be defined as a process of self-regulation by a profession - or an evaluation process by qualified individuals within any relevant field. It aims to (1.) maintain standards, (2.) improve performance and (3.) provide credibility.

While the term ‘peer’ refers to one "of equal standing with another … especially belonging to the same societal group…or [having the same] status" [4 ] the term ‘review’ refers to a critical inspection or examination, or, a repeat viewing of past events, circumstances or facts. [5 ]

Influence of Underlying Scientific Subculture s in the Peer Reviews

The existences of scientific subcultures are far too often ignored as an inherent part influencing every peer review.

The research subcultures must, therefore, be acknowledged as a major potential hindrance in objectivity of peer review processes especially when interdisciplinary research is carried out and different research traditions are being practised on the same matter.

It is also said that peer review processes are not genuine outsider audits of the scientists’ research efforts but mere "cosy comments" made by the other like-minded insiders. This flaw results in outstanding scientific failures while the review processes and scientists are "both deaf and intolerant" to the outsiders’ input who have acquired expertise in the same matters. This concern has recently been voiced in peer reviews of: (I) pharmaceutical and (II) climate sciences.

There is a sustained, prevailing failure to recognise the existence of a substantial amount of group-think and pre-conceived world-views that always permeate all peer review processes within a closed group of scientists. As a result, the scientific peer review community often appears insular with a highly inflated self-esteem about their procedures. The scientists often appear highly critical, combatant and dismissive about the audit procedures that are deployed by the humanistic or commercial researchers and remain very cold to their occasional criticisms.

The Crisis No. 1: The Pharmaceutical and Medical Peer Reviews

" The Commons Science and Technology Committee said there is no evidence the drugs are any more effective than a placebo - the same as taking a sugar or dummy pill and believing it works. Last month, doctors attending the British Medical Association (BMA) annual conference backed this view, saying homeopathic remedies should be banned on the NHS and taken off pharmacy shelves where they are sold as medicines. The treatment was described as "nonsense on stilts" and that patients would be better off buying bottled water. "

Former Liberal Democrat MP Evan Harris, who was a member of the Science and Technology Committee when it published its report:

"This is not a good start for the new Health Secretary when it comes to evidence-based policy. How does the Government justify allowing treatments that do not work to be provided by the NHS in the name of choice, when it allows medicines which do work to be banned from NHS use?"

The UK’s Government already issued its own verdict on this matter in March 2011 but this issue still deserves here a further look from a perspective of the scientific peer reviews flaws.

In March 2011 a European Union wide attempt was made to prevent the manufacturers from testing new Herbal and Homeopathic medicines in Europe. This law was proposed by the pharmaceutical scientists that included many leading peer reviewers of the pharmaceutical papers. The UK’s Government turned down this law proposal. Why the Homeopaths had defeated Goliath?

I do not give support to the efficacy of the Homeopathic preparations and I fully side with the committee that these preparations are far too often over-diluted for any real medical benefits.

The penances were paid over a little-noticed failure by the mainline science as the Health Secretary, Rt. Hon. Anne Milton, MP, issued an extension for the Homeopathic medications to be funded by the National Health Services at the annual cost of £4 million. What was the justifying "penances" the Homeopathic medication developers could offer to persuade to withdraw support from the NHS?

The Thalidomide Incident

My former sister in law’s brother is a living testament to a scientific-pharmaceutical monstrosity and a witness of the tragic consequences of a serious scientific group-think and shortcoming of inadequate scientific peer review.

Back in 1950’s thalidomide was released to the market as a relief for the pregnant women’s morning sickness after a large statistical animal experiment did not reveal any serious side effects in the test animals from the chemical this pharmaceutical product contained.

At the time of Thalidomide's development the popular theory was that it was unlikely that any drug could pass from the mother across the placental barrier and harm the developing foetus . [6 ]

The Pharmaceuticals " Never -Never Again " Mantra

Since 1960’s, the pharmaceutical peer reviewers were busy in repeating their mantra countless times that the thalidomide scandal was a "classic case", no longer possible due to "fixed" review procedures. Yet, UK’s Homeopathic medicine developers continued to warn about one extremely serious procedural safety flaw they perceived in the mainline pharmaceutical peer review guidelines.

The-post Thalidomide review regime, after a major public fury and disgust, brought in a far more vigilant pharmaceutical peer review regimes that involved extensive experimenting with humans. But even the changes did not address the primary testing flaw which UK’s Homeopathic medications developers summarise here for the Parliamentary Select Committee for Science and Technology:

When each new Homeopathic and Herbal preparation is developed to the UK market (often from highly toxic substances), the very first Homeopathic doses start tests at nil concentrations. The dose is then very gradually increased at extremely low dilutions of the active ingredient. The Homeopathic test approach has never in its history produced an acute emergency situation for the testers or participants. The new Homeopathic solutions are first introduced to testing at levels 10,000 times more diluted than where the botulinum gives its first signs of toxicity.

The testing method of Homeopathic and Herbal medicine developers has been proven 100% safe.

The Monoclonal Antibody TGN1412 Incident

Quite contrary, the peer reviewers and scientists of TeGenero AG’s testing a monoclonal antibody TGN1412, made world’s Homeopathic medicine developers laugh at the futility of Parexel’s testing procedures: a hoped-for anti-inflammatory drug led to a major human disaster when their six volunteers started to scream in extreme pain with their heads rapidly swelling, vomiting and having breathing difficulties, convulsions, and an excruciating pain. Soon, the participants were put into an induced coma to relieve their tormenting pains. The participants came to be known as "The Six Elephant Men" after their heads swelled up to three times to their natural sizes.

TGN1412 came to symbolise the allopath’s folly of the top-to-down (theory-based) scientific "safety procedures". Hilariously to the Homeopaths and Herbalists, the conduct of pre- and post-disaster reviews of TGN1412 testing accident was deemed having been correctly done to the books: the auditors iterated that the peer-reviewed safety procedures had been "well implemented" and that TeGenero AG and Parexel both fully complied with the "popular theory" about safe level of TGN1412 anti-inflammatory agent. The initial doze was set well within the (speculated) safe testing range. TeGenero AG and Parexel became the latest laughing stock for the UK’s Homeopathic medicine developers as the Homeopathic testers have never seen any unexpected, acute effects in their testing.

UK’s Homeopathic/Herbal medications developers ask for the Parliamentary Select Committee for Science and Technology to consider using their time-proven method to safely test medicines at "nil concentration" as the starting point when one deals with a previously untested chemical substance.

UK’s current approach will only cause more incidents like Thalidomide and TGN1412 in the future !

Introduction to Scientific Subculture s and Peer Review

I selected the contentions case of the UK government / Health Secretary Rt. Hon. Anne Milton, MP who back the Homeopathy against the Parliamentary Select Committee for Science and Technology / medical professionals as an example how an entirely functional idea may exist but be confined strictly into its characteristic quantum state in a segregated quantum well.

I have no interest in dispute among the Homeopaths, Herbalists and medical profession other than reviewing the practises of each, I would only volunteer medicine test organised by a Homeopath.

Although, it is easy to argue that the Homeopathic care is inferior to the Allopathic (conventional medical) tradition, few would disagree that these represent two different cultural subsets. Similarly, the peer-reviewers are said to have a tendency to have a far more positive self esteem of their own field’s methods while they all-too-often disregard research and review processes used by other research fields "as far inferior to their own practise". The segregation of research into various scientific and humanities ‘disciplines’ has erected numerous impregnable walls. As a result, these invisible barriers (prejudice), seriously stifle the free transmission and adoption of new ideas.

· The debate on peer review must acknowledge that two separate research fields should always be able speak to each other on level terms without any discrimination, nor tolerate stereotyping on basis of the varied research formats between the sciences and humanities, or even with commerce. The different forms become the primary obstruction much like a poor connection between a computer and printer that cannot communicate with each other due to their incompatible interfaces.

· A successful academic peer review prevents a dissemination of (1.) irrelevant findings, (2.) unwarranted claims, and (3.) unacceptable interpretations.

· An unsuccessful academic peer review hinders scientific processes if it (1.) stalls a dissemination of new research discoveries, or, (2.) if it amplifies a scientific subculture of prejudice and groupthink.

The biggest problem area in academic peer reviews is neither in maintaining standards, nor in providing credibility. These are usually well take care. The largest problems lie on how the peer review processes improve the performance of science and prepare it for the paradigm transitions.

Recognising the limitations of ‘peer review’

Peer review is essential to academic quality, and it is used by most scientific publications, except the generic and news type scientific journals such as: New Scientist, Science Week, Scientific American, National Geographic, Astronomy that dispense with the rigorous peer review for a fast delivery of newsworthy reports for sciences and technology. As a proper peer review process always takes time, it is often criticised as: (1.) ineffective, or, (2.) slow.

These are some of the main problems:

1. Peer reviewing is difficult when a broadly-defined subject needs to be attended involving many inter-disciplinary fields - such as scientific and humanistic research. Whose convention is to be followed when reviewing the joint research? Do any conflicting paradigms between the different academic disciplines emerge? Can these be reconciled, or should some inconsistencies be left unresolved for others to work on? Is too much being carpeted over in order to appear united to ‘fix’ a phenomenon?

2. Peer review does ignore the significance that an idea may never be widely appreciated among its contemporary scientists. This was the case of Nicolaus Copernicus and Galileo Galilei whose heliocentricity was broadly dismissed by their colleagues. The history of science continues to be littered with things taken granted when they are not. In such circumstances, a peer review is likely to be a hindrance to new paradigms: the 2005 Nobel Prize in Medicine went to two Australian medical practitioners, Dr Barry Marshall and Dr Robin Warren, who had proved only by drinking a pathogen that a bacteria was the disease vector for stomach ulcers against all research community.

3. Peer review requires a community of experts to perform impartial review. Can this be truly accomplished as an anonymous peer review - given the often limited pool of expert opinions available? Should the reviewers remain anonymous to the author (or each other) or be disclosed as in open peer review; i.e. the Shakespeare Quarterly. [8]

4. Peer review can find it impossibly hard to accept a certain degree of ‘scientific redundancy’ and ‘inherent error’ when a research must depend on grey literature for some information that is not available in a peer reviewed papers but yet has a critical bearing for the outcomes in the subject studied. A ‘peer review absolutism’ can lead to all pieces of information from a grey literature to be discarded that do not comply with the peer reviews’ criteria. The outcome of such a research has a bias: i.e. the world will see only 18 cm sea level rise by year 2100 if the peer-reviewed literature’s well-recorded melt water runoff and sea water expansion figures are used. But if the known, but little recorded other phenomena are included, the sea level rises far more, quite possibly up to 200-250 cm in this century.

5. The issue of redundancy in use of ‘grey’ and ‘peer reviewed’ literature came to the public limelight when the Himalayan glaciers that were thought to be melting away by year 2035 was found to be a printing error on 5th June 1999 issue of New Scientist magazine.

The date 2035 was circulating in various literature for years after the initial error in New Scientist which had transposed numbers misprinting year ‘2350’ as ‘2035’.

Typos (and wrong conclusions) do appear in a grey and peer reviewed literature and go undetected. Therefore, all major research publications should state the acceptable rate of source redundancy as a target figure rather than assum e that every piece of information is always totally ‘error free’. The disclosure s of the error margin s are need ed .

6. The peer reviews of cross-disciplinary and eclectic research subjects suffer from a fact that with respect to some manuscripts or research proposals, there may be just a handful of scholars who can truly qualify as experts. The interdisciplinary segregation in science or humanities research is complete if no one fully understands in dept two diverse but overlapping research fields that have become entangled with each other due to a novel approach or issue brings them together. In such circumstances, an anonymous peer review is all but impossible.

7. The peer review of eclectic research that newly fuses two distant research disciplines is the most demanding and contentious of all types of peer reviews: one discipline may face a strong resistance from another especially if it requires an alteration to its long-cherished paradigms, long seen as fundamental idea in the other discipline. A genuine interbreeding of work into another research discipline increases a probability that the weaknesses in the original major paradigms are identified, exposed and improved. Many Nobel prizes have come this way.

Most scholars reading the published articles can only be expert in a limited area. Therefore, there is a tendency of the scientific research to perpetuate the existing opinions as long as the noise from the dissenting anomalies does not become an overbearing distraction to dissipate a trust on the perceived old notion. The peer reviewers have their own ‘comfort zones’ that can lead to the problem of the warning signs being blatantly ignored.

Veli Albert Kallio

Sea Research Society, Vice-President Environmental Affairs

Royal Geographical Society, London, Fellow

Frozen Isthmuses' Protection Campaign of the Arctic and North Atlantic Oceans,
Chairman

28 March 2011

References:

1. ^ The Oxford Dictionary of the Christian Church (Oxford University Press 2005 ISBN 978-0-19-280290-3 ), article imprimatur

2. ^ Horton, Richard (2000). "Genetically modified food: consternation, confusion, and crack-up" . MJA 172 (4): 148–9. PMID   10772580 . http://www.mja.com.au/public/issues/172_04_210200/horton/horton.html

3. ^ "Coping with peer rejection". Nature 425 (6959): 6

5. ^ http://www.thefreedictionary.com/review

5a. "Homeopathy will not be banned by NHS despite cr45. 16 October 2003. doi : 10.1038/425645a . PMID   14562060

4. ^ "Peer – Definition" . Merriam-Webster Dictionary. http://www.merriam-webster.com/dictionary/peer

Homeopathy will continue to be available on the NHS despite an influential health committee condemning it as medically unproven." Health News, The Telegraph, (Monday 21 March 2011), http://www.telegraph.co.uk/health/healthnews/7910948/Homeopathy-will-not-be-banned-by-NHS-despite-critical-report.html

6. ^ a b Heaton, C. A. (1994). The Chemical Industry. Springer. pp. 40. ISBN   0751400181

7. ^ Rennie D, Flanagin A, Smith R, Smith J (March 19, 2003). "Fifth International Congress on Peer Review and Biomedical Publication: Call for Research" . JAMA 289 (11): 1438. doi : 10.1001/jama.289.11.1438 . http://jama.ama-assn.org/cgi/content/full/289/11/1438

8. ^ Cohen, Patricia (August 23, 2010). "For Scholars, Web Changes Sacred Rite of Peer Review" . The New York Times. http://www.nytimes.com/2010/08/24/arts/24peer.html?_r=1&ref=arts