Peer review in scientific publications - Science and Technology Committee Contents

Written evidence submitted by Faculty of 1000 (PR 94)


1. Faculty of 1000 Ltd ( is a post-publication peer review service, which was launched to cover biology in 2002, and then added medicine in 2006. Since then, it has grown to include a Faculty of 10,000 leading researchers and clinicians who have contributed over 100,000 evaluations of 82,000 articles.

2. This submission does not cover existing methods of pre-publication peer review or the issues relating to them, as this has already been extensively covered by previous submissions from our publishing colleagues. Our submission will therefore focus on our experiences in developing a different method of peer review, post-publication peer review, as well as discuss some of our future plans and how they affect the peer review system.


3. We define F1000 as a post-publication peer review (PPPR) service because it evaluates already published research. Our Faculties of 10,000 experts across biology and medicine are asked to highlight those publications that they believe to be particularly important, irrespective of where they are published (the majority of our evaluations - 86% - are not from what are often thought of as the top-tier journals, e.g. Nature, Science, Cell, NEJM, JAMA, Lancet, BMJ). Faculty Members are asked to provide a rating (recommended; must read; or exceptional) and then provide a short commentary ("evaluation") on why they believe the article to be so interesting and how it might impact their own research or specialty, and their names are listed against this. These evaluations are effectively short open referee reports and the service acts as a positive filtering service.

4. Multiple Faculty Members can evaluate the same article, providing a combined higher rating, or can write a dissent if they disagree with an existing evaluation. The authors of the article can write a comment in response to the evaluation, and registered users can also write comments.

5. F1000 has also recently launched an open access repository of posters and slide presentations (F1000 Posters). These posters and presentations have been previously displayed at national and international conferences and have therefore already undergone varying levels of review by the conference organisers at the abstract submission stage. Following deposition of these documents into F1000 Posters, F1000's Faculty Members then review them, and again highlight those they believe to be particularly significant.


6. F1000 is in the process of launching F1000 Research, which will provide a novel way of publishing new research findings and will also use PPPR, but in a different way to that discussed above.

2.1  Submission

7. Authors will be invited to submit new research finding reports, short papers and full papers to F1000 Research for publication and refereeing. All submissions will be visible immediately on submission, much like a pre-print service. However, it will be made clear that the submission is awaiting refereeing.

2.2  Refereeing

8. The refereeing process will take the form of two separate parts.

9. Step 1 - Quick refereeing. Referees will be suggested by the authors largely from the F1000 Faculty (which will naturally expand significantly as a result). Referees will be asked to check that the research "seems reasonable", i.e. that the work is well constructed, clear and not misleading, and that the authors are providing adequate analysis and sensible conclusions. In fact, as many previous submissions to this enquiry have discussed, it is almost impossible to ask a referee to do anything beyond that, i.e. to confirm that the research "is reasonable" without asking them to recreate the experiments themselves. This quick refereeing process therefore provides a "best value" solution in terms of speed and effort.

10. Importantly, the process will be completely open, meaning all referee names will be openly listed with their affiliations, and all their comments made on the paper will be published.

11. Step 2 - Broader commenting. An ongoing relatively open process can then take place where other Faculty referees and registered F1000 users (active scientists) can attach comments, suggestions and questions. Over time, other researchers may comment on the work and these comments may change as perceptions about the work change, particularly as the true significance (or error) of the work may not be recognised immediately. This also enables the inclusion of a "have repeated this work" review, so that when a researcher successfully repeats the experiment described (or indeed is unable to repeat it), this information can be provided to others as a very powerful PPPR tool.

12. A combination of these two steps will help authors weed out problems and improve on their original submission.

2.3  Revising

13. Authors can submit amended versions of the submission at any time, taking into account the comments made during the ongoing refereeing process, with all previous versions of the submission archived but still accessible.

2.4  Ongoing "threaded publications"

14. As researchers move forward with their research topics, they can then submit papers on follow-up research findings, which may be a continuation of the previous submitted work. This results in a threaded set of publications as research develops, rather than separate papers in different publications that we have today.


15. The major advantages are:

—  Immediate access to the latest research findings

—  Much less heavy workload in the peer review process - only one set of reviews for any one paper (rather than repeated reviewing by different journals as the paper goes down the journal chain), and shorter and simpler requirements for refereeing that more accurately reflect what is possible to achieve through peer review.

16. By operating a completely open process, referees must take responsibility for how they judge the submission. This may also reduce poor-quality submissions from scientists who hope that the refereeing system will sort the work out for them.

17. Further, the ability to deposit research findings in smaller increments - in effect to "plant the flag" on particular topics and methodologies - will encourage earlier conversation regarding the means of inquiry and unfolding results. In addition, data that are deposited at the prepublication stage can immediately be mined for alternative purposes and therein tested via the most rigorous application of peer review.

By Rebecca Lawrence, Director New Product Development, F1000

On behalf of Faculty of 1000 Ltd


Rebecca Lawrence is an employee of Faculty of 1000 Ltd

May 2011

previous page contents next page

© Parliamentary copyright 2011
Prepared 28 July 2011