Written evidence submitted by Faculty
of 1000 (PR 94)
BACKGROUND
1. Faculty of 1000 Ltd (http://f1000.com) is a post-publication
peer review service, which was launched to cover biology in 2002,
and then added medicine in 2006. Since then, it has grown to include
a Faculty of 10,000 leading researchers and clinicians who have
contributed over 100,000 evaluations of 82,000 articles.
2. This submission does not cover existing methods
of pre-publication peer review or the issues relating to them,
as this has already been extensively covered by previous submissions
from our publishing colleagues. Our submission will therefore
focus on our experiences in developing a different method of peer
review, post-publication peer review, as well as discuss some
of our future plans and how they affect the peer review system.
1 THE EXISTING
F1000 EVALUATION SYSTEM
3. We define F1000 as a post-publication peer review
(PPPR) service because it evaluates already published research.
Our Faculties of 10,000 experts across biology and medicine are
asked to highlight those publications that they believe to be
particularly important, irrespective of where they are published
(the majority of our evaluations - 86% - are not from what
are often thought of as the top-tier journals, e.g. Nature,
Science, Cell, NEJM, JAMA, Lancet,
BMJ). Faculty Members are asked to provide a rating (recommended;
must read; or exceptional) and then provide a short commentary
("evaluation") on why they believe the article to be
so interesting and how it might impact their own research or specialty,
and their names are listed against this. These evaluations are
effectively short open referee reports and the service acts as
a positive filtering service.
4. Multiple Faculty Members can evaluate the same
article, providing a combined higher rating, or can write a dissent
if they disagree with an existing evaluation. The authors of the
article can write a comment in response to the evaluation, and
registered users can also write comments.
5. F1000 has also recently launched an open access
repository of posters and slide presentations (F1000 Posters).
These posters and presentations have been previously displayed
at national and international conferences and have therefore already
undergone varying levels of review by the conference organisers
at the abstract submission stage. Following deposition of these
documents into F1000 Posters, F1000's Faculty Members then review
them, and again highlight those they believe to be particularly
significant.
2 NEW PEER
REVIEW INITIATIVES
AT F1000
6. F1000 is in the process of launching F1000 Research,
which will provide a novel way of publishing new research findings
and will also use PPPR, but in a different way to that discussed
above.
2.1 Submission
7. Authors will be invited to submit new research
finding reports, short papers and full papers to F1000 Research
for publication and refereeing. All submissions will be visible
immediately on submission, much like a pre-print service. However,
it will be made clear that the submission is awaiting refereeing.
2.2 Refereeing
8. The refereeing process will take the form of two
separate parts.
9. Step 1 - Quick refereeing. Referees
will be suggested by the authors largely from the F1000 Faculty
(which will naturally expand significantly as a result). Referees
will be asked to check that the research "seems
reasonable", i.e. that the work is well constructed, clear
and not misleading, and that the authors are providing adequate
analysis and sensible conclusions. In fact, as many previous submissions
to this enquiry have discussed, it is almost impossible to ask
a referee to do anything beyond that, i.e. to confirm that the
research "is reasonable" without asking
them to recreate the experiments themselves. This quick refereeing
process therefore provides a "best value" solution in
terms of speed and effort.
10. Importantly, the process will be completely open,
meaning all referee names will be openly listed with their affiliations,
and all their comments made on the paper will be published.
11. Step 2 - Broader commenting. An
ongoing relatively open process can then take place where other
Faculty referees and registered F1000 users (active scientists)
can attach comments, suggestions and questions. Over time, other
researchers may comment on the work and these comments may change
as perceptions about the work change, particularly as the true
significance (or error) of the work may not be recognised immediately.
This also enables the inclusion of a "have repeated this
work" review, so that when a researcher successfully repeats
the experiment described (or indeed is unable to repeat it), this
information can be provided to others as a very powerful PPPR
tool.
12. A combination of these two steps will help authors
weed out problems and improve on their original submission.
2.3 Revising
13. Authors can submit amended versions of the submission
at any time, taking into account the comments made during the
ongoing refereeing process, with all previous versions of the
submission archived but still accessible.
2.4 Ongoing "threaded publications"
14. As researchers move forward with their research
topics, they can then submit papers on follow-up research findings,
which may be a continuation of the previous submitted work. This
results in a threaded set of publications as research develops,
rather than separate papers in different publications that we
have today.
3 THE MAJOR
ADVANTAGES OF
EARLY DEPOSITION
OF FINDINGS
FOLLOWED BY
PPPR
15. The major advantages are:
— Immediate
access to the latest research findings
— Much
less heavy workload in the peer review process - only one set
of reviews for any one paper (rather than repeated reviewing by
different journals as the paper goes down the journal chain),
and shorter and simpler requirements for refereeing that more
accurately reflect what is possible to achieve through peer review.
16. By operating a completely open process, referees
must take responsibility for how they judge the submission. This
may also reduce poor-quality submissions from scientists who hope
that the refereeing system will sort the work out for them.
17. Further, the ability to deposit research findings
in smaller increments - in effect to "plant the flag"
on particular topics and methodologies - will encourage earlier
conversation regarding the means of inquiry and unfolding results.
In addition, data that are deposited at the prepublication stage
can immediately be mined for alternative purposes and therein
tested via the most rigorous application of peer review.
By Rebecca Lawrence, Director New Product Development,
F1000
On behalf of Faculty of 1000 Ltd
CONFLICT OF
INTEREST STATEMENT
Rebecca Lawrence is an employee of Faculty of 1000
Ltd
May 2011
| 
