Written evidence from the Academy of Medical
Royal Colleges (REV 05)
The Academy attempts to develop and present
a consensus view of its constituent medical Royal Colleges and
Faculties.
The Academy and its constituent organisations
are charities, existing to maintain and improve the standard of
healthcare provided to patients. They are not Trades Unions and
do not take part in negotiations on terms and conditions of service.
The Academy and its constituent organisations
therefore speak from the perspective of doctors and with the expertise
of doctors, but with the intention of maximising the best interests
of patients.
SUMMARY
- The Academy supports the introduction of revalidation.
- The current proposals aim to make the annual
appraisal interview satisfy summative and formative functions.
We strongly support this aim. We believe that it is possible,
but not easy; there is a risk that a rigid, formal emphasis on
the summative elements might produce an inappropriate and inefficient
process.
- We wish to streamline the process. We suggest
that the GMC's insistence on checking against 12 "attributes"
of good practice at every appraisal is a source of unnecessary
complexity. We suggest a simpler alternative that is based on
the 12 "attributes", but which makes their involvement
less overt and mechanical (see Paragraph 12).
- It must be accepted that to assess an individual
doctor in the context of his/her form of practice will demand
some subjective judgements to be made. The need for good, consistent
judgement has implications in training, provision of advice, quality
assurance and other areas. We fear that these areas have not yet
been adequately considered.
- The proposed revalidation process results in
potential conflicts of interest. These need to be eliminated or
acknowledged and managed.
- Quality assurance is vital and must include quality
assurance of outcomes, not just quality assurance of process.
- We have concerns about the potential consequences
of enthusiastic reliance on relatively untested IT systems.
EVIDENCE FROM
THE ACADEMY
OF MEDICAL
ROYAL COLLEGES
1. The Academy supports the introduction of medical
revalidation. We have worked with all the relevant agencies in
an attempt to develop an acceptable and effective system of medical
revalidation and to satisfy the responsibilities placed on Medical
Royal Colleges in the White Paper, "Trust, assurance and
safety: The regulation of health professionals". We believe
that we are close to defining a workable system.
2. We do not wish this submission of evidence
to appear negative. Much has been achieved. However, there remain
tensions and unresolved questions in a number of areas and it
is appropriate for this submission to concentrate on these problems.
How to assess doctors?
3. After qualification, the practice of different
doctors diversifies enormously, to an extent that makes it impossible
to use formal examinations to assess competence. To attempt to
do so would be hugely expensive and for many specialist doctors
the result would be unfair and irrelevant to their practice. We
therefore support the plan to use an enhanced, structured form
of appraisal to deliver revalidation. Only in this way can revalidation
assess a doctor in the specific context of his/her own practice.
Unfortunately, such a system of individual assessment brings problems,
notably in respect of defining appropriate standards and ensuring
that every doctor is judged fairly against those standards.
Summative or formative?
4. It is inevitable in any profession that a
few practitioners will fail to maintain appropriate professional
standards. Informal discussions suggest that the identity of these
doctors is often known to their colleagues, but in the absence
of a regular objective evaluation it can be very difficult to
force such doctors to improve (or to cease practising). It is
therefore appropriate for revalidation to attempt to provide a
"summative" tool by which such doctors can be identified
with sufficient confidence for appropriate action to be taken.
5. But it is acknowledged that the large majority
of doctors practice to high standards. For them, a process designed
only to identify inadequate standards risks being
a waste of time and resources.
6. Revalidation need not waste resources. Even
good doctors often find it difficult to assess their own strengths
and weaknesses. It is therefore a legitimate goal of revalidation
to provide objective "formative" assessment of the strengths
and weaknesses of good doctors, to help them to improve still
further. For most doctors, confirmation that the minimum standard
has been achieved should be as swift as possible, so that resources
can be focused on further improvement.
7. The current proposals aim to make the annual
appraisal interview satisfy summative and formative functions.
This is probably possible and certainly desirable, but not easy.
It will demand highly skilled appraisers in large numbers. We
believe that whether or not it succeeds will need to be monitored.
How to make appraisal efficient and effective?
8. The proposals for revalidation centre on the
annual appraisal of each doctor. Through this process, each doctor
will be expected to demonstrate compliance with the 12 "Attributes
of good medical practice", as developed by the General Medical
Council (GMC) from its publication, "Good Medical Practice".
9. The Medical Royal Colleges were asked by the
GMC to set out how doctors in each specialty should satisfy this
requirement. We did as we were asked, but we believe that the
resultant specialist frameworks of evidence for revalidation,
as tested in the Pathfinder Pilots, have proved to be cumbersome
and inefficient. This view is supported by the results of the
GMC's recent consultation.
10. We suggest that the GMC's 12 attributes were
developed in the context of the GMC's experience in assessing
doctors with problems in performance. They are well suited to
the summative assessment. But for most doctors, the complexity
of the 12 attributes makes proving adequate performance too time-consuming.
Single items of information about a doctor's work can map variably
to multiple attributes; too much time is spent considering the
mapping, ticking boxes.
11. The GMC has defended its emphasis on the
12 Attributes of GMP with reference to the results of its recent
consultation. But that consultation offered no alternative, nor
did it ask how the Attributes should be integrated into revalidation.
It also showed an overwhelming demand for the system to be made
simpler and more proportionate. There is a conflict between these
two requirements. We believe that streamlining the system depends
upon making the involvement of the 12 Attributes less overt.
12. We therefore suggest that, having first defined
the doctor's area of practice, the annual appraisal should start
with the doctor undertaking three tasks:
(i) Provide and discuss evidence of the quality
of the care you deliver (including outcome measures, audit,
peer review, referral practice, formal proficiency tests as appropriate).
The nature of the evidence that should be submitted, and its evaluation,
will inevitably differ considerably between specialties.
(ii) Provide and discuss evidence of what
others think of your overall performance (including patient
feedback, multi-source feedback from colleagues, complaints, compliments
etc.). This element will vary less between specialties, although
some differences are inevitable—for example, some doctors
do not interact directly with patients.
(iii) Provide and discuss evidence of how
you keep your professional knowledge and practice up to date
(ie continuing professional development). The Colleges already
provide detailed guidance on CPD, but the appraisal process should
check that the CPD is appropriately tailored to the doctor's individual
needs.
Simple statements around probity and health may also
be demanded by the GMC to complete the coverage of the "12
attributes".
13. If the structure of such a portfolio of evidence
has been pre-defined so that the information presented covers
the GMC's 12 attributes then we do not see any reason also to
test each doctor mechanically at each appraisal against each "attribute"
in turn. In practice, to do so generates complexity and additional
work, without benefit. If the spread of evidence presented by
a doctor is incomplete or raises any cause for concern, the process
can pursue a summative verification of practice against the GMC's
12 attributes. If not, it should pursue a formative approach towards
further improvement.
14. It is our belief that if doctors are presented
with an objective, independent evaluation that questions the quality
of their practice, most will strive to improve their standards.
The proposed five-year revalidation system gives doctors time
to do this. As a result we hope that the need for formal remediation
processes (or the non-availability of a positive recommendation
for revalidation) will be relatively infrequent. If it is not,
then the quality of the revalidation system must be examined as
closely as the quality of the doctor's practice. Quality assurance
is discussed below.
DEFINING STANDARDS;
THE UNAVOIDABLE
NEED FOR
GOOD JUDGEMENT
15. The White Paper charged the Medical Royal
Colleges with setting standards for "recertification".
The subsequent merger of "relicensing" and "recertification"
into one process, "revalidation", has strong practical
arguments in its favour, but the result has been to diminish the
role of the Colleges.
16. The GMC defined generic standards on the
basis of its publication "Good medical practice" and
the associated 12 "Attributes of good medical practice".
The Medical Royal Colleges were then asked by the GMC to set out
how, in the context of each specialty, a doctor should produce
information or evidence to demonstrate compliance with each of
these 12 "Attributes".
17. This has not been achieved. The results so
far contain a few objective measurements, but most represent a
discussion of the forms of documentation and evidence that a doctor
might provide to satisfy the GMC's Attributes. They do not define
what the content should be. For example, it is easy to say that
doctors should undertake some form of audit of their practice
each year. It is much harder to define in advance what should
be audited and what represents an acceptable result from such
audit.
18. This is in part an inevitable consequence
of the huge diversity of medical practice. Revalidation not only
has to cover GPs and hospital consultants with extremely specialist
and overlapping areas of practice. It must cover forensic pathologists,
public health physicians, research doctors in pharmaceutical industry,
expert witnesses, health tribunal doctors, medical managers and
educators of many varieties, and so on.
19. The diversity of medical practice makes it
impossible to use formal postgraduate examinations as a revalidation
tool, as discussed above. But it also makes it inevitable that
the evidence presented, being assessed against the individual
doctor's area of work, can only be assessed by the application
of good judgement, not mechanical measurement. This cannot (and
must not) be reduced to a tick-box exercise.
20. The need for good judgement as part of this
process is problematic, because it raises the possibility of poor
or inconsistent judgement. We believe that this has been inadequately
discussed. It has been widely assumed that medical revalidation
can be delivered on the basis of objective measurements. We do
not believe this to be the case.
COVERING UNUSUAL
AND COMPLEX
FORMS OF
MEDICAL WORK
21. Revalidation has been developed and piloted
almost entirely in the context of NHS practice. We remain concerned
that doctors in unusual types of medical practice will find it
difficult to comply with the "12 attributes" approach
of the GMC, and we commend the simplified approach set out above
as a way to resolve this.
22. Some doctors undertake several radically
different forms of work. It is self-evident that they must maintain
their standards in all of them, but it is not yet clear how this
will be confirmed. It is a matter of concern that the process
could be extremely onerous.
23. Some doctors work in several different institutions
or environments; sometimes undertaking similar work in each, sometimes
radically different types of work in each. It is self-evident
that information from all these workplaces should be available
at the appraisal process, but it is as yet not entirely clear
how this will be achieved. There are concerns that commercial
confidentiality and other barriers will inhibit the flow of the
necessary information.
DELIVERING GOOD
AND CONSISTENT
JUDGEMENT
24. We suggest that the delivery of good judgement
of the quality of practice is possible only if the appraiser understands
the appraisee's work, so wherever possible both should be from
the same or a similar area of practice.
25. We suggest that good judgement will require
appraisal training in the context of the specialty in question,
not merely generic training.
26. We suggest that if there is any hope that
the delivery of this process is to be consistent across the UK,
then this training will have to be developed and preferably delivered
on a national basis. The Medical Royal Colleges anticipate that
they will have a leading role in developing and delivering this
training, but we are concerned that an adequate source of funding
to support this work has not yet been identified.
27. Despite our best efforts, it is inevitable
that judgements will vary to some extent. This has a number of
consequences:
(i) Over the five year cycle, each appraisee
should be evaluated by a number of different appraisers.
(ii) Uncertainty and disagreement is inevitable,
so we should be prepared for it. An efficient mechanism should
be available by which appraiser and appraisee can obtain independent
advice. To ensure consistency, this too should be coordinated
at a national level. The Medical Royal Colleges expect to have
a role in delivering this advice, but to do it well will demand
resources and a source of funding has not yet been identified.
(iii) A quality assurance system is vital. This
is discussed below. It should assess the quality of judgement,
not just the quality of process, and it should start at the appraisal
interview, not at the recommendation of the Responsible Officer.
CONFLICTS OF
INTEREST
28. The inevitable need for good judgement, rather
than mechanical measurement, makes it particularly important to
avoid conflicts of interest in those who make the judgements.
29. A conflict of interest is generated by the
proposal that the Responsible Officer, who makes the final recommendation
on revalidation to the GMC, should normally be the Medical Director.
The Medical Director has responsibilities to deliver the targets
of the employing organisation. In some circumstances these could
differ from, or even conflict with, the ideals of good practice.
For example, a doctor whose standards are questionable might be
essential to delivering a service and difficult to replace; would
the Medical Director then be as stringent as with a doctor who
delivers adequate care, but too slowly to meet Trust targets?
30. There are of course many other potential
sources of conflict of interest, including personal and financial
sources.
31. There are differences of opinion within the
Academy as to how this problem should be addressed. Some Colleges
take the view that the Medical Director should not be the Responsible
Officer. Others take the view that the Medical Director is a reasonable
choice, as he/she is in a good position to understand problems
and deliver solutions; the problem of conflicts of interest should
then be managed by recognising the conflict and having transparent
processes to deal with it.
32. Whatever the outcome of this debate, it is
agreed by all that part of the solution will be to have a robust
quality assurance system, whereby poor decisions can be identified
and corrected. Responsible Officers and appraisers must themselves
be subject to revalidation; those who repeatedly make poor decisions
(in either direction) should be subject to the same processes
as doctors who make poor decisions in other areas of medical practice.
QUALITY ASSURANCE
33. The need for quality assurance of revalidation
has been widely accepted. However, action has been based largely
around quality assurance of process, such as the AQMAR tool developed
by the Revalidation Support Team. There has been comparatively
little work done on the quality assessment of outcomes. We believe
that this is at least as important as process, and arguably more
important, because of the need to monitor the quality of judgements
made, as discussed above.
34. The GMC has had some preliminary discussions
with us about how to achieve quality assurance of judgement, but
as yet with no firm decisions and no pilot testing.
35. To deliver independent audit of revalidation
decisions will be technically difficult and will have resource
implications. This must not be allowed to derail the process,
because it is essential if patients and doctors are to have confidence
in the system. We note with concern that the GMC consultation
report mentions only "a possible GMC programme
of sampling and auditing" (our emphasis).
36. We believe that quality assurance of judgement
should take several forms.
(i) Whenever a decision is made not to recommend
revalidation, but the subsequent GMC Fitness to Practise Panel
decides that revalidation is appropriate, there should be a detailed
external review of the revalidation process in the relevant institution.
(ii) Conversely, if a doctor who has achieved
revalidation is found by the GMC (by some other route) to be unfit
to practise, there should be a detailed external review of the
revalidation process in the relevant institution.
(iii) There should be ongoing selective independent
audit of the judgements made by responsible officers and, if possible,
appraisers.
37. To make an audit of outcomes more efficient,
it should be targeted on areas or institutions that are regarded
as "high risk". The GMC has developed preliminary proposals
as to how this might be achieved. These mechanisms will need to
be monitored and updated.
38. We believe that our Fellows and members of
the public will expect the Medical Royal Colleges to have a prominent
role in quality assurance of revalidation, especially in quality
assurance of outcomes.
MAKING IT
COST-EFFECTIVE
39. We anticipate that there will be disagreements
about the overall cost of introducing revalidation. To a large
extent this is because of a lack of clarity about what items should
be included in the cost. If all doctors were already participating
fully in all the various elements—appraisal, CPD, audit,
and so on—then the cost of tying these together into a revalidation
system should not be huge and will, we believe, represent acceptable
value for money. However, we are far from such full participation
in all these elements, so in practice the overall increase in
cost may be considerable.
40. However cost is measured, we have a duty
to make revalidation as efficient as possible, consistent with
its achieving its aims of protecting patients from bad doctors
and helping good doctors to get even better.
41. We should not accept that the model initially
implemented is necessarily the best, but should strive for continuous
improvement of efficiency. We should periodically review whether
the system is delivering value for money.
42. We believe that the pilots undertaken so
far have identified ways to improve efficiency and that ongoing
feedback and evaluation of the pilots will provide more. As a
result we welcome the Secretary of State's decision to delay implementation
by one year. It would have been unwise to implement revalidation
immediately the Pathfinder pilots have concluded, without time
to analyse the results
INFORMATION TECHNOLOGY
43. We are particularly concerned about IT support.
This was initially identified as an essential tool to make revalidation
more efficient, but (despite considerable expense) the e-tool
commissioned by the Revalidation Support Team for the Pathfinder
Pilots is proving to be cumbersome, unreliable and unpopular.
44. We understand that when revalidation "goes
live" there will be no "officially approved" system
of IT support and that commercial suppliers will be invited to
offer their products to individual organisations. This may well
be a good method to ensure the development of good solutions over
time, but it means that revalidation will "go live"
using relatively untested software.
45. This leads us to suggest that, at least in
the initial implementation, revalidation should be designed to
demand as little reliance on IT support as possible. For example,
systems that demand that all documentary evidence to be presented
at appraisal is scanned and uploaded to a central server risk
heavy resource utilisation at best, and could easily make revalidation
impossible for some doctors.
DECIDING WHEN
TO "GO
LIVE"
46. We agree with the proposals outlined by the
GMC. A date by which all doctors must be involved in revalidation
must be set and publicised well in advance.
November 2010
| 
