Select Committee on Health Written Evidence

Memorandum by the Medical Decision Making Research Group, The University of Birmingham (PS 04)


  The Medical Decision Making Research Group is based in the School of Health and Population Sciences at the University of Birmingham. Dr Olga Kostopoulou is an applied cognitive psychologist with a particular interest in patient safety and diagnostic error. Professor Brendan Delaney is a general practitioner with interests in health informatics, technology evaluation and evidence-based practice.


  Since UK General Practitioners have a "gatekeeping" role controlling access to specialist services, diagnostic error and delay in primary care have far-reaching effects on patient experience throughout the NHS. Diagnostic error is the commonest cause of litigation against GPs in both the UK and USA, but has been poorly studied in comparison to other types of errors, eg prescribing and administrative, due to lack of awareness of its occurrence by practitioners and patients, and a reluctance to report individual diagnostic errors.

  Research in decision making indicates that in solving familiar problems, clinicians are able to perform accurately and quickly through matching of "patterns", "prototypes" or previous remembered instances. However, these "intuitive" strategies are likely to break down in more complex problems. Recent research commissioned by the Dept of Health Patient Safety Research Programme has demonstrated that asking appropriate diagnostic questions strongly predicts accuracy of diagnosis in difficult cases.

  Unfortunately, clinicians are not always able to recognise when a problem requires a more deliberate approach. Furthermore, any attempts to provide decision support systems have failed due to lack of integration with the clinician's workflow and the existing health record systems. Opportunities exist to improve and support diagnosis both in developing training tools and in providing more sophisticated computerised decision support. The electronic health record is now available in every consultation in UK General Practice. With advances in informatics, appropriate systems can link information about individual patients and knowledge about the diagnostic value of symptoms to provide diagnostic decision support. Support could be provided automatically and in the background, serving as a reminder, only when critical.

  We suggest that the Parliamentary Health Committee prioritise research in:

    1.  Training tools for diagnostic skill

    2.  Development and evaluation of better informatics tools to support GPs in diagnosis.


  1.1.  Prompt and accurate diagnosis in primary care is an essential part of the UK healthcare system. General Practitioners are the point of first contact with the health care system and act as the gateway to specialist care. Primary care is characterised by a wide range of potential diagnoses, relatively unstructured presentations and a low prevalence of serious morbidity. Although this may seem a benign environment for patient safety, the sheer volume of episodes, 90% of contacts in the UK healthcare system, mean that only very low risks can be tolerated. Data from both major UK medical defence organisations show that diagnostic error is the reason for most patient claims against GPs (63%-66%).[31],[32] It is also the commonest reason for malpractice claims in the ambulatory care setting in the USA (59%).[33] Conditions that have been associated with diagnostic error in patient claims, GP self-reports of memorable errors, and a recent literature review[34] are mainly cancers (particularly ovarian, breast, colorectal and bone cancers), coronary disease, and infections, eg meningitis. Nevertheless, diagnostic error remains under-researched due to the difficulties involved in identifying when a diagnostic error has been made and measuring its impact.

  1.2.  Identifying diagnostic error is difficult for a number of reasons:

  1.2.1.  GPs may not be aware of it due to lack of feedback. For example, patients may get better despite a wrong diagnosis, or go to a different doctor, or enter secondary care where their final diagnosis may not be fed back to the GP. Occasional, incomplete and delayed feedback has important, negative consequences for reflective practice, learning and improvement, and can perpetuate doctors' persistence with wrong beliefs and practices. Immediate feedback on a large number of simulated cases has been suggested as a way of improving experiential learning (learning in practice)[35] but evidence for this is scarce and it has only been implemented with medical students.[36] The effectiveness of this approach on clinical practice requires urgent investigation.

  1.2.2.  GPs may be reluctant to admit to a diagnostic error for fear of litigation and loss of patient and colleagues' trust. Self-reporting systems, developed either at a national level or for research purposes, receive a very small number of reports on diagnostic errors,[37] ,[38] ,[39] whilst most reports are about failures in systems or processes.[40] In contrast, when GPs are asked about the most serious errors in their career, they usually refer to past diagnostic errors, suggesting that diagnosis is central to the medical profession and that diagnostic errors have the most serious consequences.[41],[42] ,[43] Seriousness of consequences could in turn explain why diagnostic error accounts for most patient claims.

  1.2.3.  Patients may not be aware of a diagnostic error having occurred. In fact, there is very little overlap between what doctors consider an error and what patients consider an error. Patients in 10 family medicine clinics in the US were invited to report errors in their care; they made 126 reports, only 18 of which were about errors (none of them diagnostic).[44] In another US study, only 4 of the 53 medical errors reported by family physicians at in-depth interviews led to litigation, although in almost half of the errors, the patient died as a result.[45] These studies suggest that many medical errors go unnoticed by patients, while diagnostic errors with serious patient consequences often result in litigation—though it is not possible to estimate percentages, due to difficulties in measuring the actual rate of diagnostic error.


  2.1.  Diagnostic error can be the result of factors in the healthcare system and of clinical judgment (what we refer to as "cognitive factors"). Cognitive factors seem to be the most prevalent cause of diagnostic error.[46] A US study of closed malpractice claims (patients alleging missed or delayed diagnosis) in the ambulatory setting estimated that cognitive factors (eg judgment errors, vigilance and memory lapses, lack of knowledge) were implicated in virtually all diagnostic errors, either alone (in 55% of errors) or in association with patient- and/or system-related factors.[47] The most frequent breakdowns in the diagnostic process were failure to order appropriate diagnostic tests (55%), failure to follow up appropriately (45%), inadequate history taking and physical examination (42%), and incorrect interpretation of diagnostic tests (37%), mostly imaging. It is apparent that failure to gather sufficient and appropriate information was responsible for most errors.

  2.2.  It is not possible to study cognitive processes by simply asking doctors how they make a diagnosis, as reasoning processes are not available for conscious report. Instead, it is necessary to carry out experiments with observation whilst doctors solve a particular problem, inferring the underlying process from the observable data (questions asked, comments made etc). In a recently completed study of difficult diagnoses in general practice, funded by the Dept of Health under the Patient Safety Research Programme, we identified information gathering as the most important determinant of accuracy.[48] Specifically, requesting more "critical information", ie information with diagnostic value for any of the relevant differential diagnoses, was associated with greater diagnostic accuracy by the GP, irrespective of experience (length of practice). Requesting more information or spending longer on a case did not predict accuracy; what mattered was requesting the "right" information. This finding suggests the importance of two factors for diagnostic accuracy: 1) formulating an appropriate set of differential diagnoses and 2) selecting appropriate information to test these diagnoses. Ongoing, in-depth analyses of misdiagnosed cases suggest that hypothesis generation is the key. In the majority of misdiagnoses, the correct hypothesis had not been considered at all by our study participants. In the absence of the correct hypothesis, appropriate information was not gathered or was dismissed.[49]

  2.3.  We do not advocate that GPs (or other clinicians) engage in exhaustive information gathering and generate complete lists of differential diagnoses at each and every consultation. This would go against how experience develops with practice. With experience, the need for slow and effortful analytical reasoning in familiar problems is reduced. We solve familiar problems faster and more successfully, because we have acquired stores of "patterns", "scripts", or prior instances that we can quickly and unconsciously match to the presenting problem.[50] We thus know what to do, without necessarily analysing why. Clinicians quickly recognise what is wrong with a patient from just a few features, very early on in the consultation or entertain a small number (2-4) of alternative hypotheses.[51] They do not systematically evaluate all possible hypotheses and do not gather large amounts of information before they make a diagnosis, as medical students are told to do. The advantage of this is reduction in time, risk and cost from unnecessary investigations and referrals. The disadvantage is that they will occasionally miss a diagnosis (15% of the time has been quoted for the medical specialties).[52]

  2.4.  Both researchers[53],[54] and clinicians[55] talk about "premature closure", ie stopping the search too quickly and adopting a diagnosis that is not sufficiently supported by the data. This has been attributed to a tendency to put more weight on first impressions and is considered to increase with age,[56] though the evidence for the latter is inconsistent.[57] It is plausible that with experience, confidence in one's diagnostic ability increases, so that if clinicians think that they recognise the cause of the patient's complaints, they are less likely to pursue other possibilities. This can be a problem in the less straightforward cases where "things are not what they seem". The question is how to support performance in these situations, without damaging performance on the easier cases that clinicians see every day and diagnose with accuracy and efficiency.


  3.1.  We argue for a 2-pronged approach. One centers around training and the other around diagnostic support. The training approach advocates providing practice on a range of carefully constructed diagnostic scenarios, with feedback. In the first instance, training could target GP registrars and, if found effective, it could extend to GPs. Training could be delivered over the Internet, so that clinicians could complete it at their own time. We recommend the development and testing of such a training package for General Practice.

  3.2.  Secondly, a long-term research programme is required that will develop easy-to-use decision-support systems, integrated with the electronic record and the clinician's workflow. Although diagnostic support systems have been developed since the `70s, they were not designed in a user-friendly way and did not integrate with the patient record. This meant that users took time to learn how to operate them and had to enter the data into the program, which meant that data were often incomplete. This made the system onerous and time-consuming for clinicians. Furthermore, clinicians are often not aware of their need for diagnostic support,[58] therefore, optional systems (that rely on the clinician realising the need and accessing them voluntarily) end up not being used. By the time that information is collected, clinicians have already made up their mind about the diagnosis and management and are therefore less likely to consult the system.

  3.3.  A recent qualitative study of patient safety features in the GP electronic health record highlighted the potential to link information in the record with external data to provide decision support and safety alerts.[59] A Cochrane review of the impact of computerised decision support systems found ten systems that supported diagnosis, none of which were in the primary care setting.[60] Four of these studies showed improvements in practitioner performance. The review also showed that systems that automatically prompted users, rather than requiring user activation, and teams where the research and system development were integrated were associated with positive effects on performance, whilst those that failed had been designed in isolation from the user base. There is potential to develop systems that are well integrated with the computer record and operate in the background, on the basis of the information that the clinician records during the consultation and information available in the patient record. Such systems could be activated at the end of the consultation, only to alert the clinician about diagnostic possibilities that need to be considered and to advise on how to test for them. Consideration needs to be given to the timing of alert and the type of complaint that will trigger activation of the system. Thorough evaluation of the system in practice is essential and priority should be given to funding research and development in this area.


  4.1.  In 2007, the Agency for Healthcare Research and Quality (AHRQ) announced interest in research on diagnostic errors in ambulatory care settings ( The AHRQ co-sponsored the first US national conference on diagnostic error in medicine that took place in Arizona in 2008 (May 31-June 1).

  4.2.  The EU has an interest in patient safety research too. As part of Framework Programme 7, a network on patient safety has been funded, led by the University of Manchester, with a dedicated workstream on diagnostic error led by our group at the University of Birmingham.


  5.1.  Similar to the Agency for Health Care Research and Quality, the National Institute for Health Research should prioritise a programme of research into the causes and potential solutions to diagnostic error. In particular, research on diagnostic training tools for clinicians and on well-designed computerised decision support is required.

  5.2.  The General Medical Council should require Medical Schools to place greater emphasis on the teaching of decision making, including diagnosis.

Dr Olga Kostopoulou and Professor Brendan Delaney

School of Health and Population Sciences, College of Medicine and Dentistry,

The University of Birmingham.

September 2008

31   Silk N. What went wrong in 1000 negligence claims. Health Care Risk Report, November 2000: 13-16. Back

32   The Medical Defence Union. Training and education: Primary care development programme-Risk management and delay in diagnosis 2004. Back

33   Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, et al. Missed and Delayed Diagnosis in the Ambulatory Setting: A Study of Closed Malpractice Claims. Ann Intern Med 2006;145(7):488-496. Back

34   Kostopoulou O, Delaney BC, Munro CW. Diagnositic difficulty and error in primary care-a systematic review. Fam Pract. (Accepted subject to minor revision). Back

35   Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008; 121(5 Suppl):S2-23. Back

36   Papa FJ, Oglesby MW, Aldrich DG, Schaller F, Copher DJ. Improving diagnostic capabilities of medical students via application of cognitive sciences-derived learning principles. Medical Education 2007;41(4):419. Back

37   Dovey SM, Meyers DS, Phillips RL, Jr., Green LA, Fryer GE, Galiher JM et al. A preliminary taxomony of medical errors in family practice. Quality and Safety in Health Care 2002; 11(3)233. Back

38   Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice; development of an error classification and pilot study of a method of detecting errors. Quality and Safety in Health Care2003;12(6):443. Back

39   Kistopoulou O, Denaley BC. Confidential reporting of patient safety events in privaty care: results from a multilevel classification of cognitive and system factors. QSHCBack

40   Phillips RL, Dovey SM, Graham D, Elder NC, Hickner JM. Learning from Different Lenses: Reports of Medical Errors in Primary Care by Clinicians, Staff and Patients: A Project of the American Acamemy of Family Physicians National Research Network. J Patient Safety 2006;2(3):140-146. Back

41   Fisseni G, Pentzek M, Abholz HH. Responding to serious medical error in general practice-consequences for the GPs involved: analysis of 75 cases from Germany. Fam Pract 2008;25(1):9-13. Back

42   Kostopoulou O. Do GPs report diagnostic errors£ Fam Pract 2008:25(1)1-2. Back

43   Bhasale A. The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring. Fam Pract 1998;15(4):308-318. Back

44   Phillips RL, Dovey SM, Graham D, Elder NC, Hickner JM. Learning from Different Lenses: Reports of Medical Errors in Primary Care by Clinicians, Staff and Patients: A Project of the American Acamemy of Family Physicians National Research Network. J Patient Safety 2006;2(3):140-146. Back

45   Ely JW, Levingson W, Elder NC, Mainous AG, Vinson DC. Perceived causes of family physicians" errors. J Fam Pract 1995;40(4):337-344. Back

46   Graber ML, Franklin N, Gordon R. Diagnostic Errors in Internal Medicine. Arch Interim Med 2005; 165:1493-1499. Back

47   Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, et al. Missed and Delayed Diagnosis in the Ambulatory Setting: A Study of Closed Malpractice Claims. Ann Intern Med 2006;145(7):488-496. Back

48   Kostopoulou O, Oudhoff J, Nath R, Delaney B, Munro C, Harries C, et al. Predictors of diagnostic accuracy and safe management in difficult diagnostic problems in Family Medicine. Med Mecis Making. Pre-published on June 12, 2008. Doi:10.1177/0272989X08319958. Back

49   Kostopoulo O, Devereaux Walsh C, Delaney B. Missing celiac disease in family medicine: the importance of hypothesis generation. Med Decis Making in press. Back

50   Norman GR, Brooks LR. The Non-Analytical Basis of Clinical Reasoning. Adv Health Sci Educ Theory Pract 1997;2(2):173. Back

51   Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, MA: Harvard University Press, 1978. Back

52   Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008; 121(5 Suppl):S2-23. Back

53   Voytovich AE, Rippey RM, Suffredini A, Premature conclusions in diagnostic reasoning. J Med Educ 1985;60(4):302. Back

54   Redelmeier DA. The cognitive psychology of missing diagnosis. Ann Intern Med 2005;142(2):115. Back

55   Ely JW, Levingson W, Elder NC, Mainous AG, Vinson DC. Perceived causes of family physicians" errors. J Fam Pract 1995;40(4):337-344. Back

56   Eva KW, Cunnington JPW. The difficulty with exerpience: Does practice increase susceptibility to premature closure? Journal of Continuing Education in the Health Professions 2007;26(3):192-198. Back

57   Friedman CP, Gatti GG, Franz TM, Murphy GC, Wolf FM, Heckerling PS, et al. Do Physcians Know When Their Diagnosis Are Correct? Implications for Decision Support and Error Reduction. J Gen Intern Med 2005; 20(4):334-339. Back

58   Friedman CP, Gatti GG, Franz TM, Murphy GC, Wolf FM, Heckerling PS, et al. Do Physcians Know When Their Diagnosis Are Correct? Implications for Decision Support and Error Reduction. J Gen Intern Med 2005; 20(4):334-339. Back

59   Morris CJ, Savelyich BS, Avery AJ, Cantrill JA, Sheikh A. Patient safety features of clinical computer systems: questionnaire survey of GP views. Qual Saf Health Care 2005;14(3):164-8. Back

60   Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293(10):1223-38. Back

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 30 October 2008