Select Committee on Health Written Evidence

Memorandum by Work Psychology Partnership (MMC 52)


  1.  Specifically, our work in 2006-07 comprised of advising on selection methodology for; (1) Foundation Programmes in the UK; (2) the GP selection process (shortlisting tests and selection centres) and; (3) selection into all other specialities at ST1. The time scale for delivery was extremely tight and we expressed our concerns at the outset (June 2006). We were also concerned about the lack of piloting opportunity. The selection methodologies for both Foundation and GP have been shown to work well.

  2.  In relation to Specialty selection, The Work Psychology Partnership (WPP) responded to a DH Invitation to Tender in May 2006. The scope of work states; "The number of applicants expected to apply for entry into Specialty Training is approximately 6,000 and that applications will be via a single electronic national portal entry system (separate project) the working assumption for the closing date will be 5 January 2007". We were informed that there would be 16 specialties and there were existing Person Specifications submitted by the AoMRC. Although there were 16 specialties in the original scope, in late autumn 2006, WPP were asked to deliver an additional 110 Person Specifications (signed-off by College representatives) to include those in transition at levels ST2, ST3, ST4 plus FTSTAs). At the outset we were asked to deliver a shortlisting process for ST1. We were not asked to deliver the selection methodology for doctors in "transition" via ST2, ST3, ST4 and FTSTAs. We were led to believe transition arrangements would be delivered via local processes.

  3.  In relation to shortlisting for ST1, WPP recommended to COPMeD the use of a national standardised test in 2007. However, we were informed that this was not the preferred option for 2007. By contrast, in collaboration with the GP National Recruitment Office, WPP designed the shortlisting test for GP, which has shown to work well.

  4.  Given the time scale (approximately 12-16 weeks) we had no option but to use existing application form materials. Our input into design of the application form for 2007 was directly based on existing application forms previously used for entry into specialty training (including those used by the London and Yorkshire Deaneries). WPP were sent the application form template by the Project Manager (then Ms Carole Mistry, latterly, Ms Janet Brown).

  5.  On reviewing existing application forms, "white space" questions were widely used. This is not an invention from WPP. "White space" questions were used in the past, and are used frequently for shortlisting purposes in medical selection. However, our advice was that if used, they would not be a long term solution given concerns regarding plagiarism (see GP experience where our advice and work has involved the development of other short listing solutions). We were clear that the use of these questions was an interim solution and that other short-listing procedures should be developed. Specifically, we highlighted all the risks to the COPMeD steering group (including need for training, standardised rating scales etc). In addition, we were led to believe that MTAS would comprise software to detect plagiarism.

  6.  Relevant experience, posts completed and other CV related information are clearly important indicators of competence and would normally be used to make selection decisions. However, we were told that there should be no weighting of previous relevant experience. This is unusual in a selection context. The decision not to score experience and previous posts was not ours, it was the brief given. This was guidance from COPMeD and MMC team regarding PMETB standards. This is not an interpretation or new development proposed by Work Psychology Partnership. Similarly, we were advised that gaining a College exam could not be "scored/used to rank individuals", even though these are clearly robust indicators of competence. The shortlisting scoring framework received legal and HR approval from the Department of Health. WPP were not responsible for HR input at any stage. HR expertise was provided by the DH.

  7.  We were led to believe that there would be sufficient posts to applicants so that the vast majority of applicants would gain a training post. Interview capacity was determined locally. WPP were told that the competition ratios would not be too high and so short listing would be relatively "light touch". The actual numbers of applicants was vastly underestimated by various stakeholders. In fact, the shortlisting system had to sift over 128,000 applications for approximately 20,000 posts. This inevitably meant that several thousand competent doctors would not be invited to interview, let alone offered a post. Our scope of work was for ST1 applicants only and we believed that the vast majority of applicants would be shortlisted.

  8.  All our work was signed off by the COPMeD steering group and the Department of Health (we have a full contact log to this effect). WPP can provide a full log of all communications with all stakeholders on request. Both parties stated that they were happy with our work and that we had fulfilled our contract.

  9.  WPP were not involved in any way with the development of selection methodology for Academic posts. We were asked to provide advice in November 2006 but we declined because of our concerns regarding the lead time to develop appropriate scoring frames for the academic community. We were also concerned about the time scale for consultation. As a result, Professor Thomas (Chair, COPMeD steering Group) informed us that this was to be delivered by "Mark Walport's group". WPP had no involvement in academic recruitment.

  10.  At the outset, given that the original project manager left her post (Carole Mistry), there was no formal discussion about contracting arrangements and we worked at risk for several months. We tried to address this on several occasions as the team was working at risk for several months. There were gaps in project management and we were unclear about who was leading our contract management. I asked for an urgent meeting with DH staff in January 2007 as I had major concerns about how our contract was being managed and the work demands therein. As suppliers, throughout this project there was a lack of appropriate project management and as a result, reporting arrangements were very unclear. We were not presented with a contract to sign from the DH until 17 February 2007, which was nine months after we had commenced the work. We expressed our concerns throughout this time as the work plan shifted significantly out of scope.

  11.  In October 2004, Professor Patterson presented a list of all the risks of inadequate preparation and development of the selection methodology at a meeting with the MMC team (then Dominic Hurndall and Dr Andrew Havers and others). She recommended a selection methodology comprising selection centres that could be validated over two years with large scale stakeholder consultation. Following this meeting, Professor Patterson received no further correspondence from the MMC team regarding selection methodology, and no pilots were delivered.

  12.  On all counts, as suppliers, WPP delivered our original scope of work plus major and significant out of scope activities in an extremely tight timescale. The design brief was changed significantly and our early advice on selection methodology was not taken up. Within this time we made significant personal efforts to consult with all key stakeholders, including presentations to the AoMRC STC committee, and delivering an additional 19 workshops with stakeholders in an 8 week period. WPP cannot be held responsible for a reported lack of consultation. Our expertise and long-standing reputation has been questioned as a result of this process. We are particularly unhappy that detailed information regarding our involvement and our responsibilities has not yet been made clear by the various stakeholders involved, who were responsible for setting, managing and "signing-off" our scope of work.

November 2007

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2008
Prepared 8 May 2008