Caught red-handed: Why we can't count on Police Recorded Crime statistics - Public Administration Committee Contents


3  Removal of National Statistics status

47. 'National Statistics' is the quality mark for official statistics and, in the words of UKSA, it "requires the statistics to be produced, managed and disseminated to high professional standards. The statistics must be well-explained and meet users needs."[39]The Statistics and Registration Services Act 2007 stipulates that an official statistical series must adhere to the statutory Code of Practice for Official Statistics (the Code) in order for it to be designated as a National Statistic. The Code enshrines the principle of "sound methods and assured quality" and requires producers of official statistics to "seek to achieve continuous improvement in statistical processes by, for example, undertaking regular reviews".[40] The Code also requires that official statistics be well-explained and accessible, and managed impartially in the public interest, "to serve the public good".[41]Since it was established in April 2008,UKSA has performed the role of conducting assessment reviews of each set of official statistics to determine the extent of their compliance with the Code of Practice.

48. The initial UKSA assessment of PRC, published in April 2011, confirmed the National Statistics designation, judging that it complied with the statutory Code of Practice. The 2011 UKSA assessment, like the National Statistician's review of crime statistics published later that year,focussed on presentational aspects of the statistics and the provision by the Home Office of explanatory and methodological material to users. It paid less attention to the quality of the statistics and issues surrounding the creation and handling of the raw data at force level, although it did observe that "there is scope for more discussion about quality, including the accuracy and extent of crime recording by different police forces."[42]

49. The Home Office asked the National Statistician to carry out a review into crime statistics, which reported in 2011. The terms of reference for the review, set by the Home Office,asked ONS to "consider which body outside the Home Office is best placed to have future formal responsibility for the publication of crime statistics" due to concerns about the trustworthiness of crime statistics published by the Home Office.[43]After the initial UKSA assessment report, and in accordance with a recommendation in the National Statistician's 2011 review, the ONS took over the role of producer of the quarterly crime statistics outputs. This included taking over, from the Home Office, the responsibility to provide reassurance to the UKSA assessment team as to the quality of the data collection processes and methodology. However, it was also decided that the Home Office should retain its role in the initial collection and validation of forces' monthly data returns, on the basis of "existing relationships between the Home Office and the police service" and in order to "avoid an increase in bureaucracy in the police supply of data to both ONS (for crime statistics purposes) and to the Home Office (for other purposes)."[44]

50. ONS's statisticians do not, in any case, have direct access to the locations at which these data are generated: inside police forces. To a large extent they have been and remain external observers, reliant on the data submissions and reassurances from the Home Office. They do not have an audit function and are reliant on any audits performed internally within forces or externally by other organisations.

51. Professor Stephen Shute, the Chair of the Crime Statistics Advisory Committee, told us that it would be "a very sad state of affairs" were such a de-designation to occur, while his colleague on CSAC Professor Mike Hough expressed the concern that "if they were downgraded, the system would collapse."[45] Nevertheless, following evidence exposed by this PASC inquiry, UKSA made a second assessment of crime statistics, published in January 2014. They removed the designation'National Statistics' from PRC data. The separate CSEW retained its National Statistics status.[46] In its assessment report, UKSA noted:

·  accumulating evidence that suggests the underlying data on crimes recorded by the police may not be reliable. This evidence includes HMIC assessments of data recording practices;

·  ONS's own report, in January 2013, which raised concerns that the degree of compliance with the standards for police crime recording may be falling; and

·  high profile concerns raised at the Public Administration Select Committee and the Home Affairs Select Committee.[47]

52. This assessment, unlike the one in 2011, also concluded that ONS did not have sufficient knowledge of (and therefore did not publish enough information about) the processes involved in the recording of crime by police forces and the checks carried out on the data received from police forces, to be assured that they are accurately recorded. It did not provide enough information to users about the quality of the statistics-most importantly the accuracy and reliability of the statistics. The assessment report also referred to an unpublished memorandum of understanding between the ONS and the Home Office which outlines the roles and responsibilities of each department in the production of crime statistics, and recommended that the ONS should publish this information.

The Crime Statistics Advisory Committee (CSAC)

53. The Crime Statistics Advisory Committee (CSAC) is a non-statutory body established in 2011 by the National Statistician. It has seven permanent members, from the Home Office, ONS and other government bodies working on crime statistics,and eight non-executive members who are mostly academics. The National Statistician is an ex-officio member. Its terms of reference state that it:

    advises the Home Secretary, the Office for National Statistics and HerMajesty's Inspectorate of Constabulary on how best to ensure that official statisticson crime for England and Wales are accurate, clearly presented, comprehensive,transparent and trustworthy taking account of the needs of users and providers.[48]

It is clear from the CSAC's correspondence and its latest Annual Report (for 2012-13)that it has done little on the issue of the divergence in trends between CSEW and PRC. This is despite the National Statistician saying in her report that established CSAC that it should advise "on any changes to the data requirements from the police needed for crime statistics and any chances to Home Office Counting Rules".[49] The Annual Report states only that it "has been considering issues arising from the divergence of data between the Crime Survey for England and Wales (CSEW) and Police Recorded Crime (PRC)", with a view to having a meeting at the end of 2011.[50] It is clear that the Committee's power is limited. Professor Hough,member of CSAC, told us"We can only really operate at arm's length and look at the governance structures that affect local crime recording. [...] We do not have larger, more direct levers."[51] Professor Shute, the Chair of CSAC, added:

    [...] we do not have an executive function. We have done a number of things to try to bolster confidence in crime data. We can analyse trends and we can look to see if those disclose worrying aspects of the current system. [...] We can and have offered advice on how the data ought to be presented in a way that is comprehensible to users and to the public more generally, and we have done. There are a number of things we can do, but there are obviously limitations on what we can do. We are an advice body [...]

54. We commend UKSA for acting in response to the evidence exposed by PASC's inquiry, to strip Police Recorded Crime statisticsof the quality designation 'National Statistics'. However, the fact that it took our inquiry, and a whistleblower from the Metropolitan Police Service, to expose sufficient evidence suggests serious shortcomings in UKSA's ability and capacity in their assessment function. We acknowledge their recent decision to remove the designation 'National Statistics', but this cannot mitigate what amounts to a long-standing failure of a number of bodies to address the thoroughness of the assessment of Police Recorded Crime, despite a series of previous reviews which identified shortcomings.

55. This raises serious concerns around the decision to designate Police Recorded Crime as National Statistics in 2011. It has been quoted by ministers that the ONS described the system for recording crime in England and Wales as "one of the best in the world" in 2012.[52]This was after the cessation of regular external audit of force crime recording in 2007.All can see now that this reflected a lamentable complacency. The then National Statistician took no action at that time. This was wrong-the then National Statistician, or UKSA, once established, should have pressed for other process to be put in place to ensure the integrity of crime data.

56. The reviews of crime statistics by UKSA and the ONS in 2011 failed to expose the unreliability of recording practices within police forces themselves. An opportunity was therefore missed to gather evidence and identify issues which could have called into question the designation of Police Recorded Crimeas 'National Statistics'at a much earlier stage.

57. It isdeplorable that ONS can have overseen the production of crime statistics, which were a set of National Statistics, with what appears to have been very limited knowledge of the 'quality assurance'steps that the data went through before being sent to the ONS.The ONS has been too reliant on too little information about the audits performed within police forces or by HM Inspectorate of Constabulary. Overall, the ONS has been too passive in carrying out their duties in relation to crime statistics. This cannot continue.

58. The fragmentation of responsibility between individual forces, Home Office and the ONS was not satisfactory and contributed to the failure of the Police Recorded Crime series to meet the standards of the Code of Practice with which official statistics must comply. No single organisation has taken overall responsibility or accountability for ensuring an acceptable quality of crime statistics, which has led to their inadequate quality.

59. We endorse UKSA's recommendation that the ONS should publish a clear statement of the respective roles and responsibilities of the Home Office and the ONS in the production of Police Recorded Crime statistics.

60. We recommend UKSA works closely with the Home Office in its role as the first recipient of raw data from forces, and ensures the Home Office takes active primary operational responsibility and accepts accountability for ensuring the integrity of the data which it collates, validates and submits to the ONS for publication. UKSA should hold the Home Office directly accountable for its role in the recorded crime statistics process, including its validation and quality assurance processes as well as its policy guidance to forces and Police and Crime Commissioners, and should in future examine the Home Office's processes and procedures directly rather than at one remove.

61. The Crime Statistics Advisory Committee (CSAC), which contains representation of all of the main stakeholders in the crime statistics production process as well as the Chief Inspector of Constabulary, has failed. It has not demonstrated sufficient independence and objectivity in carrying out its role to ensure recorded crime statistics are "accurate, clearly presented, comprehensive, transparent and trustworthy" as set out in its terms of reference.CSAC has a vital role in leading the efforts to provide thatthe system guarantees the reliability and integrity of all crime statistics emerge strengthened from this episode.

62. We recommend that UKSA should review the role and composition of CSAC and the structures supporting the production of crime statistics, just as it has recently with a similar committee advising on inflation figures, to ensure that CSAC is independent and rigorous and that these statistics best meet user needs in future.

Implications for other public sector administrative statistics

63. The de-designation of PRC potentially raises concerns about data quality and integrity elsewhere in the public sector, where personal and organisational performance is measured against data which those same organisations are responsible for generating, as acknowledged by UKSA. As Ed Humpherson, Head of Assessment at UKSA,told us: "We need to be clear in the way we prioritise the reassessments of official statistics that we place those statistics that are drawn from administrative data high up our list so we get to them first."[53] UKSA have subsequently published more information on the work which they will be doing to look at statistics produced in areas where targets exist.[54]

64. We welcome UKSA's comments that it intends to prioritise in its workplan the reassessment of National Statistics based on administrative datasets, taking on board the lessons learned from the declassification of Police Recorded Crime.

65. UKSA must not in future grant to, or maintain, the kitemark of 'National Statistics'on any set of statistics where it has failed to verify whether the underlying datameets the standard required. They should, as a matter of urgency, review all other similar statistics where collection processes are beyond the control of the ONS. UKSA should review the Code of Practice for Official Statistics to determine whether it needs to be revised to allow for the new emphasis on administrative data.


39   UK Statistics Authority website Back

40   UK Statistics Authority, Code of Practice for Official Statistics, January 2009 Back

41   As above Back

42   UK Statistics Authority, Assessment Report 102, Crime Statistics in England and Wales, April 2011, para 3.16 Back

43   National Statistician's Review of Crime Statistics: England and Wales, June 2011, Annex A: Terms of reference Back

44   CST17 Back

45   Q205 and Q206 Back

46   Police Recorded Crime data continues to be published as usual in the ONS's quarterly Crime Statistics bulletin, but with the loss of National Statistics status signposted where these data appear. Back

47   UK Statistics Authority, Assessment Report 268, Statistics on Crime in England and Wales, 15 January 2014 Back

48   UK Statistics Authority, Crime Statistics Advisory Committee terms of reference, August 2013 Back

49   National Statistician's Review of Crime Statistics: England and Wales, 2011 Back

50   Crime Statistics Advisory Committee, Annual Report 2012-13 Back

51   Q146 Back

52   Decision to designate as National Statistics: see UK Statistics Authority, Crime Statistics in England and Wales: Assessment with the Code of Practice for Official Statistics, April 2011 and Letter of confirmation as National Statistics, January 2012. Quotation from ONS: Methodological note: Analysis of variation in crime trends, January 2013. Quoted by Norman Baker MP, Q531 Back

53   Q475 Back

54   UK Statistics Authority,Administrative data and Official Statistics press statement, 7 February 2014 and UK Statistics Authority website, Administrative data and Official Statistics webpage giving more detail on the programme of work. Back


 
previous page contents next page


© Parliamentary copyright 2014
Prepared 9 April 2014