3 Removal of National Statistics status
47. 'National Statistics' is the quality mark for
official statistics and, in the words of UKSA, it "requires
the statistics to be produced, managed and disseminated to high
professional standards. The statistics must be well-explained
and meet users needs."[39]The
Statistics and Registration Services Act 2007 stipulates that
an official statistical series must adhere to the statutory Code
of Practice for Official Statistics (the Code) in order for it
to be designated as a National Statistic. The Code enshrines the
principle of "sound methods and assured quality" and
requires producers of official statistics to "seek to achieve
continuous improvement in statistical processes by, for example,
undertaking regular reviews".[40]
The Code also requires that official statistics be well-explained
and accessible, and managed impartially in the public interest,
"to serve the public good".[41]Since
it was established in April 2008,UKSA has performed the role of
conducting assessment reviews of each set of official statistics
to determine the extent of their compliance with the Code of Practice.
48. The initial UKSA assessment of PRC, published
in April 2011, confirmed the National Statistics designation,
judging that it complied with the statutory Code of Practice.
The 2011 UKSA assessment, like the National Statistician's review
of crime statistics published later that year,focussed on presentational
aspects of the statistics and the provision by the Home Office
of explanatory and methodological material to users. It paid less
attention to the quality of the statistics and issues surrounding
the creation and handling of the raw data at force level, although
it did observe that "there is scope for more discussion about
quality, including the accuracy and extent of crime recording
by different police forces."[42]
49. The Home Office asked the National Statistician
to carry out a review into crime statistics, which reported in
2011. The terms of reference for the review, set by the Home Office,asked
ONS to "consider which body outside the Home Office is best
placed to have future formal responsibility for the publication
of crime statistics" due to concerns about the trustworthiness
of crime statistics published by the Home Office.[43]After
the initial UKSA assessment report, and in accordance with a recommendation
in the National Statistician's 2011 review, the ONS took over
the role of producer of the quarterly crime statistics outputs.
This included taking over, from the Home Office, the responsibility
to provide reassurance to the UKSA assessment team as to the quality
of the data collection processes and methodology. However, it
was also decided that the Home Office should retain its role in
the initial collection and validation of forces' monthly data
returns, on the basis of "existing relationships between
the Home Office and the police service" and in order to "avoid
an increase in bureaucracy in the police supply of data to both
ONS (for crime statistics purposes) and to the Home Office (for
other purposes)."[44]
50. ONS's statisticians do not, in any case, have
direct access to the locations at which these data are generated:
inside police forces. To a large extent they have been and remain
external observers, reliant on the data submissions and reassurances
from the Home Office. They do not have an audit function and are
reliant on any audits performed internally within forces or externally
by other organisations.
51. Professor Stephen Shute, the Chair of the Crime
Statistics Advisory Committee, told us that it would be "a
very sad state of affairs" were such a de-designation to
occur, while his colleague on CSAC Professor Mike Hough expressed
the concern that "if they were downgraded, the system would
collapse."[45] Nevertheless,
following evidence exposed by this PASC inquiry, UKSA made a second
assessment of crime statistics, published in January 2014. They
removed the designation'National Statistics' from PRC data. The
separate CSEW retained its National Statistics status.[46]
In its assessment report, UKSA noted:
· accumulating evidence that suggests the
underlying data on crimes recorded by the police may not be reliable.
This evidence includes HMIC assessments of data recording practices;
· ONS's own report, in January 2013, which
raised concerns that the degree of compliance with the standards
for police crime recording may be falling; and
· high profile concerns raised at the Public
Administration Select Committee and the Home Affairs Select Committee.[47]
52. This assessment, unlike the one in 2011, also
concluded that ONS did not have sufficient knowledge of (and therefore
did not publish enough information about) the processes involved
in the recording of crime by police forces and the checks carried
out on the data received from police forces, to be assured that
they are accurately recorded. It did not provide enough information
to users about the quality of the statistics-most importantly
the accuracy and reliability of the statistics. The assessment
report also referred to an unpublished memorandum of understanding
between the ONS and the Home Office which outlines the roles and
responsibilities of each department in the production of crime
statistics, and recommended that the ONS should publish this information.
The Crime Statistics Advisory
Committee (CSAC)
53. The Crime Statistics Advisory Committee (CSAC)
is a non-statutory body established in 2011 by the National Statistician.
It has seven permanent members, from the Home Office, ONS and
other government bodies working on crime statistics,and eight
non-executive members who are mostly academics. The National Statistician
is an ex-officio member. Its terms of reference state that it:
advises the Home Secretary, the Office for National
Statistics and HerMajesty's Inspectorate of Constabulary on how
best to ensure that official statisticson crime for England and
Wales are accurate, clearly presented, comprehensive,transparent
and trustworthy taking account of the needs of users and providers.[48]
It is clear from the CSAC's correspondence and its
latest Annual Report (for 2012-13)that it has done little on the
issue of the divergence in trends between CSEW and PRC. This is
despite the National Statistician saying in her report that established
CSAC that it should advise "on any changes to the data requirements
from the police needed for crime statistics and any chances to
Home Office Counting Rules".[49]
The Annual Report states only that it "has been considering
issues arising from the divergence of data between the Crime Survey
for England and Wales (CSEW) and Police Recorded Crime (PRC)",
with a view to having a meeting at the end of 2011.[50] It
is clear that the Committee's power is limited. Professor Hough,member
of CSAC, told us"We can only really operate at arm's length
and look at the governance structures that affect local crime
recording. [...] We do not have larger, more direct levers."[51]
Professor Shute, the Chair of CSAC, added:
[...] we do not have an executive function. We
have done a number of things to try to bolster confidence in crime
data. We can analyse trends and we can look to see if those disclose
worrying aspects of the current system. [...] We can and have
offered advice on how the data ought to be presented in a way
that is comprehensible to users and to the public more generally,
and we have done. There are a number of things we can do, but
there are obviously limitations on what we can do. We are an advice
body [...]
54. We commend
UKSA for acting in response to the evidence exposed by PASC's
inquiry, to strip Police Recorded Crime statisticsof the quality
designation 'National Statistics'. However, the fact that it took
our inquiry, and a whistleblower from the Metropolitan Police
Service, to expose sufficient evidence suggests serious shortcomings
in UKSA's ability and capacity in their assessment function. We
acknowledge their recent decision to remove the designation 'National
Statistics', but this cannot mitigate what amounts to a long-standing
failure of a number of bodies to address the thoroughness of the
assessment of Police Recorded Crime, despite a series of previous
reviews which identified shortcomings.
55. This raises
serious concerns around the decision to designate Police Recorded
Crime as National Statistics in 2011. It has been quoted by ministers
that the ONS described the system for recording crime in England
and Wales as "one of the best in the world" in 2012.[52]This
was after the cessation of regular external audit of force crime
recording in 2007.All can see now that this reflected a lamentable
complacency. The then National Statistician took no action at
that time. This was wrong-the then National Statistician, or UKSA,
once established, should have pressed for other process to be
put in place to ensure the integrity of crime data.
56. The reviews
of crime statistics by UKSA and the ONS in 2011 failed to expose
the unreliability of recording practices within police forces
themselves. An opportunity was therefore missed to gather evidence
and identify issues which could have called into question the
designation of Police Recorded Crimeas 'National Statistics'at
a much earlier stage.
57. It isdeplorable
that ONS can have overseen the production of crime statistics,
which were a set of National Statistics, with what appears to
have been very limited knowledge of the 'quality assurance'steps
that the data went through before being sent to the ONS.The ONS
has been too reliant on too little information about the audits
performed within police forces or by HM Inspectorate of Constabulary.
Overall, the ONS has been too passive in carrying out their duties
in relation to crime statistics. This cannot continue.
58. The fragmentation
of responsibility between individual forces, Home Office and the
ONS was not satisfactory and contributed to the failure of the
Police Recorded Crime series to meet the standards of the Code
of Practice with which official statistics must comply. No single
organisation has taken overall responsibility or accountability
for ensuring an acceptable quality of crime statistics, which
has led to their inadequate quality.
59. We endorse UKSA's recommendation that the
ONS should publish a clear statement of the respective roles and
responsibilities of the Home Office and the ONS in the production
of Police Recorded Crime statistics.
60. We recommend UKSA works closely with the Home
Office in its role as the first recipient of raw data from forces,
and ensures the Home Office takes active primary operational responsibility
and accepts accountability for ensuring the integrity of the data
which it collates, validates and submits to the ONS for publication.
UKSA should hold the Home Office directly accountable for its
role in the recorded crime statistics process, including its validation
and quality assurance processes as well as its policy guidance
to forces and Police and Crime Commissioners, and should in future
examine the Home Office's processes and procedures directly rather
than at one remove.
61. The Crime
Statistics Advisory Committee (CSAC), which contains representation
of all of the main stakeholders in the crime statistics production
process as well as the Chief Inspector of Constabulary, has failed.
It has not demonstrated sufficient independence and objectivity
in carrying out its role to ensure recorded crime statistics are
"accurate, clearly presented, comprehensive, transparent
and trustworthy" as set out in its terms of reference.CSAC
has a vital role in leading the efforts to provide thatthe system
guarantees the reliability and integrity of all crime statistics
emerge strengthened from this episode.
62. We recommend that UKSA should review the role
and composition of CSAC and the structures supporting the production
of crime statistics, just as it has recently with a similar committee
advising on inflation figures, to ensure that CSAC is independent
and rigorous and that these statistics best meet user needs in
future.
Implications for other public
sector administrative statistics
63. The de-designation of PRC potentially raises
concerns about data quality and integrity elsewhere in the public
sector, where personal and organisational performance is measured
against data which those same organisations are responsible for
generating, as acknowledged by UKSA. As Ed Humpherson, Head of
Assessment at UKSA,told us: "We need to be clear in the way
we prioritise the reassessments of official statistics that we
place those statistics that are drawn from administrative data
high up our list so we get to them first."[53]
UKSA have subsequently published more information on the work
which they will be doing to look at statistics produced in areas
where targets exist.[54]
64. We welcome
UKSA's comments that it intends to prioritise in its workplan
the reassessment of National Statistics based on administrative
datasets, taking on board the lessons learned from the declassification
of Police Recorded Crime.
65. UKSA must not in future grant to, or maintain,
the kitemark of 'National Statistics'on any set of statistics
where it has failed to verify whether the underlying datameets
the standard required. They should, as a matter of urgency, review
all other similar statistics where collection processes are beyond
the control of the ONS. UKSA should review the Code of Practice
for Official Statistics to determine whether it needs to be revised
to allow for the new emphasis on administrative data.
39 UK Statistics Authority website Back
40
UK Statistics Authority, Code of Practice for Official Statistics,
January 2009 Back
41
As above Back
42
UK Statistics Authority, Assessment Report 102, Crime Statistics
in England and Wales, April 2011, para 3.16 Back
43
National Statistician's Review of Crime Statistics: England and
Wales, June 2011, Annex A: Terms of reference Back
44
CST17 Back
45
Q205 and Q206 Back
46
Police Recorded Crime data continues to be published as usual
in the ONS's quarterly Crime Statistics bulletin, but with the
loss of National Statistics status signposted where these data
appear. Back
47
UK Statistics Authority, Assessment Report 268, Statistics on
Crime in England and Wales, 15 January 2014 Back
48
UK Statistics Authority, Crime Statistics Advisory Committee terms
of reference, August 2013 Back
49
National Statistician's Review of Crime Statistics: England and
Wales, 2011 Back
50
Crime Statistics Advisory Committee, Annual Report 2012-13 Back
51
Q146 Back
52
Decision to designate as National Statistics: see UK Statistics
Authority, Crime Statistics in England and Wales: Assessment with
the Code of Practice for Official Statistics, April 2011 and Letter
of confirmation as National Statistics, January 2012. Quotation
from ONS: Methodological note: Analysis of variation in crime
trends, January 2013. Quoted by Norman Baker MP, Q531 Back
53
Q475 Back
54
UK Statistics Authority,Administrative data and Official Statistics
press statement, 7 February 2014 and UK Statistics Authority website,
Administrative data and Official Statistics webpage giving more
detail on the programme of work. Back
|