Annex
A SIMPLE DEMONSTRATION THAT "THE CLAIM
THAT BRIGHTER LIGHTING REDUCES CRIME IS UNFOUNDED"
By P R Marchant
The major systematic review on street lighting
and crime by Farrington and Welsh (2002) Home Office Research
Study 251, see www.homeoffice.gov.uk/rds/pdfs2/hors251.pdf, suggests
that claims for the effectiveness of lighting against crime are
justified. The review at first sight appears to be an appropriate
statistical synthesis of studies on street lighting and crime,
but on close examination the statistical claims and methods are
unfounded. In three cases examined there is a clear conflict between
the evidence and the reviewers' interpretation of this. One of
the principal problems is easily seen. The graph of the Bristol
lighting and crime study data, given below, shows no evidence
for the crime reduction benefit of lighting. However the review
gives the result for the same data as being extremely statistically
significant. It is suggested that such a difference between the
newly lit and the control areas occurring purely by chance is
less than one in a billion, but this is manifestly wrong. Two
other component studies, Bristol and Dudley are examined.
A major flaw with the review is to use methods
which ignore the large variation (overdispersion) in the data
and implicitly assume that crimes are independent events, which
is implausible in the extreme. As but one example of the problem,
one can examine the contribution from the Bristol study that used
data on crime from the beginning of 1986 to mid 1990. The treatment
area had brighter lighting introduced between July 1987 and March
1989 and the control area had its lighting left unchanged. The
reviewers compared the ratio of crimes committed in the first
year and the final year in both areas. It is claimed, by the reviewers,
that the benefit of lighting shown by this contributing study
is clear (z-statistic 6.6, consistent with its confidence interval
in the "forest plot" of Figure 3.1 HORS 251, being very
well displaced from the null line). The original paper (Shaftoe,
H (1994)) from which the data was taken makes no such claim for
the crime reduction benefit of lighting. Indeed anyone is free
to check for themselves with the data from Shaftoe.
Just inspection of the plot of the two time
series, noting when the new lighting went in, shows nothing to
support a claim of the benefit of lighting. (Remember, the reviewers'
claim that this data shows a very highly statistically significant
result for the benefit of lighting with probability of occurring
by chance of less than one in a billion, corresponding to z=6.6.
Their claim is literally incredible).
A requirement of the statistical method used
by the reviewers is that a typical range for fluctuation is equivalent
to approximately the square root of the mean, in this case say
around 20 something for the control and 30 something for the treatment
area. But the fluctuations are on average around 100 or so, this
shows that in this case the wrong method has been used and the
results are invalid, showing the reviewers' method is incorrect
in underestimating the true variability. This is consistent with
the conclusions of Shaftoe, the original investigator in the Bristol
study who could find no evidence that the new lighting reduced
crime, in stark contradiction to the reviewers' claim.
It is not just the Bristol study. The fundamental
problem is the review's method. The Birmingham Market Study, Poyner
and Webb (1997), another of those included in the review, also
clearly shows the excess variation via the two time points in
each of the four settings, (treatment and control and before and
after the intervention of brighter lighting). There is a much
larger drop than square root mean in the treatment area and also
an excessively large rise in the control area, before the new
lighting went in, for example:
One might ask why the variability in both the
Bristol and Birmingham studies is so much larger than that expected.
The answer is that crime events are not "statistically independent"
as the method used by the reviewers assumes, but are instead correlated.
Crime is perpetrated by people. There is repeat victimisation.
One criminal may be responsible for many crimes and this one person
changing behaviour can cause a large change in the number of crimes
committed and recorded.
Different statistical methods are needed to
deal with such variability. Where it has been possible to reanalyse
the data the appropriate methods have not provided evidence for
brighter lighting reducing crime.
There are also other problems with the review.
One is of not comparing like with like for the individual studies
in general. This is because brighter street lighting is applied
to more crime-ridden areas and the comparison areas are less crime-ridden.
This is rather like performing a trial of treatment for the common
cold on a group of people suffering quite lot but the people in
the control group are not very poorly at all. Then after following-up
the patients some time later, and finding them all virtual cold-free
a great success is claimed for the new treatment, as though this
had been responsible for bringing down the cold symptoms in the
ill group more than the standard treatment did in the not so ill
group. There are further statistical issues for example the reviewers
make the "unit of analysis error", as whole areas are
assigned to treatment rather than individuals.
There are a number of other shortcomings with
the review, however the Bristol study is sufficient to show how
and why the reviewers have got it very wrong. Crime reduction
is frequently presented as a potent argument for increased lighting—here
I have shown that there is no scientific basis for this claim.
A fuller examination of the review has been
written Marchant (2003), which gives much more detail.
References:
Farrington D.P. and Welsh B.C. (2002a) The Effects
of Improved Street Lighting on Crime: A systematic review, Home
Office Research Study 251, http://www.homeoffice.gov.uk/rds/pdfs2/hors251.pdf
Marchant P.R. (2003) The Claim that Brighter Lighting
Reduces Crime is Unfounded submitted for peer review and publication
with the Royal Statistical Society
Poyner, B. and Webb B. (1997) Reducing Theft from
Shopping Bags in City Center Markets, in Situational Crime
Prevention: successful case studies ed. R.V. Clarke. P83-89,
pub. Harrow and Heston.
Shaftoe, H (1994) Easton/Ashley, Bristol: Lighting
Improvements, in S. Osborn (ed.) Housing Safe Communities:
An Evaluation of Recent Initiatives pp 72-77, Safe Neighbourhoods
Unit, London.
30 April 2003
| 
