Examination of Witnesses (Questions 1-19)
MS KAREN
DUNNELL AND
MR MIKE
HUGHES
19 MARCH 2008
Q1 Dr Gibson: Thank you for coming and
helping us in what we hope will be an interesting inquiry about
Government statistics and policy-making. We are very aware that
this change is taking place in structures and we do not want to
get into that, but we would like some advice from you to carry
it forward. I will ask you the first question, and then other
Members of the Committee will pile in. We want to try and get
it over in an hour. Tell us about the different types of statistics
that you record, the national statistics and official statistics
and the differentiation between them. What are the differences,
and are they important?
Ms Dunnell: The Government and
its agencies of course produce a very wide range of statistics;
probably about half of these are badged as national statistics;
and we refer to the whole lot as official statistics. National
statistics are characterised by the kind of quality badge that
represents the fact that they adhere to the statistics produced
under proper professional mechanisms, and adhere to the code of
practice, which means they are fit for purpose, they are relevant,
they are high quality, they are accessible in terms of being on
the website and so on, and they are produced with integrity—a
whole series of things like that. The Government's statistical
surveys work to a very clear code of practice. They were badged
in 2000, when the last reform of government statistics took place;
and at that time all statistics produced by ONS, which represents
about a fifth of all national statistics, were going to be national
statistics, and the rest were decided by Ministers in departments.
Since then, some extra statistics have come into the fold, if
you like; and basically that is something that Ministers advise
on; and then I and my colleagues rubber-stamp. We would do an
assessment and decide—
Q2 Dr Gibson: Do you know which class
of statistics you are handling?
Ms Dunnell: Yes.
Q3 Dr Gibson: How is that decision
made? You say Ministers, but statistics are statistics are statistics,
are they not?
Ms Dunnell: Yes, but this is one
of the very interesting things about statistics; that in the UK,
in 1999, when the last reforms were decided upon and consulted
on, and indeed in the debate that has taken place more recently
in Parliament, everyone decided that this should remain in the
gift of Ministers, so it is a parliamentary decision, the latest
one.
Q4 Dr Gibson: Could you hazard a
guess why Ministers want to keep control of this?
Ms Dunnell: Well, I could hazard
a guess about it, yes. It is partly of course that many of the
statistics that are produced, which we refer to as official statistics,
are those figures which emanate from administrative and management
systems, for example in the health service, the prison service
or the police service, and they are used very intensively for
management and administrative purposes and may not ever in some
senses fulfil the very strict codes of practice that we work to,
particularly on things like publication processes.
Q5 Dr Gibson: Do you think the words
"statistically proven" are just used in a loose kind
of way by politicians? I assume if the Prime Minister will say
to me "statistics have proven that we are not as poor as
we used to be" and all this kind of stuff—the word
"statistically" gets thrown into the hat somewhere!
Is that your feeling?
Ms Dunnell: It may do, but on
the other hand we do produce a large number of series, the whole
intention being to demonstrate how our economy and society are
developing; so in many cases, for example child poverty, we do
have in the Government statistical survey methods of working out
whether it has gone up or down. That is entirely statistically
good practice.
Q6 Dr Gibson: Do you feel you are
ever compromised in different positions and different committees
you are on? Is that a possibility; or can you sit back and be
quite neutral about everything?
Ms Dunnell: Yes, my job, partly
as a statistician, and as a senior civil servant, is to be impartial
and to give advice based on the statistics that I am there to
talk about.
Q7 Dr Iddon: Who decides the boundaries
between national and official statistics, and are there statistics
that are produced as official statistics that you feel ought to
be national?
Ms Dunnell: Yes, there probably
are some of those. I think that the new UK Statistics Authority
is planning to take a pretty systematic look at the whole of official
statistics and give some advice about whether the boundaries are
right.
Q8 Mr Boswell: Will that require
a re-definition of the criteria, to see what they are; or at least
a fresh look at that?
Ms Dunnell: One of the things
that the UK SA will do is review the existing code of practice,
which we have had in place for more than ten years now; and then
they will apply those new criteria, which I do not suppose will
be very different because there is an international set of codes
of practice for statistics, which are pretty similar across the
world.
Mr Hughes: One of the things that
the Act did was to give the Statistics Authority the powers to
oversee the whole of the UK official statistics system, rather
than focusing just on national statistics. It gives the Authority
the power to raise questions with Ministers in Parliament about
their treatment of official statistics and whether they should
be national statistics.
Q9 Mr Boswell: Will that be a public
dialogue do you anticipate; or will it be a matter of private
advice to Ministers who, if their official statistics are controlling
them rather than—
Mr Hughes: I think the answer
to that is still not clear. The Authority is currently looking
at its policy on transparency, and will seek to be as open and
transparent as possible; but in the context of dealing with Ministers,
you will have to respect that dialogue.
Q10 Graham Stringer: You have said
that you want to improve the public perception of statistics,
and that the SA will help you to do that. How will you do that?
Is that just an aspiration? Can you put some flesh on the bones
to convince us that you have a real plan that will improve?
Ms Dunnell: It is aspirational
but of course we believe it is very important that this work we
do, which is extensively used across Government, is much better
regarded by the public. The UK SA will have plans, and indeed
we always have things that we are trying to do to improve trust.
I think the main one will be what the UK SA has already decided,
which is to set up a web hub, a single place on the Internet,
where all official statistics will be published together. That
will be very separate, and seen to be very separate from the ministerial
statements about those statistics. The intention is to for that
to come into place on 1 April. That will make, hopefully, quite
a big difference. The other things are down to quite a lot of
things that I can try to do in my position as National Statistician,
which is to raise the standards of what we do across the Government's
Statistical Service, particularly in terms of things like making
the statistics we produce understandable, interesting, and used
in a relevant way, because the more people who can understand
what we do and use them in their work or, for example, at local
level in helping to get the kind of things they want at local
level, then gradually we will build up respect and regard for
these very important statistics. Those are the kinds of things
we try to do.
Q11 Graham Stringer: Do you think
that will be sufficient when less than a fifth of people currently
believe official statistics? Do you think your website and making
your work technically superior will change people's perception?
Is there reliability in statistics?
Ms Dunnell: That is the intention.
I agree that it is probably quite a long, slow process; but the
real secret is widening the community of people who understand
statistics and what they are useful for and get some benefit out
of using them. That is really the challenge.
Mr Hughes: One of the other major
planks with trying to improve trust—and it is very much
a perception problem—is reducing both the time and the number
of people who would see the statistics in advance. That is something
that the Government, during the passage of the Bill, made a clear
commitment to do. I think there is a perception by those outside
that because Ministers and their senior officials may see the
figures in advance it gives them the opportunity to present their
arguments in a way that is more helpful for Government. I think
that is a major plank in all of this as well. As you probably
know, the Government has just gone out to consultation on that
very issue.
Q12 Graham Stringer: Have you read
the transcript of the Public Accounts Committee session with your
predecessor, Mr Cook, about the last census?
Ms Dunnell: Yes—not very
recently.
Q13 Graham Stringer: What have you
learned from that, because Mr Cook, following that session, became
a fairly major national figure—and I certainly did not think
it was fit for purpose and most people in Manchester did not think
it was fit for purpose. What have you learnt not just from the
transcript of that but from that whole sorry episode about the
last census?
Ms Dunnell: We have learnt an
enormous amount from the last census, which, as we know, had a
few failings. Of course, we are now deeply into planning for the
next census in 2011. The key things that we learnt about what
went wrong with the last census, or one of the things anyway,
was that we did not have a proper up-to-date and fully comprehensive
address register. In Manchester, for example, some streets were
entirely missing because the address register that we were using
had been checked something up to two years before, so one of the
lessons is that we must create a single address register from
the existing address registers, and we must do that ourselves
and have it up-to-date as close to census date as possible. That
was a very important one. The other was that actually the response
to the census was very good: 94% of households returned their
forms. Then the statistical challenge of course is to do the follow-up
survey and to use that information to estimate the remaining part
of the population. What happened last time was that some areas,
the difficult areas—mainly inner-city areas—had a
much lower response than that—because that was the average.
This time, we are having a different approach to collecting the
information, which means we are going to focus the enumerators,
that is the people on the ground, in the difficult places, and
have a much more immediate management system that tells us where
we are not getting a response so that we can move people in to
the more difficult areas. Those are the key things that we are
planning to do differently, which will have a big impact on the
quality of the census.
Q14 Graham Stringer: Statistics are
collected by departments in a decentralised way. Is there any
way that you think you can improve the communication and the quality
of those statistics collected in a decentralised way into a centralised
set of statistics?
Ms Dunnell: As I said, all the
1,200 or so people who work in the Government Statistics Service
all do work to the code of practice, and so that is part of this
continuous process of ensuring that what they do adheres to that;
and of course 27 of the Government departments and agencies have
somebody whom we call the statistics head of profession, and they
meet regularly with me and my senior colleagues and work on things
like education, training, career development, standards and all
of that type of thing. That is all in place.
Mr Hughes: One of the major planks
once again of the Act is this new concept of an assessment, where
the UK Statistics Authority will be developing work programmes
to look across the whole of the official statistics to make sure
they are fit for purpose. It will be the major element of their
responsibility to ensure quality of the statistics not only in
the ONS but across the GSS as a whole.
Q15 Dr Gibson: Are there some good
departments and some bad departments so far as statistics are
concerned? I am not going to name them, but there must be a differentiation
between professionalism—or do you think they are all good?
Ms Dunnell: I do not think we
would be able to say which were good and which were bad. There
is a variation in every department, probably including ONS.
Mr Hughes: Given the huge number
of statistics produced across Government, there will always be
the odd slip-up; and Andrew Dilnot's book exemplified some of
those; but every one of those heads of profession who is accountable
to Karen will be seeking to ensure that standards are met. I have
been in the GSS thirty years, and I have never been able to see
that sort of distinction.
Q16 Dr Gibson: Oh, dear—it
is all good—okay, fine!
Mr Hughes: I did not say it was
perfect.
Q17 Mr Boswell: Do you make use of
reality checks with statisticians outside, either in overseas
administrations, which might, as it were, come and inspect you,
or indeed in the private sector where private sector companies
also need to—
Mr Hughes: We have very, very
strong links with academe and we have methodology committees where
we have a wide sprinkling of academics in different areas of statistics
so that we engage very closely with them on our methodologies
and techniques. That is one area. We also work in a very strong
international network and we are continually looking at best practice
there. To pick up your specific point, as part of the European
statistical system, all of the statistical systems of the EU have
just gone through a major peer-review process where we contributed
in assessing some of their offices and they in turn assessed us.
There is that continual check and balance in the system. We, if
you do not mind me saying so, did very well in that process.
Dr Gibson: People can be proud of their
work occasionally!
Q18 Dr Turner: Karen, you, as National
Statistician—and I quote you—said: "It is my
responsibility to ensure we deliver statistics that are of high
quality and integrity", which is obviously a very worthy
aim, and we would expect nothing less; but you have just quoted
one example where the statistics were not of the highest integrity.
It was not necessarily the fault of the statisticians, but of
the actual integrity of the data collection process itself in
the last national census. I am fairly confident that is not the
only area of statistics that gets published where there are question
marks about the quality of the data collection. What steps do
you have in mind to try to ensure that you can improve on the
quality of data collection?
Ms Dunnell: Data collection is
very important. What we have to remember about the census that
makes it different from many of the other statistics that we produce
is that we only do it every ten years. This is something that
we are thinking about very actively for 2011, because, as we know,
society changes an enormous amount in a ten-year period, and each
subsequent ten-year period seems to change even faster. Getting
the arrangements on the ground absolutely perfect for a situation
that has moved on considerably is quite challenging. That is one
of the particular issues of the census, which is why this time—and
in answer to your question this is another of the lessons we learned—we
will work much, much more closely with local authorities in the
planning process, to use the information and intelligence they
have about their areas and how they have changed in order to make
sure that the practical arrangements are as good as they can be.
That does not mean that at the end of the day something that involves
getting quite a complex form back from every single household
in the UK is not a risky business. It would be really silly to
say that you can put your hand on your heart at any point and
say "this is going to work 100%"! It is the only statistical
activity in social statistics where people have by law to fill
the form in, which is a big advantage. Even so, there are many
risks to the process. Those are the kinds of things we try to
manage.
Q19 Dr Turner: The definitions are
another problem because you often find the press comparing sets
of statistics, but they are not actually like for like because
there are subtle differences in the collection bases. That then
helps undermine public confidence in statistics or certainly in
what the Government uses those statistics for. Do you have quality
control mechanisms to review your statistical processes before
they are published?
Ms Dunnell: For many of them we
do. The census is a very good example of what you have just mentioned
about changing classification. When you do something ten years
apart you have a real conflict between doing it exactly the same
so that you get a very good measure of change, which is often
at small area and local authority level and constituency level,
which is what people are interested in; but on the other hand
you have to make sure that the statistics are relevant for the
time on which they are collected. For example, in 2001 we revised
the ethnicity classification, which had been used for the first
time in 1991, because the whole nature of the ethnic groups living
in the UK had changed quite considerably in that ten-year period,
so we changed it. This time we are trying very hard to keep it
the same because we have had different kinds of changes. We have
had a lot of migration from the EU; therefore, for the first time
we are asking questions about citizenship and country of birth,
which will help identify those people from Europe and other parts
of the world that come into the white ethnicity category. So you
have all the time to balance keeping things the same so you can
measure change over time and reflect what is on the ground. That
is one example. I can think of other things.
Mr Hughes: As part of the National
Statistics, when it was introduced in 2000, one of the planks
was a quality review process. All the Government departments were
obliged to go through this process of statistics, and something
like 75 to 80 major reviews have been undertaken in that time
period. Many of those reviews have included external representatives—academics
and users—to peer-review the statistics produced.
| 
