You are here: Parliament home page > Parliamentary business > Publications and Records > Committee Publications > All Select Committee Publications > Commons Select Committees > Innovation, Universities and Skills > Innovation, Universities and Skills

CORRECTED TRANSCRIPT OF ORAL EVIDENCE To be published as HC 443-i

House of COMMONS

MINUTES OF EVIDENCE

TAKEN BEFORE

INNOVATION, universities, SCIENCE AND SKILLS COMMITTEE

 

 

THE USE OF government STATISTICS

IN evidence-BASED POLICY-MAKING

 

 

 

WEDNESday 19 MARCH 2008

MS KAREN DUNNELL and MR MIKE HUGHES

 

Evidence heard in Public Questions 1- 67

 

USE OF THE TRANSCRIPT

1.

This is an uncorrected transcript of evidence taken in public and reported to the House. The transcript has been placed on the internet on the authority of the Committee, and copies have been made available by the Vote Office for the use of Members and others.

 

2.

The transcript is an approved formal record of these proceedings. It will be printed in due course.

 

Oral Evidence

Taken before the Innovation, Universities, Science and Skills Committee

on Wednesday 19 March 2008

Members present

Mr Tim Boswell

Mr Ian Cawsey

Dr Evan Harris

Dr Brian Iddon

Graham Stringer

Dr Desmond Turner

 

In the absence of the Chairman, Dr Ian Gibson was called to the Chair

________________

Witnesses: Ms Karen Dunnell, National Statistician, and Mr Mike Hughes, Director of the National Statistics and Policy Group, Office for National Statistics, gave evidence.

Q1 Dr Gibson: Thank you for coming and helping us in what we hope will be an interesting inquiry about Government statistics and policy-making. We are very aware that this change is taking place in structures and we do not want to get into that, but we would like some advice from you to carry it forward. I will ask you the first question, and then other Members of the Committee will pile in. We want to try and get it over in an hour. Tell us about the different types of statistics that you record, the national statistics and official statistics and the differentiation between them. What are the differences, and are they important?

Ms Dunnell: The Government and its agencies of course produce a very wide range of statistics; probably about half of these are badged as national statistics; and we refer to the whole lot as official statistics. National statistics are characterised by the kind of quality badge that represents the fact that they adhere to the statistics produced under proper professional mechanisms, and adhere to the code of practice, which means they are fit for purpose, they are relevant, they are high quality, they are accessible in terms of being on the website and so on, and they are produced with integrity - a whole series of things like that. The Government's statistical surveys work to a very clear code of practice. They were badged in 2000, when the last reform of government statistics took place; and at that time all statistics produced by ONS, which represents about a fifth of all national statistics, were going to be national statistics, and the rest were decided by Ministers in departments. Since then, some extra statistics have come into the fold, if you like; and basically that is something that Ministers advise on; and then I and my colleagues rubber-stamp. We would do an assessment and decide -----

Q2 Dr Gibson: Do you know which class of statistics you are handling?

Ms Dunnell: Yes.

Q3 Dr Gibson: How is that decision made? You say Ministers, but statistics are statistics are statistics, are they not?

Ms Dunnell: Yes, but this is one of the very interesting things about statistics; that in the UK, in 1999, when the last reforms were decided upon and consulted on, and indeed in the debate that has taken place more recently in Parliament, everyone decided that this should remain in the gift of Ministers, so it is a parliamentary decision, the latest one.

Q4 Dr Gibson: Could you hazard a guess why Ministers want to keep control of this?

Ms Dunnell: Well, I could hazard a guess about it, yes. It is partly of course that many of the statistics that are produced, which we refer to as official statistics, are those figures which emanate from administrative and management systems, for example in the health service, the prison service or the police service, and they are used very intensively for management and administrative purposes and may not ever in some senses fulfil the very strict codes of practice that we work to, particularly on things like publication processes.

Q5 Dr Gibson: Do you think the words "statistically proven" are just used in a loose kind of way by politicians? I assume if the Prime Minister will say to me "statistics have proven that we are not as poor as we used to be" and all this kind of stuff - the word "statistically" gets thrown into the hat somewhere! Is that your feeling?

Ms Dunnell: It may do, but on the other hand we do produce a large number of series, the whole intention being to demonstrate how our economy and society are developing; so in many cases, for example child poverty, we do have in the Government statistical survey methods of working out whether it has gone up or down. That is entirely statistically good practice.

Q6 Dr Gibson: Do you feel you are ever compromised in different positions and different committees you are on? Is that a possibility; or can you sit back and be quite neutral about everything?

Ms Dunnell: Yes, my job, partly as a statistician, and as a senior civil servant, is to be impartial and to give advice based on the statistics that I am there to talk about.

Q7 Dr Iddon: Who decides the boundaries between national and official statistics, and are there statistics that are produced as official statistics that you feel ought to be national?

Ms Dunnell: Yes, there probably are some of those. I think that the new UK Statistics Authority is planning to take a pretty systematic look at the whole of official statistics and give some advice about whether the boundaries are right.

Q8 Mr Boswell: Will that require a re-definition of the criteria, to see what they are; or at least a fresh look at that?

Ms Dunnell: One of the things that the UK SA will do is review the existing code of practice, which we have had in place for more than ten years now; and then they will apply those new criteria, which I do not suppose will be very different because there is an international set of codes of practice for statistics, which are pretty similar across the world.

Mr Hughes: One of the things that the Act did was to give the Statistics Authority the powers to oversee the whole of the UK official statistics system, rather than focusing just on national statistics. It gives the Authority the power to raise questions with Ministers in Parliament about their treatment of official statistics and whether they should be national statistics.

Q9 Mr Boswell: Will that be a public dialogue do you anticipate; or will it be a matter of private advice to Ministers who, if their official statistics are controlling them rather than -----

Mr Hughes: I think the answer to that is still not clear. The Authority is currently looking at its policy on transparency, and will seek to be as open and transparent as possible; but in the context of dealing with Ministers, you will have to respect that dialogue.

Q10 Graham Stringer: You have said that you want to improve the public perception of statistics, and that the SA will help you to do that. How will you do that? Is that just an aspiration? Can you put some flesh on the bones to convince us that you have a real plan that will improve?

Ms Dunnell: It is aspirational but of course we believe it is very important that this work we do, which is extensively used across Government, is much better regarded by the public. The UK SA will have plans, and indeed we always have things that we are trying to do to improve trust. I think the main one will be what the UK SA has already decided, which is to set up a web hub, a single place on the Internet, where all official statistics will be published together. That will be very separate, and seen to be very separate from the ministerial statements about those statistics. The intention is to for that to come into place on 1 April. That will make, hopefully, quite a big difference. The other things are down to quite a lot of things that I can try to do in my position as National Statistician, which is to raise the standards of what we do across the Government's Statistical Service, particularly in terms of things like making the statistics we produce understandable, interesting, and used in a relevant way, because the more people who can understand what we do and use them in their work or, for example, at local level in helping to get the kind of things they want at local level, then gradually we will build up respect and regard for these very important statistics. Those are the kinds of things we try to do.

Q11 Graham Stringer: Do you think that will be sufficient when less than a fifth of people currently believe official statistics? Do you think your website and making your work technically superior will change people's perception? Is there reliability in statistics?

Ms Dunnell: That is the intention. I agree that it is probably quite a long, slow process; but the real secret is widening the community of people who understand statistics and what they are useful for and get some benefit out of using them. That is really the challenge.

Mr Hughes: One of the other major planks with trying to improve trust - and it is very much a perception problem - is reducing both the time and the number of people who would see the statistics in advance. That is something that the Government, during the passage of the Bill, made a clear commitment to do. I think there is a perception by those outside that because Ministers and their senior officials may see the figures in advance it gives them the opportunity to present their arguments in a way that is more helpful for Government. I think that is a major plank in all of this as well. As you probably know, the Government has just gone out to consultation on that very issue.

Q12 Graham Stringer: Have you read the transcript of the Public Accounts Committee session with your predecessor, Mr Cook, about the last census?

Ms Dunnell: Yes - not very recently.

Q13 Graham Stringer: What have you learned from that, because Mr Cook, following that session, became a fairly major national figure - and I certainly did not think it was fit for purpose and most people in Manchester did not think it was fit for purpose. What have you learnt not just from the transcript of that but from that whole sorry episode about the last census?

Ms Dunnell: We have learnt an enormous amount from the last census, which, as we know, had a few failings. Of course, we are now deeply into planning for the next census in 2011. The key things that we learnt about what went wrong with the last census, or one of the things anyway, was that we did not have a proper up-to-date and fully comprehensive address register. In Manchester, for example, some streets were entirely missing because the address register that we were using had been checked something up to two years before, so one of the lessons is that we must create a single address register from the existing address registers, and we must do that ourselves and have it up-to-date as close to census date as possible. That was a very important one. The other was that actually the response to the census was very good: 94 per cent of households returned their forms. Then the statistical challenge of course is to do the follow-up survey and to use that information to estimate the remaining part of the population. What happened last time was that some areas, the difficult areas - mainly inner‑city areas - had a much lower response than that - because that was the average. This time, we are having a different approach to collecting the information, which means we are going to focus the enumerators, that is the people on the ground, in the difficult places, and have a much more immediate management system that tells us where we are not getting a response so that we can move people in to the more difficult areas. Those are the key things that we are planning to do differently, which will have a big impact on the quality of the census.

Q14 Graham Stringer: Statistics are collected by departments in a decentralised way. Is there any way that you think you can improve the communication and the quality of those statistics collected in a decentralised way into a centralised set of statistics?

Ms Dunnell: As I said, all the 1,200 or so people who work in the Government Statistics Service all do work to the code of practice, and so that is part of this continuous process of ensuring that what they do adheres to that; and of course 27 of the Government departments and agencies have somebody whom we call the statistics head of profession, and they meet regularly with me and my senior colleagues and work on things like education, training, career development, standards and all of that type of thing. That is all in place.

Mr Hughes: One of the major planks once again of the Act is this new concept of an assessment, where the UK Statistics Authority will be developing work programmes to look across the whole of the official statistics to make sure they are fit for purpose. It will be the major element of their responsibility to ensure quality of the statistics not only in the ONS but across the GSS as a whole.

Q15 Dr Gibson: Are there some good departments and some bad departments so far as statistics are concerned? I am not going to name them, but there must be a differentiation between professionalism - or do you think they are all good?

Ms Dunnell: I do not think we would be able to say which were good and which were bad. There is a variation in every department, probably including ONS.

Mr Hughes: Given the huge number of statistics produced across Government, there will always be the odd slip-up; and Andrew Dilnot's book exemplified some of those; but every one of those heads of profession who is accountable to Karen will be seeking to ensure that standards are met. I have been in the GSS thirty years, and I have never been able to see that sort of distinction.

Q16 Dr Gibson: Oh, dear - it is all good - okay, fine!

Mr Hughes: I did not say it was perfect.

Q17 Mr Boswell: Do you make use of reality checks with statisticians outside, either in overseas administrations, which might, as it were, come and inspect you, or indeed in the private sector where private sector companies also need to -----

Mr Hughes: We have very, very strong links with academe and we have methodology committees where we have a wide sprinkling of academics in different areas of statistics so that we engage very closely with them on our methodologies and techniques. That is one area. We also work in a very strong international network and we are continually looking at best practice there. To pick up your specific point, as part of the European statistical system, all of the statistical systems of the EU have just gone through a major peer-review process where we contributed in assessing some of their offices and they in turn assessed us. There is that continual check and balance in the system. We, if you do not mind me saying so, did very well in that process.

Dr Gibson: People can be proud of their work occasionally!

Q18 Dr Turner: Karen, you, as National Statistician - and I quote you - said: "It is my responsibility to ensure we deliver statistics that are of high quality and integrity", which is obviously a very worthy aim, and we would expect nothing less; but you have just quoted one example where the statistics were not of the highest integrity. It was not necessarily the fault of the statisticians, but of the actual integrity of the data collection process itself in the last national census. I am fairly confident that is not the only area of statistics that gets published where there are question marks about the quality of the data collection. What steps do you have in mind to try to ensure that you can improve on the quality of data collection?

Ms Dunnell: Data collection is very important. What we have to remember about the census that makes it different from many of the other statistics that we produce is that we only do it every ten years. This is something that we are thinking about very actively for 2011, because, as we know, society changes an enormous amount in a ten-year period, and each subsequent ten-year period seems to change even faster. Getting the arrangements on the ground absolutely perfect for a situation that has moved on considerably is quite challenging. That is one of the particular issues of the census, which is why this time - and in answer to your question this is another of the lessons we learned - we will work much, much more closely with local authorities in the planning process, to use the information and intelligence they have about their areas and how they have changed in order to make sure that the practical arrangements are as good as they can be. That does not mean that at the end of the day something that involves getting quite a complex form back from every single household in the UK is not a risky business. It would be really silly to say that you can put your hand on your heart at any point and say "this is going to work 100 per cent"! It is the only statistical activity in social statistics where people have by law to fill the form in, which is a big advantage. Even so, there are many risks to the process. Those are the kinds of things we try to manage.

Q19 Dr Turner: The definitions are another problem because you often find the press comparing sets of statistics, but they are not actually like for like because there are subtle differences in the collection bases. That then helps undermine public confidence in statistics or certainly in what the Government uses those statistics for. Do you have quality control mechanisms to review your statistical processes before they are published?

Ms Dunnell: For many of them we do. The census is a very good example of what you have just mentioned about changing classification. When you do something ten years apart you have a real conflict between doing it exactly the same so that you get a very good measure of change, which is often at small area and local authority level and constituency level, which is what people are interested in; but on the other hand you have to make sure that the statistics are relevant for the time on which they are collected. For example, in 2001 we revised the ethnicity classification, which had been used for the first time in 1991, because the whole nature of the ethnic groups living in the UK had changed quite considerably in that ten-year period, so we changed it. This time we are trying very hard to keep it the same because we have had different kinds of changes. We have had a lot of migration from the EU; therefore, for the first time we are asking questions about citizenship and country of birth, which will help identify those people from Europe and other parts of the world that come into the white ethnicity category. So you have all the time to balance keeping things the same so you can measure change over time and reflect what is on the ground. That is one example. I can think of other things.

Mr Hughes: As part of the National Statistics, when it was introduced in 2000, one of the planks was a quality review process. All the Government departments were obliged to go through this process of statistics, and something like 75 to 80 major reviews have been undertaken in that time period. Many of those reviews have included external representatives - academics and users - to peer-review the statistics produced.

Q20 Dr Turner: To what extent does the UK Statistical Service sample surveys rather than conducting national registers, as most countries do - national databases?

Ms Dunnell: For some purposes we need registers, and we have registers. For example, we have in ONS, and maintain for use across Government, a register of businesses in the UK, which we use to produce statistics and also of course to use as a sampling frame for all the business surveys that feed into the national accounts and balance of payments and things like that. Very often, because one of the great advances in the 20th century was the development of sampling, the statistical theories about sampling, it is much more cost-effective to carry out a sample, take the sample and collect the information from them, and then make an estimate. You could then say quite precisely within which boundaries your estimate is likely to fall. For many, many purposes, this is what we do, because it is cost-effective and quicker, and it is often higher quality because you are collecting the information for a statistical purpose, whereas with quite a lot of registers and administrative registers, for example, the information is collected for administrative purposes and may not be of sufficient quality for the kind of statistics that we produce.

Q21 Dr Turner: Where do consultants fit in to the government's statistical work; to what extent do you employ them; and, when you do employ them, who determines the methodology and reviews their work?

Ms Dunnell: We do not use consultants very much for the actual statistical work. Where we do use consultants is for quite a lot of the IT side of things. We are often trying to develop things when we have not got the right skills. On the other hand, we do have contracts with academics quite often to do things like peer reviews, as Mike has mentioned, but also to carry out a particular type of analysis or to help us carry out development work. For example, we are working at the moment on planning a new major survey of disability in the community, and we are using an academic on a contract to help us develop that, partly because he used to work for us and has a lot of expertise in this field, but partly because we do not have that particular skill ourselves. Also, of course, we work extensively with organisations like ESRC and will often put out to contract pieces of work, for example, in relation to the census, when we usually get a university contract in collaboration with the ESRC to provide services to academic users on the census. Similarly, we have a contract with Southampton University to run an MSc, which many people in the GSS go on. So there are a wide variety of those things. We also use contracts for some of our data collection on occasion.

Q22 Dr Turner: It occurs to me that you obviously must make fairly heavy use of IT.

Ms Dunnell: Yes.

Q23 Dr Turner: You know, of course, the disastrous history of governments of all colours in promoting large national IT schemes. Do you get involved by the Government, or do you have any input to Government considerations of large-scale IT schemes, because they must be relevant to you as presumably you would want to access information from them from time to time; so are you consulted?

Ms Dunnell: Yes. In fact, every Government department has something called a chief information officer, which is a bit of a misnomer, but basically head of IT, as I call them. The head of the whole Government - there is a head of profession, rather like myself - who works in the Cabinet Office, and he runs a council of all the heads across Government, in the same way that I run my head of profession department meetings and so on for statistics; so we are very plugged in to that. In fact, one of the things that the CIO council has done is to create a shared service for basic IT infrastructure in Government departments, and we were one of the pilots for that; and indeed just signed up to it. Our basic infrastructure is actually moving to this service called Flex at the end of this month. We do get very involved in that. It is for providing all our basic telephony and equipment and storage and security and so on. The thing that we keep very much to ourselves, and will continue to do, is all the work we do on statistical computing, which involves this worldwide sharing of good practice and expertise and development about how to best take that forward. On things like data access, we are very plugged in, and I am a member of Gus O'Donnell's security advisory team, which is advising the whole of government - and Mike is my substitute on it - about how to do much, much better with data security. Of course, we have all had to do audits of our security arrangements and so on. Given that keeping personal data confidential is absolutely at the fundamental core of our business, we are quite active in advising and helping to get the standards across Government right.

Q24 Dr Turner: You have not lost any disks?

Ms Dunnell: We have not, no.

Dr Gibson: How would they know?

Q25 Mr Boswell: While we are thinking about IT, presumably this means that you or collectors working for you make increasing use of direct inputting of electronic information on site. Perhaps you can comment on that, and, if so, does that lead to improvements in accuracy as well as timeliness, and also on the security side is that equally satisfactory?

Ms Dunnell: Doing data collection using electronic methods has many, many advantages, one of which is that as you enter the data you can do all kinds of checks on it and the signposting to the next question is all automatic; so you tend to get much better quality data. We have very, very strict security arrangements for the transferring of data into the office. One of the reasons why we are going quite slowly with Internet collection is because of the security side of things; we have to be absolutely sure that data coming down the line cannot be interfered with or looked at in any way.

Q26 Mr Cawsey: I would like to ask about how Government policy should be more evidence-based, and the role that you can play in that. You are probably aware of Michael Blastland and Andrew Dilnot's piece on the tiger that is not, where they argue that the tendency to ignore the need for statistical verification is only now beginning to change in the UK. Do you agree that this verification has not until now played a part in the formulation of Government policy?

Ms Dunnell: I do not necessarily agree with them, no, because the Government's statistical service produces an enormous range of statistics, which are used across the piece every day in Government departments, local authorities and all kinds of bodies for planning, evaluation and all kinds of parts of the policy process. I am a great admirer of Andrew Dilnot, I have to say, and he has picked out some examples from a very, very wide range of statistics to make some very important points. I think that some of the stuff he has written in there about variability and so on will get everybody to read it because it is very, very important. However, I think it is very, very obvious now that statistics are absolutely vital for government policy at all stages. We see many examples, and most recently in the budget speech in the work that has just come from the prime Minister's Delivery Unit, looking at the future of policy. That is a statistical picture about what is going on in society and the economy and is at the forefront of thinking about policy development. This does not mean, however, that this is perfect in every case and in every department, because a lot of policy-making is highly political and often happens at great speed. Our role, and one of my ambitions, is to make sure that we can improve the impact and influence of statistics and statisticians in all sorts of Government policy-making at both national and local level.

Q27 Mr Cawsey: What I am interested in is a bit chicken-and-egg really. Is it that the Minister or the department will say to you, "We have got this whiz of an idea for a policy, so give us all the statistics to back it up"; or do you go with a load of statistics and say, "This policy needs to be changed"?

Ms Dunnell: Do you want to try that one, Mike!

Mr Hughes: I think the syndrome that you outline probably did prevail fifteen or twenty years ago, but, as Karen has said, we try extremely hard to get involved in the policy process as early as possible.

Q28 Mr Cawsey: Do you ever create it, is what I am asking; do you ever start the process and say, "Look at these statistics; we are highlighting ...."

Ms Dunnell: Yes.

Mr Hughes: Yes. When I was head of profession in transport, we were producing a range of statistics for the board and actually chose to challenge some of the things that were happening on transport policy at the time. Is it right to be spending so much money on rail safety when probably there are five people killed a year, and yet eighty people a day die on the roads? It is that kind of dichotomy.

Q29 Graham Stringer: It did not change the policy, did it?

Mr Hughes: No, it did not, I would agree with you; but at the end of the day we were putting those points up to the top of the office.

Q30 Mr Boswell: I am often saying privately that I do not think social trends arise and hit us in the face until it is too late. Are you really in the forefront, in your advice to Ministers, in being able to spot social trends that previously perhaps had not been anticipated, and then actually drawing their attention to it hopefully getting them to do something about it before it is too late?

Ms Dunnell: I think it is a very important part of our role. For example, something that the ONS is responsible for on behalf of all other Government departments is the monitoring and estimating the size and structure of population. For example, the aging of the population is a very important phenomenon, which needs to be taken into account in every bit of policy that happens both nationally and locally. I think that actually statisticians have done a great deal to make sure that everybody does not forget about the aging of the population, because it is very, very important. That is a very good example. Similarly, on issues like the situation with family status, the whole evolving, monitoring system for looking at, for example, the big changes that we have had in marriage and cohabitation patterns came very much from the statisticians attempting to measure a rapidly changing situation, and then making those new statistics part of what everybody now accepts as something that we update all the time. There are similar things on the economy, which is much more difficult in some ways because it changes quite rapidly; but again we feel very strongly that it is up to us to draw attention to things that are happening in the economy, which people may not be putting enough emphasis on.

Q31 Dr Gibson: Do you have an input into the growth percentages of the economy?

Ms Dunnell: Yes - well, we produce all that every quarter, yes - all of the national accounts, yes.

Q32 Mr Cawsey: If a statistician finds a piece of information and passes it on to the Government and then finds it is ignored, what can they do to correct that?

Ms Dunnell: That is very similar to the thing that Mike said: if you are a statistician in a department, or in my case in ONS, and you feel that policy is not making enough or taking enough account of some change or something which you think is quite important, then it is one of our professional responsibilities to keep putting that piece of information in place.

Q33 Mr Cawsey: But if you keep putting it into a red box, but it never gets any further, would you put it into a red to eventually?

Ms Dunnell: We try to get all of our statistics, to be honest, into every kind of newspaper - that goes back to my answers to some of your earlier questions: in my view, the more that everybody in our community understands and appreciates statistics and their usefulness, the better. We would not do a special press release, particularly aimed at any particular newspaper; but we would take every opportunity, within our normal publication process, to draw attention to those changes that we feel are important.

Q34 Mr Cawsey: What if it was the other way round? What if a journalist approached you and said: "The Government has just announced this, this and this; what is the statistical evidence for it?" and you knew there was none, would you say that there is none?

Ms Dunnell: I am trying to think of an example. We give all kinds of advice to people all the time over the telephone. Personally I do not think - and I do not think most statisticians in the Government's Statistical Service, would make a judgment like that because it would be quite dangerous. What they would try to do is say, "Here is the statistical picture of something as we see it" and present the thing afresh.

Q35 Dr Harris: Dangerous to health, dangerous to career - what do you mean by "dangerous" - to give an honest, open view?

Ms Dunnell: Because we try not to comment particularly, and one of the statistician's basic codes of practice is not to comment about things political. Sometimes, if you have made a comment about a particular policy, it can be interpreted as political. Our task is to be objective, full of integrity, and apolitical.

Q36 Graham Stringer: Can we come back to the point about the large amount of money that the previous Deputy Prime Minister wanted to spend on transportation warning systems all over the rail system. It was very high profile. John Prescott was very keen on it. How, as a real example, did you deal with the information you had which said, "spend this money on road improvement schemes or whatever and it would save more lives"? How did you deal with that?

Mr Hughes: It is going back quite a way now, so I have to think about it more in the abstract than remembering the detail.

Graham Stringer: I think the detail would be much more fun.

Q37 Dr Gibson: Could you send us the detail?

Mr Hughes: The way that we were dealing with these sorts of things was to put together a regular briefing note, a package of material called "transport trends" that would go out regularly to the management board for them to look at. In that particular case the decision was very much swayed by political imperatives and that was why it ended up the way it did.

Q38 Graham Stringer: I think what Ian was asking about - in that situation where the statistics pointed in a different policy direction, would you have gone to the Sun or the Mirror and said: "Here are the statistics; look them up yourself; this policy will cost lives"?

Mr Hughes: We are civil servants; we work for Ministers in departments, and that is the process by which we work. I do not think it would comply with the kinds of principles that Karen has just outlined that any of us would do that sort of thing. That information is not buried in the depths of departments. That is analysis that anybody outside can equally do.

Dr Gibson: You must forgive us if we are slightly jaundiced about getting information! We spend a lot of our time trying to get information from people in Government departments.

Q39 Dr Iddon: I want to move on to targets. Do you have any role in the formulation and measurement of targets the Government sets?

Ms Dunnell: Again, it is quite varied between departments whether or not statisticians get involved. One of the things that we are trying quite hard to do now is to ensure that statisticians do get involved and that they get involved at the early stages of the formulation of targets. One of the particular problems of course is that people set targets that are quite difficult to measure, and then quite a lot of short-term and sometimes not very well thought-through things happen in order to measure the progress of targets, and these are the kinds of things where, if statisticians do get involved early on, they can be avoided. It is very important.

Q40 Dr Gibson: Do you do CO2 and greenhouse gas emissions?

Ms Dunnell: We do not in ONS, but the GSS does them, yes.

Q41 Dr Iddon: So things can only get better! Is it possible, using statistics, to measure the quality and output of public services?

Ms Dunnell: Yes, in many ways. As you probably know, we have a unit in ONS that has been set up to try to measure the productivity of public services, partly because the traditional method in the national accounts is to say that output for example of the NHS is equal to the financial input, and a big review was done several years ago which recommended that we should use statistics and research methods much more effectively to look at other ways of doing it. Therefore, a unit has been set up, which has looked quite thoroughly now at education and health and the administration of social security and so on. It is developing this with the departments in question, which also extensively use academic colleagues and help them to devise much better ways of measuring the output of health or education services and the quality of them; but it is quite challenging.

Q42 Dr Iddon: The Statistics Commissioner has suggested that the emphasis on the use of statistics as performance indicators and targets has politicised your professional area, and also suggested that we are trying to push the boundaries too far. What would you say to that?

Ms Dunnell: I certainly do not believe that it has politicised statistics. I think it has led to the development of statistics that may not be national statistics, and the use of statistical information about what is actually happening in public services, which is sometimes quite difficult to interpret - I am not quite sure what your question about boundaries meant, to be honest.

Q43 Dr Iddon: It is about putting statistics beyond their own capabilities.

Ms Dunnell: Yes, I think that there possibly are some examples of that. I think much more likely, however, is that administrative systems are set up that produce answers about things, and they are not set up in a proper scientific and statistical way. We would prefer to get involved in the process and set up proper systems to produce statistics that everybody can trust and that are based on proper methodology and codes of practice and so on.

Q44 Mr Boswell: I think we are familiar with the fact that targets, or statistics are used as proxies for an outcome that inevitably has happened.

Ms Dunnell: Yes.

Q45 Mr Boswell: Do you have any evidence that when a particular statistic or variables are also targets as proxies for some kind of quality measurement of public service, they get abused? I am thinking rather of the Goodhart's law about a monetary aggregate: once it becomes the objective breaking down because people gain it. Is there any element of this?

Ms Dunnell: I think that is part of the danger with it. For example, if you take hospital waiting lists, you can see from a political point of view that it is something that the population truly understands, because it is one of the things that everybody who goes into hospital experiences and it has a great meaning for people. I do think that it is one of those targets where it is possible for people to work extra hard in one sense or another in order to meet a target, and that of course may be the right way to propel different organisational units into higher levels of activity; but it may not be. That is why it is very, very important from a national statistics point of view to make sure that these statistics are produced properly - as Mike was saying, form some kind of portfolio of statistics about what is going on in hospitals so that people can make a wider judgment about it.

Q46 Dr Harris: You just said that you thought hospital waiting list statistics were widely understood because everyone experienced waiting lists.

Ms Dunnell: I am sorry, I did not mean to imply that waiting list statistics were widely understood, but I think that the notion of being on a waiting list is widely understood. It has a lot of political meaning.

Q47 Dr Harris: If you take someone on a waiting list, what relevance is it to how long they wait how many other people are waiting? They can wait one day and there could be 2 million people also waiting one day. That would be 2 million people waiting for one day, which would show a fantastically high-capacity, excellent service. One hundred thousand people waiting for five years - much lower numbers - a bit of a disaster! The waiting list numbers coming down was what the target was - or what about maximum waiting time? What relevance is a maximum waiting time to a non-urgent operation if I have an urgent cardiac complaint?

Ms Dunnell: I am sorry, I did not really mean to get into a big debate about it; I am just using it as an illustration that some targets - I am trying to think of one at the moment -----

Q48 Dr Harris: You are not going to find waiting list ones, are you?

Ms Dunnell: They do not have meaning to the population to whom they apply.

Q49 Dr Harris: A waiting list is actually probably the worst thing you could think about for an individual patient experience.

Ms Dunnell: I would probably entirely agree with you. I was just using it as an example of something which at least the public understands. If you have a target - for example, the Treasury will have targets probably about the balance of payments, but not too many people will understand what the figure means or have any understanding. That was the point that I was trying to illustrate.

Dr Harris: I will come back to this.

Q50 Dr Iddon: In 2003 the Royal Statistical Society concluded that performance monitoring in public services was poorly conducted, and it called for a number of changes including reporting of measures of uncertainty and of random sampling. Have any changes been made following that report in 2003 and are any projected to happen in the future?

Mr Hughes: It is not only the RSS that made those sorts of comments; the NAO was saying very similar things, and also the Statistical Commission looked at this. There is now a much stronger engagement of analysts in these sorts of processes. It is a political decision as to what the target should be. Quite often those targets have been in areas where traditionally statistics have not been collected before, so there has been a fairly large learning process about how you can compile statistics from administrative sources. Our traditional mechanisms have been surveys and things of that sort. There is a far higher level of engagement now by analysts across the piece. It may not always be a statistician that is doing it; it could be a researcher. We belong to a large analytical community in Government where jobs are interchangeable at times. The exemplification for this is that certainly with PSAs in the latest 2007 CSR there is a hope that there will be a senior analyst on the boards looking at those indicators to make sure that they do have data integrity.

Q51 Dr Iddon: Mike, I think the real question is, was that 2003 RSS report relevant to your work and did it help you to see things a little more clearly?

Mr Hughes: I think it did, Dr Iddon, but not just the RSS one; as I say, the Stats Commission's report, and the NAO saying very similar things at the same time.

Q52 Mr Boswell: Can we turn to the user end? I must say that in my limited ministerial career it was useful to have even a rudimentary knowledge of statistics at least once or twice in particular! I know that your official guidelines for measuring statistical quality talk about providing the user with sufficient information to judge whether or not the data are of sufficient quality for their intended use. That is something that you might like to expand on briefly, but are most civil servants, and indeed dare I say Ministers, statistically literate enough to understand the messages your statistics carry; and do you feel it is important that we should encourage them to do that?

Ms Dunnell: I would have to say that I think statistical literacy generally in the UK - and that applies to civil servants and politicians and most people actually - is very, very low. Nevertheless, our statistics are scrutinised not only by the statistically illiterate, but the statistically literate watch everything we do like hawks, and that is why we have a policy for national statistics where measures of quality such as competence intervals and information about sample sizes, et cetera, is placed on the website and is easily accessible when you are actually using the statistics. We are now trying - we have a goal, without a target, to make sure that when we produce our key statistics we have competence intervals or some measure around them in press releases and in the publications. A huge number of people still disregard competence intervals, I have to say.

Q53 Mr Boswell: I think they do. I think you are in two minds on this: professionally you are bound to be cautious because, like a scientist, you are making assertions which are constrained by competence limits, and so a lot of your users will not necessarily understand what you say, even if you explained it to them; and some of them may have a motive for trying to pick things off the shelf to justify, say, a policy or a target. How are you going to work at getting those messages fitter for purpose into the public debate, and into the decision-making process? How can we make this a less ill-informed dialogue?

Ms Dunnell: One of the ways we try to do it when we publish any series is to put it in the context of the past. For example, every month we produce a report on what is happening in the labour market, and every month we include a graph which goes back over a few years, so that you can interpret what is usually tiny changes in employment rates, for example, and getting a view on a graph over a period of time does help people to interpret the importance of this month's blip, which is going to be basically a blip around a point in time. We do try to do that. We are doing a lot of work trying to improve the graphs we use and the words we use to describe them. It is by constantly trying to do that that, hopefully, people will get used to it.

Q54 Dr Harris: Would you say that one justification for the fact that the Government Ministers get statistics some time earlier than everyone else is that it gives them an interval to understand them, and therefore to not over-interpret them; and that if they do not use that opportunity there is little merit in them having statistics in advance?

Ms Dunnell: I personally believe that we should have any pre-release time very short. The proposal at the moment that is being consulted on is that it is 24 hours. From a purely statistical integrity point of view I would prefer it to be no time at all. It is very much ingrained in our culture, the notion of pre-release, and if we can get the time cut down and the number of people that releases go to cut down, we will make significant progress.

Q55 Dr Harris: I do not blame the Government for this - I would do exactly the same - put a spin on statistics. Is it the case that you use that interval when you are the provider of statistics to say, "This is what can be said; that is the limit of what can be said" and written advice saying, "This is what can be said on the basis of these statistics"? Otherwise, what is the point from your point of view? I know it is not your preference, but given there is this pre-release period how are you using that to ensure that you counter unreasonable spin?

Ms Dunnell: The labour market is quite a good example of that, and we will probably have to adapt it when we get new rules in. Basically, ONS produces the labour market data and has done for ten or so years now, and we produce the monthly thing. Then we have a pre-briefing with the stands and the economists from departments that are interested in the labour market, which is mainly DWP, Treasury, DBERR and so on. They will come to a briefing with our statisticians, so that everybody can discuss the meaning of the latest trend in the context of the trend over the last year or so, so that they all have an opportunity to think about what these latest changes may be. Then those people go back and do their own briefing to their own Minister. That is the thing that everybody finds very, very useful at the moment, but it does open up the possibility (a) for leaks and (b) spin. That is why we need to cut the time down.

Q56 Dr Harris: That was not my question, was it? I was suggesting why do you not use the time to directly talk to the special advisors and the press officers about what you think should or could or could not be said - could and could not be said about statistics? Then if it is ignored at least you have a record for your integrity - you have made a lot about that - so that you can sleep at night; and if it builds up you can say, "I resign if this continues". Why do you not do that?

Ms Dunnell: Because the briefing mechanism we have, in your words, does that, but it is ONS talking to the people in departments who do that.

Q57 Mr Boswell: It cannot be negotiation.

Ms Dunnell: It is not a negotiation. We are trying to get a team of statisticians with a lot of experience of understanding the labour market and policies around the labour market to say, "What do these statistics tell us this month?" Then we get the best advice we can about what the statistics are telling us.

Q58 Dr Harris: So you do not have any interaction with the people who are doing the spinning and the media briefing and -----

Ms Dunnell: Well, I -----

Q59 Dr Harris: ----- Ministers the line to take, you as statisticians I mean?

Ms Dunnell: In my department, yes, I have a relationship with my press office. We have a dialogue about the way that we will present statistics in press releases and so on, and that is exactly the same as the dialogue that people returning from our labour market briefing to their departments will have with their press office.

Q60 Dr Harris: I meant, do departmental statistics people have relationships with the media facing people?

Mr Hughes: Yes. Typically, what would happen is for whatever set of statistics it was, the Minister and a small group of people around the Minister will see the statistical release; so it is not a case of saying, "You cannot say this or you cannot say that" because that is what the public is going to get - that statistical release and those statistics. Thereafter there may be an engagement between the private office or the press office and the statisticians about the meaning of some of those statistics, but I have very rarely seen a situation where the Minister would then seek to countermand or in any way put a different argument to the one the statistics are saying.

Dr Harris: Really!

Q61 Dr Gibson: The media have statisticians attached to them as well; they give them prizes every year. You -----

Ms Dunnell: I have sat on that panel.

Dr Gibson: The Times always wins.

Q62 Dr Harris: You must be talking about different Ministers than I am because they are always over-hyping data, and I would do the same in their position; but the question is not whether that happens or not - of course it happens - it is understandable - the question is what you, the statisticians, are doing to protect your position by saying that you counselled against that particular over interpretation being done?

Mr Hughes: At the end of the day, it is a Minister's prerogative to present which statistics on their policy he wishes to do. All I was saying was that I have never encountered a situation where Ministers have sought in any way to undermine the statistics that have been put into the public domain, which is what I thought was possibly an issue - well, in any way countermand or contradict statistics that have been published.

Q63 Dr Harris: I am talking about interpretation, not contradiction.

Mr Hughes: I am sorry, that was the point I was seeking to make.

Q64 Dr Harris: Finally, if I may, can I deal with this question about these fascinating waiting list issues. Your view was that a patient understands the waiting list targets, but I would like to ask you about this question of targets versus continual performance monitoring. Are you arguing that it is more useful for a patient waiting for a procedure to know whether the hospital has zero or not zero people waiting more than 18 months, or to know what the average waiting time for that procedure is?

Ms Dunnell: I was not trying to make a comment about that at all because that is quite a complex question, and different patients will want to view it in different ways. All I was trying to say was that at least patients understand what a waiting list is. Sometimes we come up with targets that the community cannot relate to.

Q65 Dr Harris: So when they come up with a high-profile thing like a waiting list, do you provide advice to anyone as to what gains might take place? It may come as a surprise to you that when waiting times get political action, suddenly there develops a waiting list to get on the waiting list, and that rather subverts, at least for the patient and the public, the whole point of this.

Ms Dunnell: Yes.

Q66 Dr Harris: Do you say that if you are going to go for that data collection this is what you must guard against happening?

Ms Dunnell: Yes. Part of the statistician role, if you take something like waiting lists, would be to provide advice about which measure of waiting list might be most appropriate. But at the end of the day it will be up to the politicians or the policy-makers to decide; but it is the job of the statisticians to say, "If you use the arithmetic means you will get that kind of answer; if you use the median, you will get that kind of answer; if you use the proportion of people who have to wait more than six months, you will get that kind of answer" That is the kind of thing that a statistician would do; they would lay out those options. Actually, Andrew Dilnot's book has a very good section, I think, on explaining how you can use the same data and turn it into a range of statistics, which gives you a slightly different story. That would be the job of the statistician in the department to set that out quite clearly so that the policy-maker and the Minister understand what difference it makes depending which particular measure he chooses.

Q67 Dr Gibson: So the check would come in politicians being sharp enough to scrutinise as Ministers; in other words it is our job, is what you are saying.

Ms Dunnell: Yes.

Dr Gibson: We have got to get savvy about it. That is a challenge. Thank you very much for coming along. It is a fascinating world you live in. It is extraordinarily important information that you have given us and it will help us make a judgment in our report.