Select Committee on Treasury Minutes of Evidence

Examination of Witnesses (Questions 20-39)



  Q20  Susan Kramer: Picking up on some of those issues, and there is the whole issue of independence, and believability. Many countries already have some form of statistics legislation, to put that structure in place. To what extent do you think the introduction of that kind of legislation would improve confidence in national statistics here in the UK?

  Professor Rhind: We think in the longer term it is a necessary condition for having some form of over-arching legislation. There is quite a lot of legislation at present, but it is piecemeal, for example, the Census of Population Act, the Statistics of Trade Act and so on. What we research or seek is a statistical act which would have some things in common with those in over a hundred other countries, but which updated them because the circumstances have changed somewhat. What we looked for was a greater degree of independence for the National Statistician. What we looked for was a replacement for this Statistics Commission as a statutory commission and reporting to Parliament rather than to Government, and there are a series of other things that we would like to see embedded in that, including, perhaps, access by statisticians to data already held elsewhere in Government so as to minimise the burden on respondents and filling in forms multiple times. There are some quite complex legal constraints in that at the moment.

  Q21  Susan Kramer: What you are saying is what implications are there for the ONS in that?

  Professor Rhind: I would have thought that perhaps—you must ask the National Statistician—but I would have thought wholly beneficial. There is an issue about how the National Statistician can be both the guardian of quality across the whole of British Government in statistical terms, and also head of the management in a major producer organisation and the Conservative Party produced some proposals before the election on how to handle that. Our proposals are a little less radical.

  Mr Alldritt: The proposals for legislation that we published last year in May 2004 took a particular form, after we took extensive legal advice on the way to achieve the goals within the United Kingdom structure of existing legislation. The way that we saw things working was to have an act that created a right for the National Statistician to draft a code of practice which would be approved by a statutory commission and we saw that as the most effective way of enhancing the influence of the National Statistician. It is slightly indirect, but it is very difficult under English law to give a civil servant in one organisation authority across different government departments.

  Q22  Susan Kramer: Okay. So what kinds of resources are implied for the Statistics Commission if you were to go along this path?

  Professor Rhind: My judgment is that actually, if we went along the path recommended in our report, our report was deliberately constructed to be a minimal change mechanism. We took the view that radical change in the British Government to facilitate greater public trust in statistics was unlikely to be successful, so we looked for ways of achieving it by minimal change. Our view is that, with a statutory commission, reporting to Parliament, we would probably need no more resources, or very little more, because the leverage that that gives to a commission is actually somewhat greater than the leverage that we have earned over the last six years.

  Q23  Susan Kramer: Given these proposals that you made, I believe it was in May 2004, on the Statutory Code of Practice of the Statutory Commission, what response have you had from Government, or what timetable do you expect to hear a response from the Government?

  Professor Rhind: We have not received anything formal, except comments to the extent that this was thoughtful and very interesting, and all the rest of it. It has to be considered, in my view, in the Framework for National Statistics, the review which is about to start, I cannot see how some consideration cannot be given to the legislation question if that same work is going to be revised.

  Q24  Mr Love: Earlier this week, you published a report on PSA targets and that was characterised in one of the heavy duty newspapers as suggesting that measures can be too complex, and rely on data that are not timely and of low, or unknown, quality. In some cases, departments have even set targets for figures that do not exist. Is that an accurate assessment of what is in the Report?

  Professor Rhind: It is accurate as far as it goes. We looked, deliberately, bottom-up, as it were, at each one of the 102, 109, 120, whatever target—the number depends upon how they are grouped; some of them are shared by different departments—and what we found were, some were very well specified. For example, a number of the Home Office targets were actually very clearly specified. We saw others where work was still in progress a year and a bit after the agreement of the PSA target, to actually find ways of measuring whether things had been achieved. In addition to that, we saw a number of others where the targets seemed to have been forced into a very quantitative framework where it was very difficult to see how that could be made to work and where, in our view, qualitative targets were actually rather more appropriate. I think some of the Foreign and Commonwealth Office ones where the influence or the control that the FCO has over some of the processes towards that is pretty limited, and we had a feeling that there had been, "Well, we have a process; we must carry it forward; we must have numerical targets" and in some cases we thought those were inappropriate. We have drawn up some generic conclusions across the piece but there is a rich variety of different stories in each one of the individual targets, but the span of them is the one I have just described.

  Q25  Mr Love: I understand you have released a Report in draft form because you are looking for comments on it and you suggest a number of recommendations and pick out in particular that technical notes should be expanded into target protocols and I think that is an important area. You also talk about "a robust cross-departmental planning system". What are the main changes that you think are necessary in order that we can have the sort of trust and confidence in the PSA and the figures that go into the PSA targets and therefore the targets themselves?

  Professor Rhind: Forgive me—I am not going to not answer your question—if I can just put a real dilemma that I think faces people who are setting targets, and that came out in a meeting yesterday. To create good statistics—reliable statistics, if they are not already in existence—is not a five minute job. The Allsop Report that was published in 2004, begun in 2003, made recommendations about regional statistics; to get them to a situation where they are robust, is likely to take until 2010 or thereabouts. Now, what do you say to a minister—"I am sorry, you cannot set any targets because we do not have robust information yet; wait for two governments until we get them." Clearly, you have in some cases to use what is already there. I think what we were looking for was to say, "Let us not pretend that there are ways of measuring some things or let us not get into the situation of using wholly unsuitable data because you need to have something for measuring these PSA targets". Frankly, if there are some targets which are not measurable, let us make it clear that those things will have to be judged on a qualitative basis.

  Mr Alldritt: There is another specific point whichwe mention in the report, which is, in quite a lot of cases, targets are being measured not against regularly published official statistics but management information coming from within systems, within government departments, and we think that where this is happening that is cause enough to require regular publication of the statistics in an open and routine way.

  Q26  Mr Love: Let me come in on that, because you mention there, that is obviously a major part of the recommendations you are making, about the independence of the statistics that are used, and the numbers that are used. You mentioned earlier that a significant amount of national statistics is not produced by the ONS; it is produced within departments and you talked about the National Statistician having greater powers in order to monitor that situation. We have a system, as I understand it, of audit based quality reviews and they are supposed to be carried out on a five-yearly cycle. How is that process getting on, and should that be strengthened?

  Professor Rhind: You are absolutely right in the sense that there should be a five-year cycle across all the domains of statistics. In fact, in the first five years about half of the areas have been looked at, and there are some areas that have not yet been studied. I would hesitate to call it an audit based approach; it has been in many cases a very detailed approach but we think, and we produced a report last week that said that we really wanted to move towards a much more typically audit based approach and selection of areas, not on the basis that we had to do everything every five years, but some assessment of the risk and of the materiality of there being something wrong in that. As I say, this came out last week and we are in discussion with the relevant government departments about how to take that forward.

  Mr Alldritt: I think it is worth saying that in a letter from the Financial Secretary this year, the Financial Secretary recognised that the quality review process had not been entirely successful over the last five years, so there is a common understanding that there needs to be improvement. The recommendations we worked up with the National Audit Office have two elements which are audit. One is that the selection of the statistics to be reviewed should be done on the basis of risk and materiality, looking at what the risks of going wrong in that area would be, rather than just everything every five years and the other is that, in the actual reviews themselves, there should be both an audit of how the statistics have been produced—are you getting what you think you are getting—and a separate process of looking at the fitness for purpose of the statistics, in other words, are they meeting a real need from decision-makers; are they serving a valid purpose? Our proposals are, therefore, quite different in kind to what is happening at the moment.

  Professor Rhind: Central to all of this is, are we getting the statistics that are needed? Are they fit-for-purpose? That presupposes that we have, in general, a much better set of inputs from users about what they need. Now getting users to specify what they need and anticipate that is not straightforward but that seems to us to be a crux of the matter.

  Q27  Mr Love: Let me press you just a little bit on that, because you mentioned the National Audit Office and, if you were to mirror the way the National Audit Office works their reports are always agreed with the department that they are auditing. Would you say that is the way forward, that agreement had to be reached, even though that might be difficult? Secondly, what sanctions should be available to the National Statistician or the Statistics Commission in order not only to get the flow of work running, but also to ensure that we look at the areas of high risk, we look at the areas that you are particularly interested in?

  Professor Rhind: I think it will be obvious to everyone that the National Audit Office carries somewhat more fire power than the Statistics Commission—at least, as yet. In 150 years' time, perhaps it will be different. In terms of how do we get what we believe to be the right way forward brought in, what would we say on that?

  Mr Alldritt: I think it is important to recognise here that we are not proposing external audits that would parallel the National Audit Office approach exactly. We are proposing there should be an external element but that the government department responsible for the statistics with the Office for National Statistics would lead the work. That may be necessary in many cases because you need the expertise of, shall we say, the Department for Education to look at educational statistics in real depth, so I do not think they could be done fully without involving the department. Now the extent of the external involvement in that, is something that is open to discussion. We have said there should be some, but clearly one way forward would be to strengthen that progressively, the external component. It could, on occasion, be the National Audit Office itself that might be involved.

  Q28  Mr Love: Let me as a final question just ask you; I have some confidence in the newspaper that I was quoting earlier on, that that might be relatively accurate and I think the overall report is a relatively accurate summation of what went into your report. Sadly, that is not always the case. We know that statistics are, shall we say, used selectively regularly in our media. Is there anything that we can do in relation to that, and do you think the Statistics Commission has any role whatsoever in trying to counter this?

  Professor Rhind: Absolutely. I think one role that we might have of facilitating relates to an earlier point that was made, which was about the independence. We see it as our very firm belief that interpretation of statistics is something that actually statisticians must do more of, and publish more of, separately from whatever ministers might wish to publish. We believe there are other mechanisms for independent assessment, analysis and publication results, there is that particular element which is very important. Long term, I do believe that it would help to—this is going to sound very idealistic—enhance the education of those who write about statistics. I happen to know that many of the people who go into journalism schools actually come from arts backgrounds, and many have not a great experience in quantitative skills. There are ways of fixing that, but it is not a five minute job, again.

  Q29  Ms Keeble: Thank you very much. Can you say what you actually do to try to counter misrepresentation of statistics by either the media or by politicians?

  Professor Rhind: May I give you an example? Perhaps the most obvious example is our report on the revisions to economic statistics, where there was a great kerfuffle in the press about the revisions to, in particular, the GDP figures and my colleague here, Martin Weale, is an expert on it, so he may well wish to come in. What we did—this was so serious that it was affecting confidence in a central part of economic statistics. We decided we must take a look at all of this. We went out and got the evidence, we found out what happened in other parts of the world, we had a thorough piece of work done on analysing those economic statistics and we published a report that said in this particular case, we think the interpretations in the media, in the main, were inappropriate and incorrect. We have stood up to the media and to government departments and to others when it is appropriate. That had a very interesting effect; I think there has been very little by way of the same strength, or consistency, or avalanche of criticism about those particular statistics since then. I think we pointed out to the government departments concerned a number of lessons that they could learn, for example, the very terminology made some people think that these were something quite different to what they actually were. The fact that it is called "revisions" as opposed to "updates"—updates because new information comes in—really made some people think that this was a process that was, if you like, crudely put, being fiddled. So we have gone back to the government department and said, "You can do some things better, but we have published a report which said, we have studied this particular process over a 10 year period and it has worked pretty well".

  Q30  Ms Keeble: I think you have set out very clearly what the scale of the problem is, because there can be real issues about public confidence, if the public feel that it is thin, that they are being conned. However, in terms of the example that you have just described, you have obviously done some fairly weighty work on it and produced a publication.

  Professor Rhind: And it took some time.

  Q31  Ms Keeble: Yes, I am sure. The problem with journalists—and I speak as a former one, before I came here—is that they can work very selectively and capture the public imagination. How do you set about capturing the public imagination with your statistics? Is it a challenge?

  Professor Rhind: It is. Statistics are not widely regarded as the most fascinating subject on the face of the planet. What we can do, I think, is try and foster—and we have done quite a lot of work in this—good communications by government departments, not written in a way that only a PhD in Statistics can understand but written in plain English, and trying also to make clear why this is important and what it shows is important. A 0.1% change may not be really something significant. So the communications issues are areas in which we believe there is still substantial scope for improvement. I happen to believe that meeting with journalists and other intermediaries on a regular basis is a very good idea because some of the things that happen are because of misunderstandings and poor communication; for example, a new development was that the Office for National Statistics convened a meeting of journalists in the Bank of England, about six or nine months ago, which was the first time it had happened and they described some of their economic statistics and that is not going to fix every problem, but at least those sorts of interactions are tremendously important. At the end of the day, there are only a relatively small number of intermediaries who help shape the public's view of all of this, so it should be possible to—

  Q32  Ms Keeble: You rightly said that the work is very important and that also means looking at not just how you measure the results of government policy but the way that you can determine it and that is key. Looking at, for example, the whole area of social exclusion and poverty and, there, policy formulation relies on reliable statistics—in particular, small scale statistics to select areas if you look at programmes like the New Deal and estate-based programmes. There is real difficulty, is there not, about producing those, both to do with poverty and also to pick up the small-scale areas. What progress are you making on that, and how reliable do you think is the statistical underpinning of the programmes that we have currently got? If I can just ask my supplementary so I get them all off my chest at once, how do you then feed into the small-scale statistics, statistics associated with race which, as we have seen, is explosive when you get it wrong?

  Professor Rhind: As a geographer, you will expect me to be very keen on fine-grained geographical statistics and indeed I am. The prime source of a lot of fine-grained statistics is the Population Census and we are involved in discussions with the Office for National Statistics and the other census-taking agencies about how that will be carried forward in 2011, what the questions will be, and so on. Indeed, they are involved in a very big consultation exercise at the moment. The Neighbourhood Statistics, which are now widely available on the Web, are, I think, one of the unrecognised success stories in British statistics. It came out of a policy need, interestingly enough, social cohesion policy needs and related areas, about poverty and so on. It became clear that we could not make any sensible statements about all of that on a fine-grained geographical, small-area basis and from that the Neighbourhood Statistics have been created. Now they had to be created from lots of different sources of information; so comparability is far from perfect, but it is a triumph in what has been produced so far.

  Mr Alldritt: I did mention that the Home Office told us this morning about work they are doing to produce small area police data, for example, which would be considerably richer than what is available at the moment, but there is a lot of development work going on of that kind.

  Professor Rhind: I am not sure that answers all your questions.

  Q33  Ms Keeble: At the time that there were disturbances in the north of England, one of the criticisms, if you recall, was that while there was neighbourhood information, it was a bit patchy and that one of the factors that it did not manage to capture was ethnicity. Therefore not misallocation, but the decisions on allocation of resources, exacerbated some racial tensions because of all of that I wondered if you would comment on the way that you see the statistical work is making sure that decisions are taken properly on allocation of resources.

  Professor Rhind: Your point is very well made. Well over £80 billion per year is allocated on the basis of those sorts of geographical—

  Q34  Ms Keeble: Eighty billion?

  Professor Rhind: Eighty billion. It is more than that, now, because of the money for the NHS, as well as the money to local authorities, but may I ask my colleague to fill in?

  Mr Alldritt: On the specific point on race and ethnicity, that is an example of an area that, speaking as a statistician, is very difficult to get accurately. You have got to look at the small number of sources that are capable of delivering that. One is the Census, as the Chairman has mentioned, and the other, potentially, benefits data, from the Department for Work and Pensions, if it is coded into that in a reliable way. The way forward on an issue like that is to look at those few systems, the Health Service data to some extent, that are sufficiently comprehensive and rich enough, but there are very few of them.

  Chairman: Now we have got about 10 minutes left, and I am going to stop when the bell goes. Mark Todd.

  Q35  Mr Todd: First, a brief apology; I had to do a Standing Committee for an hour before coming here. One of the things that you were just touching on in the exchange with Sally [Keeble] was over the use of statistics and people's understanding of them. Do you think—because I am reflecting on the conversation we had at the Treasury, where it was fairly evident that civil servants do not naturally turn towards the use of evidence and statistics as support for policy choices in all departments—there is not the need for greater training in the use of statistics within the decision-makers in Government?

  Professor Rhind: I think there is a huge need for greater training, very generally. When I talk to people outside and find that some of them do not understand percentages, I get extremely concerned about how people can interpret things that come back. So I could not possibly comment on whether the same thing applies to our Treasury colleagues, but I think there is a very major issue about mathematical understanding and statistical understanding at all levels of society.

  Q36  Mr Todd: And certainly among politicians! I would have said they are more ignorant than many in this area. You are recruiting commissioners to fill spaces on the Commission. What criteria are you using, because there are two possible perspectives: recruiting highly respected statisticians, or recruiting users of statistics, who can then discriminate on what is useful and meaningful?

  Professor Rhind: The approach the Commission was set up to take, really, and has taken ever since, is that we would not see ourselves as, if you like, world-class statisticians, professional statisticians. We can get that sort of expertise, from wherever we want to, whether it is from universities, whether it is abroad, whether it is the Civil Service, so that if you look down the Commissioners you will find that most of them have run biggish organisations and faced all sorts of organisational challenges and been dealing with evidence and information. That said, we do have some people: my colleague the Chief Executive is a professional statistician; I think Martin Weale is a very well known economist, which has some relevance to statistics; and in Sir Derek Wanless we have someone who started off as a mathematician/statistician. But the general principle is, we have got people who are looking after the user interests and, to that extent, they come from a range of different communities but ones in which they have played a senior role.

  Q37  Mr Todd: Do you think there are groups who are under-represented amongst the Commissioners?

  Professor Rhind: Yes. We do not presently have anyone, I think I am right in saying, who has come from the voluntary sector, or journalism even.

  Q38  Mr Todd: Are you seeking specifically to target those in the recruitment process?

  Professor Rhind: We have put out a rather broad advertisement and on that basis we will be able to pick something that fits with the skills we already have.

  Mr Alldritt: I think it is fair to say we would like somebody with direct experience with one of the devolved administrations.

  Q39  Chairman: May I just pick up another couple of points. You expressed concern about the ONS's modernisation programme last year in a letter to the Financial Secretary. What is your current view? The risks you were worried about then, have they been dealt with?

  Professor Rhind: The first thing to say, Chairman, is that we think the modernisation programme was overdue and was absolutely crucial. There had been substantial under-investment for 20 or 30 years in systems and in some cases they were extremely fragile and very labour-intensive and the advantages of getting something into a common framework where you could inter-relate the data clearly could not be taken. So, no problems about the general principle. I do not think, in some senses, our concern was about the management of that, that is a management matter, but we are concerned to know when this will begin to deliver and what it will deliver, and I think the programme for delivery of the benefits is not quite as clear as, perhaps, it should be. I have read recently that the ONS have reviewed it after Phase 1 and are going to do it on a more incremental basis, rather than the very "big bang" approach that they initially started out. It seems to us that that is one of the factors which will determine the success of ONS and, more widely across government, statistics in the next few years. I think the new National Statistician has recognised that and indeed the Financial Secretary to the Treasury in that letter, I think, says that the modernisation programme remains the first priority of ONS.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2005
Prepared 12 December 2005