Examination of Witnesses (Questions 40-59)
MR KEITH
DUGMORE, PROFESSOR
DAVID RHIND
AND PROFESSOR
DAVID MARTIN
16 JANUARY 2008
Q40 Nick Ainger: While the ONS tell
us that they want to increase the remuneration of these enumerators,
hopefully to retain them after their training, even if they do
and they have 45 to 50,000 enumerators, is that a sufficient number
to deal with the problems which the evidence seems to indicate
have increased in relation to houses in multiple occupation and
so on? Should we not be indicating to the ONS that perhaps they
ought to look at this issue again, because 45 to 50,000 enumerators
may not be enough to ensure not only that the information comes
back but the quality of that information? I notice that in your
submission comment is made on the seminar that was held following
2001 that the questionnaires that had an involvement with an enumerator
were of a higher quality than those that were just posted back.
Could you comment on that?
Professor Rhind: I think there
is no doubt that the quality of enumerators and the presence of
enumerators does help. That is absolutely clear. This is, as always,
a trade-off between costs and quality. As I understand it, the
likely cost of the next census is something of the order of £500
million. The ONS, or, rather, the new Statistics Board, has been
given a budget of, I think, £1.2 billion over some multi-year
period and as a consequence the Census is going to be a very substantial
proportion of their total expenditure; so clearly there are some
trade-offs to be made in all of that and in one sense quality
is part of the trade-off. There is no perfect answer. I think
the real question is: how good do we want it to be and how good
can we afford this to be?
Q41 Nick Ainger: But the irony, it
is not an irony; perhaps it is a basic fact. John Thurso has just
left, but the point is that in Scotland they are not going to
be cutting back on the number of enumerators, are they?
Mr Dugmore: A very important issue
here is that in the 2001 Census the forms were delivered by enumerators
and then people were encouraged to post back. The proposal for
2011 by ONS is to post out, and that, I think, particularly in
urban areas, is a very risky thing in that, even if we have the
perfect address register, there may be multiple households living
at that address. I would say that in the more difficult areas,
one certainly needs to have enumerators delivering the forms,
and I believe that that is going to continue in Scotland with
the traditional practice, and I think it would be potentially
a very dangerous move to rely on post-out of Census forms in 2011.
Professor Martin: That is very
much a difference between the Scottish and the proposed England
and Wales model. In a sense what ONS is proposing to do is a rational
response to the set of constraints which they have, which is to
recognise that in many areas you do get a good response rate from
the post back and, therefore, the enumeration effort in those
areas becomes very light in the 2011 model. You move to a much
more flexible system where there are more enumerators who are
more mobile to deal with those more difficult areas, but it is
a very fine judgment which ONS is having to make against the budget,
and the likelihood of retaining a certain number of enumerators
at a certain wage as to where is the balance point, which is the
right number of enumerators who can do address checking beforehand
and stay right through the process and work in teams to follow
up the difficult areas, is a very tight call, but it is a very
different model to the one where Scotland are effectively doing
the same as 2001, so none of these questions are thrown into stark
relief. It would be true to say that Scotland does not have the
number of difficult areas to the same extent which ONS is facing
in England and Wales.
Q42 Nick Ainger: But, in your judgment
and experience, are 45 to 50,000 enumerators sufficient to provide
a decent and high quality Census in 2011?
Professor Martin: My view would
be that the number of enumerators is capable of delivering, but
I think the weakness is the address list. I think the address
list underpins most of these issues, because if those forms do
not go to the right places and then there is a big fall-out of
people saying, "We have not got a form", or we do not
get a response and it has to be followed up, then that is actually
where our biggest difficulty lies. With a good address list and
some resolution of the addressing issues, it might work, but I
think that is a big risk of the current model because we still
do not have confidence in those starting out lists which are going
to be where the forms are mailed to.
Mr Dugmore: I would agree with
that entirely.
Professor Rhind: I have no reason
to disagree.
Q43 Peter Viggers: A number of people
are interested in an accurate address register, including the
Electoral Commission, and the body responsible for the Electoral
Commission in the House of Commons is the Speaker's Committee,
for whom I answer in the House of Commons and I declare a special
interest in this area. Is there read-across between the Electoral
Commission's bid to government that there should be individual
registration of voters? Currently there is registration by way
of household, which the Electoral Commission thinks is not an
accurate system for finding the correct number of voters or for
distributing information. Is there read-across, is there discussion
with the Electoral Commission about individual registration in
relation to the address register?
Professor Rhind: I know of no
such discussions, Sir. It would be, in one sense, a different
population, because you would want all of the population, not
just those over voting age, but I know of no such discussion.
Q44 Peter Viggers: I might need to
institute it. Professor Martin, your study group said that all
group members articulate a strong desire for outputs to be delivered
as fast as possible in relation to the 2011 Census and in the
most flexible possible formats, and, Professor Rhind, similarly,
the Statistics Commission report says the time appears right for
users with a UK-wide interest to become actively engaged in helping
the Census Office implement their policy. In relation to the 2011
Census, are you satisfied with the dialogue that is taking place
and the input from prospective interested bodies?
Professor Martin: I think it is
accurate to say that the ONS are just at the beginning of the
serious dialogue about the outputs which will come from the Census,
and that is a process formally instituted in April of this year,
so I think it is very early to judge that. One of the constant
refrains from across the sectors of census users is that it is
possible to focus entirely on the issues we have just been talking
about and recognise that actually the usability of the resulting
statistics depends very much on the format in which they are published
and the means of delivery and the timeliness, and actually you
can get focused very much on the input issues. So we are, certainly
from my constituencies, optimistic that the right dialogue is
about to begin but it is something which is only really being
instituted in the year we are about to start on. The census planning
processes do not begin with those issues, and they will be things
we will be looking to press very hard on in the coming year.
Q45 Peter Viggers: Are you satisfied
that you have the mechanisms to promote this dialogue separately?
Professor Rhind: I think we still
have some way to go, Sir. Can I give you two examples? The Statistics
Commission has carried out a review of usability of official statistics,
and what is extremely interesting in there is that it is not just
about: "Does this data fit the needs that I have got?",
but it is where the data is made available, how it is described,
whether it is easy to use and so on, that are the crucial elements.
So, I do not think a consultation, important as it is, on the
basis of what data should we be collecting is enough. It is much
wider than that to influence how useful the outcome is for users.
The other thing, I think, is to say that different users have
some different needs. I think it has been true in the past that
much of this has been driven by the needs of central government
departments, for understandable reasons. Essentially it is government
money, in some cases extra money put in by individual government
departments, but I think the new Statistics Act makes it clear
that official statistics are for the public good and for a wider
audience than simply central government. I am not convinced that
we have yet got to the stage where the right forms of consultation
are actually in place. I know that Keith Dugmore, apart from his
day job, is also the Chairman of the Statistics Users Forum, so,
with your permission, Sir, he may have something useful to say
about that.
Mr Dugmore: Yes. Thank you, David.
I think, if we go back to the likely topics of the Census, there
is a bit of a disconnect in that there was consultation 18 months
ago on what users preferred, and a list of preferences came out,
but that has been fairly radically reworked in recent months,
we believe under pressure from central government departments,
and so there is a feeling that the Census content is particularly
still driven by central government departments rather than the
public good more generally. I would also like to pick up the point
that was made about working right down to the end as to producing
results in a form that is easily useable by many people. An illustration
of this takes us back to the need for UK-wide statistics and people
saying, "I want to compare 100 variables across the whole
of the UK and can I just get one file from somewhere, rather than
having to bolt things together in different formats?" So,
it is fairly end of the day, low-grade, technical stuff, but it
is vital as to whether the use of the data is multiplied tenfold.
Q46 Peter Viggers: Indeed, the three
census offices did not achieve a single method of disclosure in
2001 and the ONS has told us that the Registrars General have
agreed a common statement on disclosure control with a view to
a single method of disclosure control in 2011. How important is
a single method of disclosure control? Is that the key point?
Professor Martin: It is very important.
The degree of user confusion which results from doing different
things to ostensibly comparable data such that they are not comparable
once we have collected them and published them is highly problematic,
particularly to organisations who have got a cross-UK remit, and
I think that is one of the biggest issues. This is not something
which is so much a major issue for any local government or health
authority who have an essentially regional, district level remit,
but for organisations who are trying to understand the pattern
of change across the whole of the UK—it would be interesting
to see how Wales and Scotland are diverging or converging in different
characteristics with England—then it is critical because
it makes the usability of the data different if they are very,
very difficult to be certain that the results that you get are
not actually not artefact of the methods which have been applied;
so I would say this is a critical issue.
Q47 Peter Viggers: Do you sense that
central government and other bodies involved are doing enough
to ensure that there is a consistent set of standards across the
three bodies?
Professor Martin: All we have
at the moment is a headline statement of an agreement to work
together, which was greatly welcomed, but at the moment we do
not really know very much about what that will look like in terms
of actual methods to be applied. The difficulty we faced in 2001
was after quite a constructive user consultation process, a series
of late changes which led to those discrepancies, and if we look
at the experience from 2001, it was those late changes, those
divergences, which caused the greatest difficulty for users.
Q48 Mr Love: Can I turn to mid-year
population estimates. To what extent is that art or science?
Professor Rhind: Yes! Sorry; that
is not meant to be frivolous. It is clearly some elements of both.
There are lots of judgment elements which go into the methodologies,
and to some degree the situation is enhanced if you have local
experience and you know that some things are better descriptors
in some areas than others. This not a handle-turning exercise,
it is an exercise where you need a good scientific methodology,
but you also have to have awareness of what is going on elsewhere
and make real judgments, as statisticians have to often make.
Mr Dugmore: I think that ONS is
faced with a difficult situation here. One looks back at mid-year
estimates growing through the 1990s and then the comparison with
the 2001 estimates when the Census came out, and to some extent
they are in a vulnerable situation because it is difficult to
get the numbers right. I suppose, for me, it points to the merits
of having absolute counts, whether they be from a census or whether
they be from administrative systems, the actual detail that the
thing has been measured, because as soon as you enter the world
of estimation it is less desirable really; so it is a difficult
job.
Q49 Mr Love: You mentioned the importance
of local knowledge. Of course there have been many criticisms.
Earlier on Westminster and Manchester were mentioned and, indeed,
we had a submission from the London Borough of Newham (and I will
quote from their submission), which says "Trends that are
arising from the mid-year estimate in populations are quite questionable
and undermine confidence in the ONS estimates." How far do
you agree with that? Are they really questionable?
Professor Rhind: There have been,
of course, a number of examples recently—Slough is the most
recent one that comes to mind—where the mid-year estimates
have been questioned and, indeed, census figures have been questioned.
After the 2001 Census the Commission argued very strongly in a
report for 2003 that there was huge merit in much closer inter-working
between local authorities, who often knew where it was difficult
to survey areas and knew a great deal about those areas, and the
Office for National Statistics and their equivalents elsewhere.
There was some concern about that on the part of the then National
Statistician, who took the view that all of this was a matter
for the independent entity, which was ONS, and that local authorities
had a vested interest in finding ways to raise their population
counts. I think that view is less prevalent, and certainly there
has been some closer inter-working with local authorities now.
Whether that is adequate, I do not think I can judge.
Q50 Mr Love: Newham also provided
figures that showed that in 2006 they had an increase of 919 in
the number of dwellings in their area, yet the ONS figure showed
a loss of 1,500 people. That seems counter-intuitive, if I may
say so. It is not impossible, but it seems unlikely. Let me go
on. You mentioned methodology. Clearly, the methodology has to
be robust if you are to resist what is considered to be local
authorities trying to up their figures to increase their funding.
Are you confident of that methodology? Perhaps I can ask Professor
Martin.
Professor Martin: I think I would
have to say, no, not in all areas. The biggest issue is the one
which will have become clear through this discussion, which is
that the areas which are subject to greatest change are the ones
in which we have the poorest intelligence. Most mid-year estimates
for most local authorities are probably fairly good, and we have
no real reason for feeling that the methodology used or the core
information has gone far astray. Those areas in which you do have
a truly dynamic population, where you do have lots of housing
redevelopment, you have lots of in-migration, which may not be
apparent in any of our registration systems, then we do not have
a source which gives us a way of either estimating it precisely
or, indeed, of validating those estimates, and so often what the
local authority believes or sees on the ground may, indeed, be
closest to the truth, but to mirror that through a uniformly applied
methodology is extremely difficult. The problem is that the thing
we want to count is the thing which is most uncertain in those
areas.
Q51 Mr Love: Let me press you on
that, because you mentioned Camden earlier on and the study that
was carried out there, and I could mention my own area, which
has seen enormous demographic change and the speed of replacement
of people is intensifying. I am in an inner London constituency.
What steps can we take to try and address this problem of fast-moving
populations, because it is a phenomenon, in my view, frankly,
that is here to stay—it is not going to go away—so
how do we measure it more accurately?
Professor Martin: My starting
point would be back to our discussion about using those administrative
records in much more intelligent ways. They will not capture every
move of all categories of people, but, for example, things like
the National Health Service Register, which is based on GP registrations,
had lots of work done on it over the last few years, is probably
as near as we have got as a starting point to a population register,
as in the one which gets closest to what you might think of as
the whole population, and it reflects in imperfect ways a lot
of internal migration. Again, our difficulty is that the people
who move most, the people that they are most wanting to trace,
are often those least likely to register, but I have not much
doubt that the solution lies in integrating those administrative
sources, because they are the only ones which are likely to move
fast enough to capture the kinds of process we are interested
in.
Professor Rhind: Can I add to
that? No one administrative data source is going to be good enough
to do all of this. They all have imperfections. Young people,
young migrants tend not to register on the GP list until there
is something terribly wrong with them, so it really does require
multiple administrative lists, integrated and checked out and
triangulated in that particular way.
Q52 Mr Love: If you think there is
inflation in local authorities trying to measure population, GP
lists are even worse because GPs get paid on less, so they have
got a very direct incentive?
Mr Dugmore: I just wanted to make
the further point that the mid-year estimates, of course, are
of the resident population of people here for 12 months or more,
and a lot of commercial organisations, water authorities, and
so on, are concerned by the fact that there are a lot of short-term
migrants as well who appear underneath the radar, who are not
part of the resident population, and yet there may be many thousands,
tens of thousands in particular areas, and so there is a definitional
issue here of who is here at any one time and one does not want
to overlook those who are here for less than 12 months.
Q53 Mr Love: Can income on to the
French question? Of course, the criticism that is made when Manchester
or Westminster or Slough come out and say the figures are inaccurate,
we need to validate these across the whole of the country. We
cannot just take individual instances. What is the best way of
making the revisions to the mid-year estimates that will satisfy
ONS but will also satisfy clearly the great unhappiness there
is in many local authorities that they have been short-changed
in this process?
Professor Rhind: I think we would
accept that the problems and the scale of the differences between
real and measured populations, if there are two different things,
varies across the country. It is a very different sort of matter
in rural Northumberland to measure population than it is in some
other areas, and there are other complications like services bases
where the population is very mobile and moves in and out very
quickly; students, of course, are a difficulty in all of that;
so the nature of the problem varies a bit. The only way out of
this, I think, is to do a series of case studies on a properly
constructed basis in something of a representative sample of the
areas where we think we are going to have problems, and to an
extent the ONS has done that with the Camden studies and with
various others. The details of those my colleagues may be more
familiar with.
Q54 Mr Love: When answering that,
Professor Martin, can you say whether that you think that is an
area that we ought to think about in terms of our recommendations?
Professor Martin: Yes, I do. I
think it is a very constructive route forward, but it has to recognise
this issue that different types of places have different problems
in producing answers. The point I was going to make really relates
to the fact that when we put these different lists that we have,
such as an address list, together, for example, with information
that might come from the Health Service and information that might
come from other sources, we have to recognise that we need to
be more sophisticated in terms of the definition we are using:
because we tend, even in this discussion, to talk about the resident
population, but we know that, in fact, people live complex lives
in which they are often resident for part of the week in different
households and at different addresses, and there is a tendency
to want to claim those people for different purposes, which is
not to say that that is an inappropriate thing to do, we have
to recognise that we need more than one working definition which
may sum to more than what appears to be the total number of people
we have got, and that is actually a more sophisticated answer
for that: so there is a local authority who is trying to see the
number of properties they have got and home owners who are going
to agree what that is and does not preclude us from recognising
some of those people may be present for only part of the time.
A lot of difficulty we have in sorting out the mess after some
authorities' difficulties in 2001, comes down to that difference
of definition where people are not always present but they are
partially present at an address and they may have another address
as well. That causes us quite a lot of difficulty because our
current frameworks do not accommodate that to any great degree
at all.
Q55 Mr Love: That would certainly
be a major phenomenon in Westminster, one assumes! Can I take
you on finally to data sources. You have already mentioned GP
lists, and indeed there are other Health Service lists. In your
own submission you talked about survey data. I could tell you
that my own local authority, the London Borough of Enfield, carried
out an independent survey and the survey as I came in used the
electoral roll, used council tax records and used school children's
records to come up with alternative figures. Which ones of those
alternative data sources are valid? Can we put together a list
that could be agreed across the board in a way of trying to reach
a compromise figure that was more accurate locally?
Professor Rhind: I think it is
perfectly possible to create a good practice guide. The problem
of saying that you should always use this one is that there are
geographical variations in the quality of some of those things.
I think a good practice guide of those data sources you should
consider, and how you should seek to take them together, would
be a perfectly sensible and possible thing to do.
Professor Martin: Certainly, as
Professor Rhind said earlier on, the solution lies in the integration
of those sources. There is no one of them which gives us the right
answer because the biases in them are systematically different,
so they will under-report and over-report different categories
of people, because of the nature of the way the lists are collected.
There is this issue about tension between the administrative use
of any kind of register and its statistical use, when we want
to step back and shave off different biases. Yes, any trial towards
integration of administrative registers is going to have to do
exactly that, possibly via some case studies, and it is going
to involve best practice. The example I would point to is that
back in the early 1990s when we tried to work out who had been
missed in the 1991 Census—and it was quite an extensive
project which Professor Diamond, Chief Executive of the ESRC,
was involved in—we produced a new set of national corrections,
if you like, to the national estimates, and it used local sources.
It was a methodology for using local sources with different emphases
according to the nature of the local authority you are dealing
with. I think the only solution that we are likely to come to
is one which accommodates that, albeit updated to the contemporary
data sources.
Professor Rhind: With the critical
element, I think, that the judgment about what the best possible
figure is has to come from the National Statistician or ONS.
Q56 Mr Love: I accept that.
Mr Dugmore Might I just make a
related point, picking up on something that David said earlier.
At the time of the 2011 Census, I think it will be vital that
as many of these administrative files as possible are captured
for analysis to compare with the Census. That will give us a good
dummy run on how pupil level records, national insurance numbers,
pension records and so on compare with the Census results. Whilst
David said that he thought 2011 should be the last Census, I would
rather see another one in 2016 when the registers were up and
running and you could compare the two and make sure they were
telling the same story.
Q57 Jim Cousins: Professor Rhind,
you were of the view in 2006 that our migration statistics urgently
needed to be improved, but there was little sign of improvement,
and they certainly could not be used as the basis of resource
allocation. Is that your view now?
Professor Rhind: I think the Statistics
Commission has been pointing out since 2003 that we thought the
migration statistics were sufficiently imperfect that they were
having a pretty serious effect in some areas. In 2006, I wrote
to various ministers and set out what we thought the consequences
of this were, and of course since then there has been a pan-departmental
or multi-departmental study of what might be taken forward. I
think Ministers have agreed publicly that they accept that the
migration statistics are not good enough. I have not seen any
acceptance of what we will actually do about all of this to make
the situation better. I think it is absolutely clear that any
solution, following what we have said before, will have to be
a multi-departmental one, because the sources of information can
only come from various different departments, whether it is DWP
or the Home Office and so on. If I can be brief at the end of
this long statement, I do not think we have moved enormously far
since the Statistics Commission made those comments.
Q58 Jim Cousins: Is the difficulty
we have here that we have a hard-to-count group of people and
we cannot count them, or is it that our definitions are wrong
and we are not even trying to count them? Is it a "cannot
count" problem or is it a "do not count" problem?
Professor Rhind: First of all,
there are multiple sources of difficult-to-measure people. I think
there are also cases, as David Martin has already indicated, where
the definitions are such that we take one snapshot, whereas with
different uses we actually need multiple snapshots of the data,
so I am afraid I think it is both of those things.
Q59 Jim Cousins: For example, there
has been a lot of controversy about the European accession countries
and people coming to live and work in the UK, but apparently we
do not count them as residents of the UK until they have been
here 12 months.
Professor Rhind: The international
definition of a migrant, which is the basis which I think all
countries use for their standard migration statistics, is resident
for a year or intention to be here for a year. I should say of
course that the ONS have produced some experimental statistics
for short-term migrants quite recently, October or thereabouts,
but that is still at an experimental stage.
|
