Science and Technology CommitteeWritten evidence submitted by Professor Leslie Mayhew (Census 39)

The Census and Social Science

This memorandum is intended as a contribution to the inquiry into the impact of ending the Census on social science research. I consider the issues to be extremely important to the future of social science and set out in this note my views on certain key aspects, particularly the use of alternatives. I would also be very happy to provide oral evidence if invited to attend the hearing.

I am Professor of Statistics at Cass Business School and Director of Mayhew Harper Associates Ltd. I have been working with local administrative data in an academic and private capacity for a number of years both as a user and also as an advisor to local authorities and healthcare providers.

As an academic I have published extensively on demography, health and social care, social security, pensions etc. My use of data has spanned both Census and administrative data and so I am able to compare many different sources and also application areas. I am also a current member of the Office for National Statistics (ONS) National Population Projections Expert Advisory Panel.

I spent much of my early career in the civil service where I was a regular user of different sources of data (Census, surveys, administrative systems) both as professional analyst and then as a senior civil servant in the DHSS and then DSS. I also spent five years as a director in the Central Statistical Office (then part of HM Treasury) and subsequently in the Office for National Statistics itself.

Since leaving the civil service in 1998 to become an academic, I have accumulated 13 years of experience working with administrative data in local authority and PCT environment. My breadth of experience therefore covers national and local administrative data collection systems and surveys of different kinds.

I am a past user but also a strong critic of the Census and of the methodology used. I therefore consider myself to be uniquely positioned to express views particularly on questions 3 and 4 in the inquiry:

What alternatives to the Census would provide population and socio-demographic data of equivalent or higher quality?

What other existing sources of population and socio-demographic data could be improved upon?

Two papers published in 2011 by myself and with my colleague Gill Harper are highly relevant to this debate and are recommended reading for the Committee:

1.“Applications of Population Counts Based on Administrative Data at Local Level”.

  G Harper and L Mayhew (2011). Jnl. Appl. Spatial Analysis DOI 10.1007/s12061-011-9062-z.1

2.“Using Administrative Data to Count Local Populations” .G. Harper and L. Mayhew (2011). Jnl. Appl. Spatial Analysis DOI 10.1007/s12061-011-9063-y.2

A key feature of our approach is that we create comprehensive population data bases by combining different local data sets. These data sets are used by local providers of health, local authority and other services. Applications of the data base are often cross-cutting in policy or research terms and enable useful linkages between sectors of the local economy eg education, health and social care.

The data base produced may be contrasted with information available from the Census:

The results are timelier than the Census which can take several years to process before being released and be more than 12 years out of date before it is refreshed. Based on the 2011 Census basic population aggregates will not be released before July 2012 and more detailed data will not follow in some cases until much later.

On Census day this year we obtained and processed snapshots of local administrative data covering the six Olympic Boroughs in London who commissioned the work. Fully geo-referenced databases down to household level by age and sex were completed and handed over to local authorities inside six months.

The databases contain much greater granularity than is possible using Census data. Each database is able to produce statistics for any size or shape of geographical area or administrative unit and unlike the Census is not constrained by any pre-determined geographical boundaries.

The data can also be flexed by age so that any group of any age width can be considered. This is important for applications in education such as forecasting school places and in health and social welfare for which eligibility or qualifying ages often do not fit easily into standard age groups adopted by the Census.

Census data cannot be linked to information on users of different services but our data can. That makes it more valuable to service providers but also to academic users for example to measure inequalities, unmet need, inefficient provision or coverage of services.

Users believe that our data are more accurate than ONS Mid-Year Estimates (MYEs) especially in areas of London with high population influxes and turnover. For example, in the six borough project, we found that our figures were on average 8.3% higher than the ONS 2010 MYEs but these gaps may be smaller outside London. There are no comprehensive independent comparators but our data is very close to the Child Benefit caseload whereas ONS’s for the six boroughs were not (12.5% under count).

The core variables we produce are population by age and sex, household type and size, benefit and tenure status, all by geographical location. Ethnicity data are available for children attending state schools but the same information does cover other age groups in most other administrative sources. To fill this gap, we impute ethnicity of individuals and households using probabilistic methods from our extensive data base of names.

By taking different snapshots of administrative data, it is possible to analyse churn and turnover at local levels which is currently a hot topic for research. For specific applications, for example in health, social care, and education, other administrative data sets or variables of interest may be linked together eg attendance at hospital, educational attainment.

The scope of the database is therefore potentially extensive and much wider than the Census can offer. Projects can span healthcare, public health education, social care, youth crime, housing and so on. On the other hand, variables covered by the Census but not adequately covered by locally available administrative sources include: Information about occupation and place of work, self reported health, caring responsibilities and religion.

In our own work, we have been able to fill these gaps to some extent using data from local surveys and linking them to administrative data. A similar approach combining survey information with administrative data could be considered at national level, possibly by using or adapting existing national surveys.

Under our method data are processed in a secure environment by a third party (in this case ourselves). No data are shared between data owners and the final data base is anonymised for statistical purposes only ie no individuals are identifiable. Our approach is cleared by local Caldicott Guardians and by the Information Commissioner.

The cost and timeliness of our approach has much to commend to the Committee. Expanding the system to cover the country would cost roughly one tenth or less of the present cost of the Census, results could be available with six months and the exercise repeated at more regular intervals such as annually. This would benefit both local users and academics whose research would instantly become more accurate, relevant and therefore useful.

There is scope to reduce costs further by introducing standard legally binding data sharing protocols and generally reducing bureaucratic impediments in order to make the process more automated. Better addressing and general conformance to national database BS standards etc would also help linking efficiency and reduce the need for any manual processing.

I hope this brief analysis is helpful to the Committee. In the time that we have been working on this approach, we have observed steady and substantial improvements to the range and quality of administrative data. In conclusion, I would stress that our approach is practical and there is continual scope for improvement as I hope this memorandum has demonstrated.

The main risks to the system would be if any if the local data sets used were scrapped or there were legislative changes that prevented their use. The government should consider therefore either securing their future access or replacing them in due course with an alternative that does the same or better job.

Professor Leslie Mayhe
Cass Business School, Mayhew Harper Associates Ltd.

November 2011