Examination of Witnesses (Questions 60
- 79)
TUESDAY 31 OCTOBER 2006
BRITISH LIBRARY,
NATIONAL ARCHIVES
Q60 Mr Hall: I am a reader at The
National Archives and I have been going there for many years.
I do not believe that it is at all dusty. The building is a modern
one and has fantastic technology with access to primary source
documents. You have a world of people out there who get pleasure
in researching family histories, apart from anything else. Mr
Thomas, you mentioned the fact that the censuses from 1841 to
1901 are on line. I know that this is not part of our inquiry,
but what is your view about getting the 1911 census on line now?
Mr Thomas: Thank you for that
question. Government policy is that the census should remain closed
for 100 years, so we are expecting the 1911 census to be released
in 2012.
Q61 Mr Hall: We cannot wait!
Mr Thomas: Application has been
made to the information commissioner to see a particular entry
in the 1911 census and we await his decision within 28 days. That
is as far as I am prepared to go.
Q62 Mr Hall: That is probably fair
enough given that that is not what we are inquiring about. I have
been looking at my grandfather's military career in World War
I. Because of the microfilm records available at the National
Archives I have been able to establish that he was Private Thomas
McBride PW443 of the 18th Battalion, Middlesex Regiment. The microfilm
is not very good and some of the original documents—receipts
for medals from World War I—have now been stored and sealed
for a period of time in the hope that future technology will allow
us to get better images. Is that correct?
Mr Thomas: Absolutely. The original
World I records of service are in a very poor condition and only
about 25% or 35% of the original survive, so you are quite lucky
to find those records.
Q63 Mr Hall: I was delighted.
Mr Thomas: We hope in future to
be able to digitise the images and make them available in better
quality. The microfilming was done with a lottery grant and it
was the best technology available at the time, but I certainly
take your point of view. We hope that the technology will get
better and we can produce better quality images in the future.
Q64 Mr Hall: Has digital technology
increased or decreased the security of the archives in terms of
the process of digitising the images and preserving the original
documents?
Mr Thomas: I would not say that
it has made a huge difference in terms of security. We have been
microfilming records since the 1960s so we have always had surrogate
copies of the most popular records. We have done that for conservation
reasons. If we had not microfilmed the records of soldiers from
World War I they just would not exist any more, if people saw
the originals. What digital technology has done is to make stuff
available all over the world. If one is in Australia, South Africa,
Belize or whatever and one wants a census record one can get it
straight away on line. That has been a big revolution and also
a huge conservation boon because it means that we can protect
the original records.
Q65 Mr Hall: You are able to protect
the original records without any deterioration. The images are
in digital form and they are universally available?
Mr Thomas: Absolutely.
Q66 Mr Hall: What do you do about
back-ups?
Mr Thomas: At the moment we make
back-up tapes and store them at a secure offsite location. That
is okay for the moment, but what we need is a long-term preservation
programme for the back-up of the digital images. That is something
on which we hope our colleagues in the digital preservation area
will start to work this year. We have invested all this money
in digitisation, but as the technology changes we want to be able
to migrate those images to the latest generation of technology
so they are still readable.
Q67 Helen Southworth: We are talking
about ease of access. Can you give us an indicator of the sort
of change that the digital revolution has made in terms of accessing?
I have experienced the difference between having to get the time
and resources to get to London to access records and what happens
today. I know that Mr Hall has accessed these records because
he told me about it when walking down the corridor. The world
has changed from the requirement to put aside three days and spend
a couple of hundred pounds to be able to look at the records to
the need to put aside just a quarter of an hour to access records
via a computer, but is that just for us? Who is accessing them?
How widespread is this change?
Mr Thomas: The change has been
huge. I can write to you and give you the exact figures, but there
are more people accessing our website and resources on line than
come to Kew. The really big change has been in catalogues. Until
a few years ago if one wanted a catalogue at The National Archives
one would have to go to Kew and search through it in a slow and
painstaking way. Now all the catalogues are on line and one can
search through them with a search tool that gives one what one
wants very quickly. Similarly, in an increasing number of local
record offices catalogues have been made available on line. They
are searchable with a common search tool so one can find what
one wants very quickly. That has been a huge change. Now people
come to Kew with pieces of paper and say that they want certain
records that they have identified via the Internet, whereas before
they would come along and say they are interested in a certain
subject and perhaps spend a couple of days desperately trawling
through our typewritten catalogues. We have had some additional
funding to help with cataloguing locally, and that has really
improved the situation for researchers; and another big aspect
is that it is available all over the world.
Q68 Helen Southworth: Has the geography
changed in terms of where people are accessing archives? Have
the demographics changed? Are you opening up things through the
digital revolution, or are the same people able to get more?
Mr Thomas: The indication is that
we are certainly opening up things. I cannot answer that precisely
at the moment but I can provide you with more information later,
if that is acceptable. We have always served a very broad demographic
profile, not just a small group of academic historians. We serve
huge numbers of people who are interested in the history of their
own families. Quite a lot of people who came from overseas to
the United Kingdom are interested in their family histories, and
people are also interested in local history. Therefore, a very
broad spectrum of society uses the archives.
Dr Field: Our experience at the
British Library has been very similar. Certainly I would underline
the point that the ready availability across the world of information
about what is in our collections has been a major transformational
change. We are not quite in the same position as Mr Thomas in
that about two-thirds of our archives are now in the form of online
catalogues. Because of our access strategy, which is a combination
of people being able to visit our reading rooms and also order
material remotely, obviously within the constraints of copyright
law, the fact that we are making those catalogues available in
that way enables them to order items for collection. In many cases
the catalogues themselves have huge research significance. For
example, yesterday the British Library launched the definitive
world catalogue of all publications in English before 1801. That
is now available as a free resource to everyone. In the past this
resource was available only by subscription and it would have
been used largely only by people studying history. Because of
the research potential available through the new generation of
catalogues it will now literally be available to many people who
will never come to the British Library but for whom the catalogue
itself will be important.
Ms Brindley: The catalogue itself
is great but it is also about digitisation of content, because
if one is sitting at home one wants the real thing. In parallel,
we are making enormous efforts to digitise and we are doing that
with grant funding. This Committee made a previous recommendation
that this should not be done at the expense of our core collecting.
Therefore, we fund raise for it and get grants for it. For example,
we are working in partnership with Microsoft. At the moment with
funding from the JISC we are also digitising out-of-copyright
material—because copyright is a separate issue—in
the form of several million pages of newspapers. We have recently
launched about 4,000 hours of sound. This is great research material
but it is also fantastic for school children, education and all
of us who are just curious.
Q69 Helen Southworth: What is the
potential for commercial sponsorship of digitisation? What are
the issues about who then holds copyright of the digitised material?
Dr Field: We have had experience
of both. Some of our most scholarly resources, which will have
limited impact, have been dealt with essentially as commercial
initiatives where the market is largely perceived to be a few
scholarly research libraries around the world. Others have been
done on a completely open access model, which obviously is very
attractive to us if we can get the funding not simply to do digitising
but to provide the long-term sustainability of that resource.
What we are now doing as an interesting experiment in terms of
an online newspaper resource, which Ms Brindley indicated, where
we have a significant challenge—there are 750 million pages
of newspapers in the British Library's collections, of which about
three million will be available by next summer—is to try
to develop a hybrid business model which at least enables a good
degree of free public access within the UK to higher and further
education institutions, schools and public libraries. Therefore,
it is not access necessarily to everybody, but we believe that
the scale of investment required needs some innovation in terms
of business and service models.
Ms Brindley: We are also very
cautious about our public position in not giving exclusive arrangements
or time-limited arrangements. This has been very interesting in
our negotiations with Microsoft which this year is digitising
100,000 books from the British Library. That will go out freely
on our website, but they will also be able to use that same material
within their products. We have rigorous negotiations to ensure
that long term this is a public asset of which we have stewardship.
It is not just a matter of making it available. One of our biggest
challenges is to build long-term digital preservation into the
structure in the same way as TNA. One can make it available but
one then needs to sustain it and ensure that it is available for
ever. That is a big technological and financial challenge.
Q70 Helen Southworth: Looking at
the national picture, what are the major issues in relation to
archiving in terms of physical storage, conservation, cataloguing
and access? Can you describe to us the more localised situation?
Mr Thomas: Perhaps I may deal
first with the digital side and let Mr Kingsley deal with the
other matters. In terms of digital preservation, The National
Archives is developing a system which will take government records
and hopefully preserve them for ever; it will migrate them to
the next format as technology changes. Unfortunately, there has
not been a similar development in the local archives sector and
there are big concerns about what will happen to digital records
locally. There have been a number of useful pilot projects. The
Paradigm project is being run by the Universities of Oxford and
Manchester. The UK data archives at Essex University has done
a good pilot project with counties in the east of England. However,
in general there are big issues about ensuring the survival of
digital records in local authorities and local private collections.
It is a complex matter and is not just about building a machine
to preserve the stuff; it is about how you capture it in the first
place and select what is worthy of long-term preservation, how
it is catalogued and ultimately how you preserve it. That is one
of the biggest concerns at the moment, but no doubt Mr Kingsley
will tell you about others.
Mr Kingsley: Certainly, the digital
aspect is very important. In addition to what has already been
mentioned, we would flag up the difficulties that the transition
to digital record-keeping poses for collecting archives and establishing
contact at an appropriate point with the bodies whose records
will ultimately be received so as to ensure that the ground work
is laid to enable a smooth transition of records at some point
in the future. On the broader question of areas of concern, we
have recently piloted a self-assessment exercise for local authority
archive services in England and Wales which has generated a very
detailed picture of the pressures under which those services are
operating. The picture varies considerably from place to place.
Many services are doing quite well; others are really struggling.
The matters that tend to cause trouble are, first and foremost,
the accommodation and the limitations that that imposes. They
may not have enough space for their collections, or for them to
grow; they may not be offering an environment that is conducive
to the preservation of that material in the longer term; they
may not have the space to expand the range of activities that
they would like to support in terms of education and outreach
that they could potentially offer. There are also problems in
some places with the documentation of collections, particularly
the proportion of material that they have taken in which is as
yet uncatalogued. Typically, about one quarter of the holdings
of a local authority's archival services are uncatalogued. That
will represent, depending on the size of the institution, a number
of years' work by several tens of persons, so it is not an insignificant
problem. There are also difficulties in some cases with services
that face declining levels of financial support in the face of
local authorities' overall financial situation. In those cases
sometimes difficult decisions have had to be made which have impacted
adversely on the range of services that can be offered. It is
generally true to say that the wider archival sector, particularly
in local government, has not been able to take advantage of the
opportunities provided by the digital revolution as fully as The
National Archives, British Library and some of the private sector
institutions, like the Wellcome Foundation, which also operate
in this field. There are significant challenges out there. One
matter I should like to stress is the extent to which that pattern
varies from place to place. One aspect that emerges most clearly
from our self-assessment survey is that there is, in the hallowed
phrase, a postcode lottery about the quality of archival services
to be found in any given place. Because the typical user does
not access just the records in the institution that serves the
area where he or she happens to live—the research interest
may take that individual to several different record offices in
different parts of the country—that user becomes very aware
of that. Some places are perceived as strong performers and others
as offering a much weaker service, and there is no real justification
for that apart from the financial context in which the providing
local authority is operating.
Q71 Helen Southworth: I want to ask
about the growth in popularity of archives which appears to be
closely related to the growth in media coverage and IT access.
Do you think that that has grown at the expense of preservation
of the collections, or is it a zero sum gain?
Mr Kingsley: Not entirely. Archive
services have managed to attract resources from a range of different
sources, for example Heritage Lottery funds, private sector partners
and other grants, to support programmes to open up archives, whether
it be through digitisation or outreach programmes locally. It
is not a totally zero sum gain. There has been some transfer of
resources in many archive services from behind-the-scenes activities,
as it were—cataloguing and conservation particularly—into
supporting more front end activity, whether it be in the form
of outreach programmes or simply servicing the public who come
through the door. I suggest that it is a combination of the two.
Dr Field: Our approach would be
very similar. One of the points that we have tried to make clear
in our written evidence is that a long-term sustainability and
stewardship of heritage collections in the country requires all
of these things to be kept in an appropriate balance. We believe
that the British Library has made significant progress over the
past few years in trying to ensure that preservation, access and
all of those other elements about managing the stewardship of
resources that it holds for the nation are adequately resourced;
otherwise, one ends up with imbalances. One can put preservation
in jeopardy, build up backlogs and so on.
Ms Brindley: Ultimately, there
is no access without ensuring appropriate stewardship. As you
have seen from the submission, there are particular challenges
in relation to our newspapers. There is an enormous desire for
access but there is a need even for some basic developments on
quality of storage.
Q72 Helen Southworth: You made reference
to the benefits that American archives derive from philanthropists.
Can you tell us about some of the opportunities in that area,
first in terms of the big donors?
Mr Kingsley: The first and obvious
matter to point out is the funding which some American academic
institutions have received to enable them to build collections
of contemporary literary manuscripts. Just last week there was
a major conference at the British Library entitled "Manuscripts
Matter" which sought to draw attention to the importance
of those collections and the fact that the papers of many UK-based
authors find their way to American institutions that are able
to operate in the marketplace far more effectively than most British
institutions. They are able to do that simply because of the deep
pockets that private donors have given them.
Q73 Helen Southworth: Is it just
that America is a wealthier place and it is hard luck that this
material goes there?
Mr Kingsley: No. As I understand
it, the difference lies partly in the culture of giving which
operates in the United States. That is itself informed by the
taxation treatment given to institutions there. It provides a
much greater encouragement for their sorts of cultural giving
than is true in this country.
Dr Field: That matter was well
developed by Sir Nicholas Goodison in his review and the serious
recommendations that it made to the Treasury. I echo Mr Kingsley's
point that in terms of acquisitions it is extremely difficult
to get philanthropists to contribute to costs as happens in other
countries like America. It is also now quite rare to be given
collections. The British Library has received one or two very
generous gifts—for example, the Oscar Wilde bequest by Viscountess
Eccles—but that is extremely unusual. We have had some success
in our capital programme on the centre for conservation which
will open next spring. Although we have had money from the usual
charitable and lottery sources we have also had some private donations,
which we have graciously acknowledged, but it is very much an
uphill struggle. We have seriously professionalised our development
effort in the British Library both in terms of skills and numbers.
This is a big discussion. As you will know, certainly the acquisition
end of the spectrum is ongoing with government at the moment and
it is driven largely from the museums and galleries end of the
spectrum, although I have to say that some of the acquisition
challenges particularly for archival material are also pretty
significant as well. On Friday last the British Library was offered
a very important political archive—I hasten to add, not
a contemporary one—for £1.6 million. It is clearly
of national pre-eminence and has been designated as such through
the MLA procedures, but it is almost impossible to get funding
from private sources for that. The culture is simply not there.
Helen Southworth will be aware because of her involvement in one
of our conferences on literary manuscripts of fairly early date
that one of our discussion points with the Treasury is that although
there are tax incentives available they do not extend to living
authors so they can donate or sell material on favourable terms
during their lifetimes. I think you have had written evidence
from Lord Chris Smith's committee, of which we are very proud
to be a member. It has put forward some modest proposals which
we think would help in this regard. One of the things we heard
at the conference at the British Library the week before last,
to which Mr Kingsley alluded, was the very impressive gains made
in Ireland over the past few years by way of a combination of
lottery and tax incentivisation. One of the things that we fully
intend to follow up with government is the strong suggestion that
that is probably a model which is well worth looking at. It is
a relatively small country of four million people but it has pulled
in by targeted effort some very big collections of literary manuscripts.
We continue strongly to push the proposals put forward to HM Treasury
in this area.
Q74 Chairman: Obviously, digitisation
offers advantages in terms of wider public access and also storage.
You have talked about the physical constraints of storage. As
your collection grows your bookshelves increase by kilometres.
Is it your view that all material still needs to be preserved
in hard copy form, even if you have digital copies?
Ms Brindley: First, it is a statutory
requirement to collect everything that is published in the UK
and preserve it for ever. I think we should be joyous that the
UK publishing industry is so successful at one level, that is
to say, there does not appear to be any significant diminution
of print, whilst what we call born `digital material' as well
as digitised material is increasing. In that sense I think that
we have both of the problems simultaneously. As to whether the
existence of digitised materials means that you do not need to
keep the originals, I believe that we are a very long way from
that. We see digitisation primarily as enabling widespread global
access to the collections. There is not yet proven technology
which enables us to say, hand on heart, that we can guarantee
that that material will be available in hundreds of years. TNA
and ourselves are working with the leading players in the world—to
some extent we are ourselves leading—in developing systems
which need quite significant investment, but it is developmental
and it will be some years before we can say that that is the preferred
preservation format. I am afraid that it is not an "either
or"; at this stage it is, given the long transition, a "both
and" problem.
Q75 Chairman: Therefore, there is
no reason in principle, if the technology became sufficiently
developed and you were confident that you would have a permanent
digital record, why you should preserve a hard copy?
Mr Thomas: We are so far from
that state. There are benefits from digitisation in that one can
store material in a much more compressed and cheaper way, and
obviously one does not have to handle it, but like the British
Library we are a long way from a time when we would destroy anything
because it has been digitised.
Dr Field: One has to say that
for certain types of research one still needs to consult the artefact.
To come back to my example of the pre-1800 printed corpus, there
would be many elements of research that would still be enabled
only by physical examination of the actual volume and not simply
the image. Even in the contemporary world where superficially
publications are made in two formats the content is far from identical.
If one looked at an online version of a newspaper and compared
it with what you could buy from WH Smith one would see significant
variations in content, in some cases more in the print and in
other cases more in the online version. One of the challenges
in the digital publishing arena is that in some sense nothing
is really static any longer. That is a particular challenge for
us at the British Library in terms of our web archiving aspirations.
Chairman: I was not suggesting that you
take a digital copy of Magna Carta and throw away the original.
Q76 Alan Keen: Very early this morning,
probably when most of you were still asleep, I spent a couple
of hours going through these submissions. It terrifies me. Philip
Davies touched on it at the beginning when he asked what we would
lose if we did not get the money that we needed. Apart from the
good work that you are doing in extending the record, what are
we in danger of losing?
Mr Thomas: From my point of view,
the big danger is losing currently created digital records. There
are clearly physical problems with records that are in some archives
and problems about collections being sold and split up, but the
really big danger is that some local authorities or other organisations
create records digitally, as everybody does nowadays, and if procedures
are not put in place to capture those there will be a dark ages
covering the early years of this century simply because the material
will go away. Unless adequate steps are taken to preserve it,
it just will not be there in a few years. For me, the biggest
risk is the loss of digital information and records that are being
created digitally and people are not taking active steps to preserve
them.
Dr Field: The British Library
flagged that up in terms of working with Chris Mole and government
in terms of getting the Legal Deposit Libraries Act on the statute
book in 2003. We were warning of a new digital dark age. There
is still a long way to go. Websites generally, particularly those
not underpinned by commercial interests, are enormously fragile.
We have been attempting over the past two years to archive them
selectively, but because of copyright—we do not have secondary
legislation under the 2003 Act—we have to seek voluntary
permission in each case. We find that we have only a 25% success
rate in terms of permission, so we cannot archive 75% of the material
that we designate as being of likely permanent national interest
because we do not have the powers to do that. We have flagged
it up in connection with the work now started on creating a web
archive around the 2012 Olympics which Members of the Committee
may think is an entirely appropriate thing to do, but unless we
can get the 100% rate that we need we will not do that. As we
speak we are losing a lot of valuable material.
Mr Thomas: That is certainly true
for the early history of the Internet. If one tries to find historic
websites dating from before the end of 1996 they are very hard
to locate and many do not survive, including some quite well known
ones. That is a real example of what can be lost. The Internet
is such a new and revolutionary thing that it would be nice to
see what it looked like in the early days.
Q77 Alan Keen: They are the most
important archives?
Mr Thomas: Yes.
Dr Field: But we would also say—this
is confirmed by the work done by the National Preservation Office
in terms of the surveys, of which we are very pleased to have
been part—that there are pockets in terms of paper and other
traditional forms of archives where there are significant problems.
In the British Library our biggest concentration is undoubtedly
in terms of our newspaper collections. In the surveys 15% of the
materials currently are unusable and another 19% are in a hugely
fragile state. Although we have an active surrogacy and digitisation
programme, the scale of the problem is enormous. Our experience
is that this is not just UK material but by default—no one
asked us—we find that we are the custodians of a lot of
the Commonwealth's newspaper heritage and that of a lot of other
countries, including the US, because everyone got rid of the originals
and assumed that we would continue to do that.
Q78 Alan Keen: I was surprised to
read somewhere in these documents that there is a danger in storing
stuff digitally, but that is not a technical problem, is it? What
is the danger?
Ms Brindley: If one looks at the
longevity formats, for example CDs, the industry says that they
will last but tests have shown that what it means is perhaps five
to 10 years. The business that we are in is measured in hundreds
of years, so there is a physical medium issue. In addition, if
one digitises material it needs to be refreshed as the formats
change and the software needed to read it changes. It is a continuing
and dynamic problem.
Dr Field: That requires both financial
investment and also runs up against problems of the current intellectual
property framework because, like colleagues in other information
industries such as film, we do not necessarily have the statutory
powers to enable us to work in the national interest to make these
adjustments without express permission.
Mr Thomas: Ms Brindley is absolutely
right. We are just at the start of digital preservation. There
are hundreds and hundreds of theoretical articles about how to
do it. What we need is 20 or 50 years' experience of actually
doing it. That is the situation we are in.
Q79 Alan Keen: Surely, it must be
possible to designate stuff that is copyrighted and you can store
it without it being released or even accessed. Have you come across
that?
Ms Brindley: That was one of the
arguments in our submission to the Gowers Intellectual Property
review. We are particularly concerned about sound because at the
moment we cannot copy for preservation purposes that which is
in copyright, which means effectively most of the 20th century
material. We have made a strong submission that technically there
is a need to copy and recopy and we want to do it for the purposes
of preservation, respecting that there would be limitations on
access.
Dr Field: I do not know that one
can completely separate preservation from access. For one thing,
one needs to test one's preservation system to ensure that one
can actually get the material out as opposed to a theoretical
confidence that it can be accessed.
| 
