Select Committee on Public Administration Minutes of Evidence


Memorandum submitted by the Public Record Office

INTRODUCTION

  1.1 This paper considers the public records dimension of freedom of information (FOI) policy. The main lines of the public records proposals are set out in Chapter six of the White Paper Your Right to Know. Further work will, of course, be required to translate these proposals into legislative form.

A UNIFIED ACT

  2.1 The alignment of the existing access right to material older than 30 years with the new access right to material less than 30 years old should provide the public with an integrated access system which offers progressively more liberal access to documents as they become older and their sensitivity diminishes.

  2.2 The two rights will not become one. This would not only be impractical because their scope will be so different (see paragraph 2.3 below) but it would be undesirable. The 30 year date (hereafter called the archival threshold date) will remain of central importance because it will mark a step change in the public's rights. Not only will the access conditions become more liberal, but by that date the great majority of selected public records will have been transferred to the Public Record Office where they will be available on request in the reading rooms as at present. At the threshold the onus shifts to departments to make a case for the withholding of documents of continuing sensitivity: if the case is not made, then documents automatically fall open. The retention of this step change within the new access architecture will be in contrast to some FOI regimes elsewhere. In Sweden and Canada, for example, there is an integrated access right which applies regardless of the age of documents. The access exemptions do not become progressively more liberal over time and so applications for access to historic documents which have not previously been released have to be considered individually against the same criteria which apply to recent material. This entails a continuing cost for departments and delays for researchers, who have to wait on the outcome of their applications before seeing files. The retention of the threshold in our system will avoid these problems and it will ensure that the great majority of selected public records are in the Public Record Office - and fully catalogued - by the time they are 30 years old.

  2.3 The access right to public records - which is contained in section 5 of the Public Records Act 1958 - currently covers central government bodies and the courts (the precise definition appears in Schedule One of the 1958 Act). This is clearly a much more restricted range than the proposed coverage of the remainder of the FOI legislation. This is not problematic but the legislation will need to detail the differences in coverage of the two rights, possibly in different schedules. The introduction of FOI legislation will not cause the Public Record Office to become involved in the archiving of records created by non-public record bodies. For example, it will continue to be the responsibility of local authorities to make arrangements for their own records. Most commonly these are deposited in local record offices after they have been reviewed to determine their lasting value.

  2.4 One of the principal tasks to be undertaken in aligning the two rights will be the revision of the criteria governing the withholding of records for longer than 30 years. Documents not available after 30 years are either closed for a specified period or retained within departments for an indefinite period (subject to periodic reviews, where appropriate). In each case, departments seeking approval to withhold documents must demonstrate that the records comply with the criteria for non-disclosure. These were last revised in 1993 and published in the White Paper Open Government (a copy of the relevant extracts appears as Annex A to this paper). There is a similarity between the criteria for extended closure and the proposed FOI specified interests but some recasting will be required to ensure a smooth transition at the threshold. The criteria for retentions take a different form and so further work will be needed to ensure that they bear a logical - and readily understandable - relationship to the FOI specified interests.

30 YEAR RULE

  3.1 The White Paper notes that the Government gave careful consideration to lowering the threshold date from 30 years. It further notes that the heavy costs of a change and practice in other western countries influenced the decision to retain the current rule.

  3.2 Annex B to this memorandum provides estimates for a move to a 20 or 25 year rule. To set these figures into context, it might be helpful to provide some sense of the scale of the task that would be involved in effecting a transition. Each year government spends up to £35 million on storing and managing its files. Some 1,450 km of shelving are filled with these files (Source: Records Storage and Management, A Scoping Study, Cabinet Office and Public Record Office, February 1997). If the threshold date were to move below 30 years, it would be necessary to review these records more rapidly to determine which should be preserved and which contained items of continuing sensitivity. Then it would be necessary, in rapid order, to accession those documents falling in age between the new threshold and the present 30 year date. This task is, of course, achievable but it is labour intensive and consequently costly.

  3.3 International practice on access to archives is difficult to characterise. Some western countries - the USA, Canada and Sweden - do not have a threshold date. Of those which do, 30 years remains the norm. For example, it applies in Germany, Denmark, Ireland and Australia. There are a handful of examples of lower thresholds - the Netherlands is to operate a 20 year rule - but equally, there are examples of more restrictive systems. Straightforward comparisons on the basis of legislative provisions are often misleading: liberal provisions are of little value to the researcher if the administration does not transfer records to the archive efficiently. The British arrangements for effecting transfers and for providing access in the Public Record Office are well-established and bear comparison with best practice across the world.

  3.4 There are already arrangements in place for the release of some papers before they become 30 years old. Since 1992 departments have been encouraged to release papers earlier than 30 years - if they do not have continuing sensitivity. Departments have also been undertaking a review of records which have been withheld for longer than 30 years to determine which may be released. This initiative, which is continuing, has already led to the release of more than 85,000 documents.

  3.5 FOI will lead to greater access to official papers before they are 30 years old. For the most part, the release of requested material will take place under the auspices of the organisation which created the records. But if that organisation is a public record body and if the material has been selected for permanent preservation, it is possible that an FOI release will trigger an early transfer of records to the PRO. Two points should be considered here. The first is a matter of scale. Each year more than 1.5 km of shelving at the Public Record Office are filled with newly arrived records. But this represents less than 5 per cent of the total quantity of records created by public record bodies in the first place. It is not proposed that records be selected simply because they have been the subject of an FOI request: selection decisions will continue to assess the long-term historical value of documents. The second issue concerns publication. If a department receives repeated requests for the same item, it may judge it beneficial to publish the item in addition to making an early transfer to the PRO.

OPERATION OF THE 30 YEAR RULE AND THE ROLE OF THE ADVISORY COUNCIL

  4.1 At present, departments wishing to withhold papers for longer than 30 years have to make a case in accordance with the criteria set out in the 1993 White Paper Open Government. These submissions are collated and checked by the Public Record Office for independent scrutiny by the Advisory Council on Public Records. At its quarterly meetings, the Council considers the merits of the cases that have been made. It may refer matters back to departments if it is not satisfied, but if it is content, the papers pass to the Lord Chancellor for authorisation.

  4.2 The White Paper proposes one important change to this process. The Public Record Office is to be given a statutory responsibility to support the Advisory Council in its work. At present, the PRO has no formal status in the vetting of applications. This reform should lend the PRO's contribution added authority and should thereby ensure that the submissions coming before the Advisory Council have been subject to initial scrutiny and that all relevant information is available to Council members.

  4.3 The alignment of the FOI and public records regimes will bring with it a new right for the public to challenge the withholding of documents which are older than 30 years old. This will replace the current arrangement whereby researchers may bring such matters to the attention of the Advisory Council. The Information Commissioner, unlike the Council, will be able to order the disclosure of material which he/she deems not to comply with the relevant criteria. The Commissioner will undoubtedly have recourse to the organisation which created the records (or its successor) since responsibility in such matters remains with the originating body even if closed records are passed to the Public Record Office.

RECORDS MANAGEMENT

  5.1 Your Right to Know makes it clear that the success of FOI will depend on the quality of the records produced and the manner in which they are maintained. Departments are to come under a new obligation to ensure that they discharge their responsibilities in these matters.

  5.2 The Public Record Office has an important role to play here. In February 1997 a joint study with the Cabinet Office Efficiency Unit concluded that the Office should take a clearer leadership role in the managing of government's current records. In the past, the Office has - for obvious reasons - tended to focus on the selection and transfer of historic records. The Office has accepted this recommendation enthusiastically not least because the automation of government's operations represents a new challenge for record keepers. National archives across the world are recognising that they need to be involved when office systems are designed and installed if they are to produce reliable records in the long term. To this end, the Office is producing guidance to government on the creation and management of their records. This work will continue and it will be refocused to ensure that a clear direction is given on the requirements of the FOI system.

January 1998

ANNEX A

EXTRACTS FROM OPEN GOVERNMENT (Cm 2290, 1993)

  Paragraphs 9.23-9.27 of the White Paper set out the basis on which records are retained within departments. The criteria for extended closure are given in Annex C. Copies of these pages are included hereafter.

RETENTION

  9.23 As stated in paragraph 9.8 (ii) above the Public Records Act makes provision for records to be retained by departments. There are two main categories of records which are retained rather than closed. These are, first, records which are retained for administrative reasons, usually because they are awaiting review, or are in constant use. There are secondly, records whose sensitivity is such that no date can be put on their potential release, most of which fall into one of the categories for which the Lord Chancellor has given "blanket" approval to retain.

  These categories are:

   -    Security and intelligence material (renewed in February 1992);

   -    Civil and Home Defence material (now under review);

   -    Atomic Energy - pre-1956 defence-related material (now under review);

   -    Atomic Energy - post 1956 defence-related material (to be reviewed in 2006);

   -    Personal records of civil servants - retained for administrative purposes.

  9.24 Records retained by departments for other than administrative reasons are subject to regular review i.e. at least every ten years and, in the same way as closed records, subject to the test of "actual damage" caused by release. Thus when the sensitivity has passed the record will be released.

EXPLANATION OF THE REASONS FOR CLOSURE/RETENTION

  9.25 Hitherto when records have had to be closed for longer than 30 years or retained by departments no reason has been given other than to say that the provisions of the Public Records Act 1958 permit such closure or retention in accordance with agreed criteria. This is because governments have taken the view that to say more could endanger the very information that closure or retention seeks to protect. It has been the practice of successive Administrations not to disclose the contents of records withheld from public release.

  9.26 The contents of withheld records must continue to be protected but, in future, departments will, in response to queries about closed or retained records, say which of the following reasons applies and give such other information as is appropriate if this can be done without putting at risk the information which has led to the material being withheld. The reasons, as shown above, for which records are closed or retained for longer than 30 years are:

   -    administrative (e.g. to allow the reviewing process to be completed);

   -    national security;

   -    international relations/defence/economic;

   -    material given in confidence;

   -    personal sensitivity (would substantially distress or endanger persons affected by disclosure of their descendants).

  9.27 The Security and intelligence "blanket" referred to above permits the records of the security and intelligence agencies themselves to be withheld with the Lord Chancellor's approval. This is because the agencies depend for their effectiveness on maintaining the confidentiality both of their methods of operations which, despite the passage of time, are still extant, and, most of all, of the identities of people who put themselves at risk in the service of the State. However, papers originating in the agencies which have been held in other departments over the years (i.e., "intelligence-related" documents) and often reflect the product of the agencies' operations, are reviewed as part of the normal procedures and released if they are no longer sensitive.

ANNEX C

GUIDELINES ON EXTENDED CLOSURE

  Guiding Principle: All records not retained in departments should be released after 30 years unless:

    (a)   it is possible to establish the actual damage that would be caused by release, and

    (b)   the damage falls within the three criteria set out below.
Criterion (as stated in the 1993 White Paper Cm 2290) Nature of record Closure period
Exceptionally sensitive records containing information the disclosure of which would not be in the public interest in that it would harm defence, international relations, national security including the maintenance of law and order, or the economic interests of the UK and its overseas territories. All records meeting this criterion, including those concerned with national security and those containing information the premature disclosure of which would impede the conduct of the policy of HM Government abroad. 40 years
Documents containing information supplied in confidence the disclosure of which would constitute a breach of good faith. Most records meeting this criterion, including commercial and personal information supplied in confidence. 40 years, or until death where known (where appropriate)
Tax Information75 years
Personal information subject to a statutory bar during the lifetime of the person concerned. 75 years, or until death where known
Records of the decennial census of population. 100 years
Documents containing information about individuals the disclosure of which would cause either: Records meeting this criterion and containing sensitive personal information which would substantially distress or endanger a living person or his or her descendants. 40-100 years
(i) substantial distress, or
(ii) endangerment from a third party.
to persons affected by disclosure or their descendants.
Records containing information from which it is likely that a woman could be identified as a rape victim. 100 years


ANNEX B
Costs of reducing the 30 Year Rule
£ million
Total cost for first five years Recurring annual cost thereafter
Reduction from 30 to 25 years31 0.3
Reduction from 30 to 20 years60 0.5

Notes: 1.

This shows cross-government costs: the costs do not only fall on the Public Record Office.

2. Full implementation has been based on a five year period. This period is necessary to complete the increased reviews and transfer activity.

3. The cost to departments and agencies is derived from estimates in the Cabinet Office/PRO Scoping Study of Records Management (February 1997).

4. Costs reflect increased staff time and storage space.

5. Costs have not be evaluated for a further reduction to, for example, 15 or 10 years. The additional cost for these reductions - and its profile - would depend to a large extent on the period over which the change was introduced.


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries

© Parliamentary copyright 1998
Prepared 31 March 1998