House of Commons - Communities and Local Government Committee

Select Committee on Communities and Local Government Committee Fourth Report

8 Performance measurement and management

The Comprehensive Performance Assessment process

104. The 2002 Bain review suggested that the "Government should discuss with the Audit Commission and Accounts Commission an assessment for fire authorities, building on lessons from the Comprehensive Performance Assessment (CPA) process".[255] The Government announced in its 2003 White Paper that "there will be independent, rigorous quality assurance of the fire and rescue service".[256] As a result, the Audit Commission was invited to introduce a new inspection regime based on the CPA process used for local government. The aim of the CPA is to examine how fire authorities are being run, assess their performance against local and national priorities and reflect how authorities respond to local challenges. The first round of the CPA process rated 47 per cent of authorities as either good or excellent.[257]

105. Generally, there has been a positive response towards the new CPA arrangements, even though only one full round of inspections has been completed. Lancashire FRA found the new CPA process "extremely helpful in focussing our efforts towards modernising the service".[258] Devon said that "we really felt as though we had been assessed at the end of the process".[259] The RFU and UNISON also welcomed the change to the CPA arrangements.[260]

The audit burden and cost

106. With the move to the CPA process, the ODPM believed that it had lifted "the burden of inspection".[261] The Government, rather than FRAs, is required to pay the Audit Commission for the CPA. The CPA process however takes up a significant amount of the fire authorities' time. Lancashire Fire Authority estimated that the CPA process took 203 days of staff time, amounting to £52,000.[262] Devon Fire Authority quoted a figure of around £48,000.[263] The ODPM told us:

"In 2001-2, ODPM spent £1.85 million on HMFSI's inspections of the Fire and Rescue Service…the figure solely relates to the direct costs of inspection activity during that financial year. ODPM provided the Audit Commission with a total of £4.3 million of grant for their work on developing and implementing Fire CPA over the two financial years, 2003-4 and 2004-5. This figure includes Audit Commission expenditure on recruiting, training and retaining staff to take forward work on FRS performance assessment and the costs of developing, consulting on and piloting the CPA methodology. It also includes the costs of supporting self assessment, employing peer reviewers and the direct costs of inspection activity to deliver the CPA judgements. The £4.3 million covers all the Audit Commission's costs for two financial years and there was no other inspection activity during this period".[264]

The results of the first full CPA process

107. FRAs received an overall score of excellent, good, fair, weak or poor. This overall score is based on the authorities' performance against nine themes relating to what the authority is trying to achieve, how the authority delivers its priorities and what the authority has achieved to date. According to the Audit Commission, 22 out of 47 fire authorities were good or excellent, though only two received a score of excellent. 67 per cent of the metropolitan authorities which serve larger populations were good or excellent. This compares to only 31 per cent of the county authorities, serving smaller populations.[265] The Audit Commission concluded that, with fewer staff and smaller budgets, economies of scale and capacity are more of an issue for county fire and rescue authorities, which clearly impacts on their CPA assessment.[266]

108. None of the FRAs that submitted evidence challenged the score that they had received under the CPA process. The CFOA did express some disappointment because "despite the fact that almost half [of the fire and rescue authorities' scores] were good or excellent,[…] [the CPA report] really homed in on a lot of negative areas".[267] The CPA highlighted performance management, managing staff skills and capacity as well as diversity and equality as areas of weakness.[268] We welcome the introduction of Comprehensive Performance Assessment for the FRS. We note its usefulness in highlighting areas where improvements are required.

Assessing the operational activities of the FRS

109. A recurring criticism of the CPA process concerned the lack of any assessment of the FRS operational performance or capability. The FBU and the CFOA mentioned the need for an operational review of fire authorities.[269] Similarly, North Yorkshire FRA said that such assessments were needed to "enhance the operational capability of individual FRSs".[270] Greater Manchester FRA said that an independent operational review would help to reassure the public and it should be carried out by the Audit Commission as part of future CPA reviews.[271] We welcome the plans for the independent operational assessment of fire authorities to be carried out next summer. The proposed assessment should be conducted by the Audit Commission and combined with the current CPA to minimise the audit burden on FRAs.

Targets and performance indicators - are they the right ones?

110. Under the 2004 Spending Review, the ODPM had to meet a Public Service Agreement (PSA) target specifically relating to the Fire and Rescue Services. PSA 3 sets targets for reducing accidental fire-related deaths in the home and the number of deliberate fires by 10 per cent. Linked to this, FRAs have been given national Best Value Performance Indicators (BVPIs). The ODPM drew up the 2005-06 BVPIs with the help of a working group which included the Audit Commission, Local Government Association, Chief Fire Officers' Association and representatives from individual authorities. The BVPIs were tested with individual fire authorities and a full consultation exercise was carried out.[272] Some of the national BVPIs are specifically matched to the national fire PSA, encouraging fire authorities to contribute to the achievement of the ODPM's target. FRAs also set local performance indicators which are incorporated into the Integrated Risk Management Plans.

111. The ODPM considers effective national BVPIs as ones that measure outcomes rather than activity. The ODPM does not consider national targets such as response times to be appropriate because local risks and events outside the FRS's control will have too much of an impact. This is to avoid introducing "perverse incentives".[273] We support this approach to setting targets. There must be, however, a link between the BVPIs and the ODPM's national targets to ensure that the ODPM meets its PSA targets. The BVPIs do not cover the full range of activities carried out by fire and rescue authorities, for example, rescues related to road traffic accidents and flooding are not covered. The Audit Commission highlighted this in their CPA report, noting that most Chief Fire Officers and Chairs of FRAs were unhappy with the current BVPIs as measures supporting CPA.[274] It will be difficult for the ODPM to carry out an operational assessment of the fire authorities without having performance indicators which cover all of the FRAs' operational activities. We recommend that the Government ensure the CPA inspection process covers all the activities of the Fire and Rescue Authorities.

Number of deliberate fires

112. PSA 3 states that by 31 March 2010 the ODPM should reduce the number of deliberate fires by 10 per cent. We were encouraged that the ODPM has made good progress against this target. For the year to 30 June 2005, the number of deliberate fires had fallen by 16 per cent from the previous year. In its Autumn Performance Report, the ODPM attributed this fall mainly to the reduction in the number of deliberate road vehicle fires, which has fallen by 24 per cent; according to the ODPM, the reasons for this reduction included improvements in vehicle licensing, vehicle removal schemes (some funded by the Arson Control Forum) and the increasing price of scrap metal (making cars worth more at the end of their useful life).

113. As part of the requirement to reduce deliberate fires, the ODPM's National Framework expects FRAs to be more proactive about community fire safety work. The Audit Commission reported that, whilst FRAs are making progress on community fire safety work, improvements are needed in the way that the impact of this work is measured, with the introduction of meaningful performance indicators.[275] As a result, it is difficult to determine the extent to which the ODPM is meeting its PSA target relating to the work of the FRS. We recommend that the Government introduce performance indicators on community fire safety.

Number of accidental deaths

114. PSA 3 states that by 31 March 2010 the ODPM should reduce the number of accidental fire-related deaths in the home by 20 per cent and requires that no FRA has a fatality rate from accidental fires in the home that is more than 1.25 times the national average. This is another area where progress is being made.[276] In order to help achieve this target, the FRAs have a supporting BVPI for the number of deaths. In its Autumn Performance Report, the ODPM stated that its monthly figures indicated that deaths were falling. In a February 2006 press release the ODPM announced that the total number of fire-related deaths had fallen and was at its lowest since 1958. Over the 12 month period to June 2005, the ODPM stated that across the UK accidental fires in dwellings fell by 16 per cent.[277] This demonstrates that there have been successes in reducing deaths in the home. The CFOA has also cited fire prevention work as a key reason for the reduction in deaths.[278] However, the ODPM said that it does not have information on the factors that influence this target so is considering how to capture this information.[279] We congratulate the FRS on meeting the PSA target on accidental deaths. This is clear evidence that the shift to fire prevention is having a positive impact. The Government should ensure that it identifies the factors which have an impact on the number of accidental deaths (for example ethnic and socio-economic background) so that resources can be appropriately targeted in future.

Number of rescues

We received some evidence which indicated that the ODPM should consider the rescue activities of the FRAs. The FBU was concerned that the prevention agenda detracts from the rescue and intervention role of the FRS, stating that there should be more emphasis on the service intervening to deal with fires.[280] The CFOA also said that, although the FRS has achieved much under fire prevention initiatives,[281] its "primary [role] responding agency to civil emergencies[…]must be fully recognised, understood and properly resourced".[282] But neither CFOA nor the FBU suggested introducing a target for rescues. We agree that the first duty of the FRS should remain providing a response to fires and civil emergencies. The introduction of a target directly relating to rescues would not, however, serve a useful purpose. Measuring performance in such a way could prove misleading because the number of rescues could decline as a result of earlier warning, avoiding the need for rescue at all, or, perversely, could increase as a result of effective fire safety initiatives, such as smoke alarms.