90.Many passengers are sceptical about current official methods of measuring rail passenger satisfaction and TOC performance, in particular that the measures do not adequately reflect the real experience of rail travel. These concerns have been acknowledged by the DfT, the independent consumer body Transport Focus and the Rail Delivery Group (RDG, industry-wide body that includes TOCs and Network Rail), and work is under way to bring about improvements. Below we examine the concerns and assess the steps being taken to address them.
91.The National Rail Passenger Survey (NRPS), conducted by Transport Focus, is the predominant measure of rail passengers’ satisfaction with rail travel. It surveys over 50,000 passengers per year across the GB rail network. Data from the NPRS can be disaggregated by a wide range of categories, including by TOC; type of operator (regional; London and South East; and long-distance); and journey type (leisure; business; and commuter). It surveys passengers’ satisfaction with their most recent rail journey—passengers are handed a paper questionnaire as they board the train. The survey is carried out in two main tranches, with the results published in Spring and Autumn each year. The NRPS was established in 1999 and provides data in a long time-series that can be used to identify trends.
92.As noted in the introduction to this Report, the average level of overall rail passenger satisfaction appears high (80% in Spring 2016), but it is down from a peak of 85% in Autumn 2012. Disaggregated by journey type, the data show that long-distance (88%) and regional rail passengers (85%) are significantly more satisfied than those in London and the South East (78%):
Figure 2: Rail passenger satisfaction Spring 2011–Spring 2016, by sector
Source: Transport Focus, National Rail Passenger Survey (Spring 2016), June 2016
On average leisure passengers are the most satisfied (90%), followed by business passengers (82%). Satisfaction amongst commuters is significantly lower and falling—commuter satisfaction has fallen from 80% in Autumn 2012, to a five-year joint low of 72% in Spring 2016. When the data are disaggregated by individual TOC, they show that the least satisfied passengers were those using Southeastern Railway and Southern Railway commuter services in and out of London (both scored 69%):
Figure 3: Rail passenger satisfaction Spring 2011–Spring 2016, by journey type
Source: Transport Focus, National Rail Passenger Survey (Spring 2016), June 2016
93.While the low ranking of Southestern and Southern Railways reflects the weight of evidence we received from these operators’ dissatisfied passengers, a number of Southern Railway’s London commuter passengers expressed incredulity about even the comparatively low figure of 69%; they simply did not feel it reflected the overwhelmingly negative recent experience of using Southern’s commuter services. It should be noted, however, that the negative sentiment of Southern’s many London commuter passengers will to some extent be averaged out by the more positive experiences of other types of passengers on other parts of Southern’s diverse network.
94.A commonly held view was that NRPS results were deliberately manipulated to produce artificially high satisfaction scores. Some witnesses were convinced that that NRPS was deliberately carried out at off-peak times or on quieter routes, to avoid the mass of dissatisfied passengers on the busiest trains. Some argued that the NRPS’s “snapshot” method—surveying satisfaction with a single journey on a particular day—produced skewed results, which “rarely reflect the true picture of daily problems faced by regular commuters”. They felt that the NRPS should “look at the overall experience, and not just the last journey made.”
95.Some expert witnesses had a degree of sympathy for this argument. London TravelWatch, the statutory consumer body for the Capital, noted that Which’s annual rail consumer survey considered satisfaction over the previous 12 months, and was therefore able to track passenger sentiment over a longer period. Railfuture made a similar point; it believed that the Which? survey’s methodology was more likely to reveal commuter dissatisfaction.
96.Transport Focus defended the NRPS’s methodology. It noted that its sample size was by far the largest: 50,000 passengers per year in two tranches compared to 7,000 participants in Which’s annual survey. It insisted that its sampling plan ensures an appropriate spread of rail journeys by: journey purpose; peak/off peak; weekday/weekend; and staffed/unstaffed stations. It emphasised that, while the Which? survey typically produced lower satisfaction scores, Which?’s ranking of TOCs in order of passenger satisfaction tended to be consistent with the NRPS. Transport Focus emphasised two key factors that contributed to the lower scores produced by the Which? survey:
97.Broadly, industry and expert witnesses agreed that the NRPS was well-established, methodologically robust and provided useful time-series data to track trends and compare performance.
98.A number of possible enhancements to the NRPS were suggested, including adopting more modern techniques such as online surveys. The Urban Transport Group thought that the views of non-users of trains, including those who were unable to board a train due to overcrowding, were important but currently omitted from the sample. A common view was that the main tranches of the NRPS ought to be carried out more frequently.
99.The DfT’s position was that the NRPS in its current form was useful but had limitations. It told us it was working with Transport Focus and the TOCs to move towards improved, broader and more frequent measures of passenger satisfaction.
100.Transport Focus told us that it would soon be testing a new approach in relation to the recently let Greater Anglia franchise agreement, which is due to commence shortly. The new approach would focus on measuring three Key Performance Indicators (KPIs): customer experience, presentation of facilities and staff performance. Customer experience would be evaluated via passenger satisfaction surveys using the NRPS methodology; presentation of facilities and staff performance would be measured using “mystery shopper” exercises.
101.Transport Focus told us it intends to evaluate performance against each KPI in every Reporting Period of the franchise agreement i.e. at four-week intervals. Over 7,500 Greater Anglia passengers will be surveyed each year, providing the new operator with “more data in order to better improve the customer experience.” The mystery shopper exercises will cover broader issues, including “cleanliness, upkeep and repair, ticket buying facilities and the attitudes and helpfulness of staff.” The franchise agreement, when published later this year, is likely to include targets against each KPI; failures will result in Remedial Plans and/or financial penalties imposed on the TOC.
102.The National Rail Passenger Survey (NRPS) is long-established and methodologically sound, but it could and should be improved to more accurately reflect the everyday experience of passengers. We therefore welcome the proposed approach to be tested in relation to the new Greater Anglia franchise, which we intend to further scrutinise in our broader inquiry into franchising.
103.The GB-wide NRPS’s principal weakness is that it is a relatively narrow “snapshot”, carried out in two tranches per year, of passengers’ satisfaction with a particular journey. Steps should be taken to address this, while maintaining the NRPS’s key strength—provision of reliable time-series data by which to identify trends and compare TOC performance, disaggregated by a number of logical categories. We recommend that the GB-wide NRPS be carried out at least quarterly, and that the DfT fund Transport Focus to do this. We further recommend that Transport Focus take careful steps to broaden the NRPS’s sample, including by piloting the use of online surveys and surveying the views of non-users of rail travel, including those who were unable to board a train due to overcrowding.
104.The headline measure of TOC performance tends to be the Public Performance Measure (PPM)—the percentage of trains that arrive at their destinations “on time” (defined as within five minutes of scheduled arrival time for regional and commuter trains and within 10 minutes for longer distance trains). Official data show that the PPM is over 87.4% across the whole GB network. Performance across franchised TOCs ranges from 70.4% (GTR) to over 97% (c2c).
105.Broadly witnesses felt that PPM was too blunt a measure, which took little account of the passenger’s experience of train travel. Several pointed out that PPM records only what happens to trains, rather than outcomes for passengers, and in doing so creates perverse incentives for TOCs to take actions to maintain PPM figures, such as missing out stops en route to make up time to the destination station, to the detriment of some passengers. This appears to be a common experience.
106.Transport Focus argued that better performance measures, which take greater account of the experience of all passengers, were required to improve trust in the industry. Its view was that performance measures should “make sense to passengers and drive behaviours that they want to see”. These included “hard targets” for “right time” punctuality; Transport Focus’s research has shown that passenger dissatisfaction starts to increase well within the five or 10 minute leeway of the PPM measures. For high frequency services, London TravelWatch favoured an “excess journey time” performance measure, as operates on London Underground, as a way of more accurately capturing the real experience of passengers:
This is the total additional time that passenger journeys would have taken compared with the scheduled time, i.e. taking account of the numbers of passengers travelling. For high frequency ‘metro’ services, this is probably a much better measure of reliability than a PPM based on whether trains arrive at their destination [ … ] within 5 or 10 minutes of their public schedule.
107.The current Public Performance Measure (PPM) does not reflect the real passenger experience, and produces perverse incentives—most obviously, missing out intermediate stops to make up time in reaching the train’s destination—that often work against the interests of many passengers. This is unacceptable and must stop.
108.We recommend PPM be abandoned as the headline measure of TOC performance, and the DfT work with Network Rail and Transport Focus to establish updated “right-time” measures, which take account of punctuality at all stops on a train’s journey. We recommend that a replacement for PPM be implemented by summer 2017.
109.Given the huge capacity challenge facing the railway, it is perhaps unsurprising that many witnesses drew attention to uncomfortable, and potentially dangerous, levels of overcrowding. This was a major concern for passengers across the network, not just on the busiest London commuter routes. The Campaign for Better Transport (CfBT) noted that reducing overcrowding was one of the highest priorities for passengers; overcrowding was uncomfortable, highly stressful and, in many cases, forcing commuters to travel to work very early or leave work late, to the detriment of spending time with their families.
110.CfBT emphasised recent evidence that a “troubling number” of individual services were operating at nearly double their recommended capacity. The DfT’s official statistics on overcrowding measure the percentage of passengers in excess of capacity (PiXC) and percentage of passengers standing, by TOC and 11 major city regions. These show that in 2015 London services were most overcrowded: on a typical autumn weekday morning during peak hours, trains were on average 6% over capacity, with 24% of passengers standing. The next most overcrowded trains were in Manchester, at 4% over capacity, with 12% of passengers standing. In addition, the DfT identifies the “top ten” most overcrowded individual train services in England and Wales. These statistics show that in Autumn 2015, the ten most overcrowded services were between 61% and 129% over capacity. Of the “top ten”, eight were in London and two in Manchester.
111.A number of witnesses argued that levels of overcrowding should be more effectively encompassed in TOC performance measures. Professor Chris Baker and Dr Jochen Brandt, for example, both believed that trains that are overcrowded to the extent that some passengers are unable to board should be counted against TOC performance in a similar way to late and cancelled trains.
112.Persistently high levels of overcrowding on particular services on some parts of the rail network are a concern for many passengers. The current official measure of overcrowding, the DfT’s percentage of passengers in excess of capacity (PiXC) statistics, illustrates the problem only in broad average terms, in 11 city regions. In addition, the DfT identifies the “top ten” most overcrowded individual train services. There is a very strong case for more clearly identifying, and taking action to alleviate, substantial overcrowding on specific services across the network. We recommend the DfT review and redesign its PiXC statistics, with a view to more clearly identifying particular train services operating at substantially over capacity. The Department should draw on these statistics to develop a more coherent strategy for tackling overcrowding, including by incentivising TOCs, through franchise agreements, to alleviate the worst examples of persistent overcrowding on particular services across the rail network.
100 See, for example, Iain Kernaghan (); Adam Martin-Lawrence (); Grahame Mitchell ()
101 DfT (); Transport Focus (); RDG ()
102 Transport Focus ()
104 Caroline Clark (); Adam Martin-Lawrence (); Iain Kernaghan ()
105 See, for example, Iain Kernaghan (); Grahame Mitchell ()
106 Caroline Clark ()
107 Adam Martin-Lawrence ()
108 London TravelWatch ()
109 Railfuture ()
110 Transport Focus ()
111 See, for example, FirstGroup plc (); ORR (); London TravelWatch ()
112 Virgin Trains ()
113 Urban Transport Group ()
114 DfT (); Rail Delivery Group (); Virgin Trains ()
115 DfT ()
116 Transport Focus ()
117 Network Rail, ‘’, accessed 24 August 2016
118 See, for example, Dallas Roulston (); Ed Aldridge (); Dr Tom Haines (); Paul Sparks ()
119 London TravelWatch ()
120 See, for example, Tyne and Wear Public Transport Users Group (); Sherborne Transport Action Group (); Dr Jochen Brandt (); Emma Sullivan (); Shouvik Datta (); Martyn Clark ()
121 Campaign for Better Transport ()
122 DfT, , July 2016, table RAI0214
123 DfT, , July 2016
124 Prof. Chris Baker (); Dr Jochen Brandt ()
13 October 2016