Court and Tribunal reforms Contents

7Engagement and evaluation

179.HMCTS recognises the value of communication and engagement with its stakeholders in development of the reform programme and the importance of evaluating the programme’s impact. However, our evidence suggested that HMCTS has not engaged well enough with its stakeholders, and that its approaches to evaluation are unlikely to maximise objective scrutiny.

Stakeholder engagement

180.The NAO’s first report on court and tribunal reforms found that the programme’s tight timetable created challenges for stakeholder engagement; it recommended that HMCTS allow more time to engage with affected parties.289 Developing this theme, the PAC recommended that: “By November 2018, HMCTS should publish plans on how and when it will engage with stakeholders and be clear about how it will act on the feedback received and adjust plans if necessary.”290

181.In response, HMCTS accepted the need for greater, and more active, stakeholder engagement and published a plan in November 2018, which describes the three strands of its approach:

182.A minority of witnesses thought that they had been adequately engaged in the reform programme.292 Much of our evidence indicates dissatisfaction with the quality and frequency of HMCTS’s stakeholder engagement and the transparency of the programme. Many witnesses thought that there had not been enough public consultation on reform proposals, with few mechanisms for those with experience of the court and tribunal system to become involved.293 It was suggested, for example, that self-represented defendants in housing possession cases be consulted about their experiences of making emergency applications for warrants of possession to be set aside.294 Ken Butler from Disability Rights UK reflected:

If you stopped anybody in the street and asked them if they had heard about this, most people would not know about it. I am not even sure that people who are current users of the tribunal have had their opinions asked for. For example, if someone has had their face-to-face appeal hearing for PIP, does anybody ask them afterwards, “What do you think of this? This is what we are thinking of doing.”295

183.Others accepted that engagement was taking place, but thought that it was insufficient. Witnesses from the legal profession were particularly unhappy. The Law Society had not been directly informed about key HMCTS announcements.296 The Chartered Institute of Legal Executives (CILEx) accepted that HMCTS had recently increased its engagement on the progress of reforms, but had experienced this as being limited to strategic and communications elements, not engagement on individual projects.297 The Bar Council was unhappy about engagement meetings scheduled during court hours. Hammersmith and Fulham Law Centre complained about being “left in the dark” on the testing of reforms carried out by the Government, and said that HMCTS “roadshows” had not had sufficient capacity or geographical spread to ensure access for stakeholders. The Public Law Project was among those who commented that much of the design for the programme was taking place “within closed focused groups and similar processes.”298

184.Some witnesses from the legal profession doubted whether HMCTS consultations were of any real value, suggesting that policy decisions had already been made and that consultation was therefore “lip service”.299 Matt O’Brien from the Criminal Law Committee of Birmingham Law Society said defence practitioners doubted whether their views were really considered. He complained of a wider sense of consultation fatigue:

We have had so many consultations from the MoJ, the LAA and the SRA on different issues, not specifically the subject of court reform, where people have become very engaged and submitted detailed responses but there is no evidence that they have been taken into account at all, so I think a sense of fatigue creeps in.300

185.Similar misgivings about HMCTS engagement were expressed by other witnesses, including NGO agencies providing advice and representation. IPSEA said it had not been consulted directly on the reforms, nor had consultation taken place via the local Tribunal User Groups which it attended.301 The Free Representation Unit accepted that there had been improvements in communication with their own organisation, but thought that information was still limited.302 Citizens Advice had experienced communication largely taking place through updates from individual HMCTS project teams:

This methodology has however sometimes made [it] difficult to keep track of so many parallel changes and to get a big picture view of how the reforms will ultimately come together.303

186.The consequences of failure to co-ordinate HMCTS initiatives with police IT systems was illustrated by Detective Chief Inspector Kirby from Thames Valley Police, who commented that it sometimes felt change was being “directed upon us, rather than us being involved in supporting it”. To fulfil its commitment to providing evidence digitally by 2020, his force had invested in new technology—only to find that this system did not connect with the court IT infrastructure.304

187.The PCS Union saw the involvement of court and tribunal staff in the reform programme as key to the programme’s credibility. PCS described Government consultation with trade unions on behalf of staff as “virtually non-existent” and pointed out that staff members “understand the legal system and deal with the most vulnerable of users”. The union expressed concern about the impact of the loss of staff expertise on the quality of justice, as well the effect on communities of job losses arising from centralisation of roles within HMCTS.305

Judicial engagement in the reforms

188.The Lord Chief Justice emphasised that the involvement of the judiciary in the detail of the reform programme was seen as central to its success and confirmed that the senior management of HMCTS was receptive to judicial input. He commented:

We provide the most ready source of expertise and are particularly sensitive to issues such as access to justice and open justice. There is much expertise in HMCTS but in a large project with many strands many who have been drafted in to the reform project have little direct experience of the administration of justice.

He explained that all levels of the judiciary have been engaged with HMCTS from the beginning of the reform programme. Judicial input is overseen by a Judicial Reform Board led by Lady Justice Thirlwall, along with the Senior President of Tribunals and District Judge Tim Jenkins. The Judicial Executive Board discusses reform at almost all its meetings.306

189.He also explained that the senior judiciary’s “Judicial Ways of Working” (JWOW) project has sought to achieve engagement with judicial office holders at all levels. In April 2018, four JWOW documents were published, relating to reform plans for crime, civil, family and tribunals. All judicial office holders were encouraged to respond to documents relevant to their areas of work via an online survey and attend local reform events to express views. Responses were received from or on behalf of 10,000 judicial office holders, and 800 people attended 38 events. Survey responses, along with views expressed at events, were used to inform the senior judiciary’s approach to reform and discussions with HMCTS.

190.In addition to JWOW, a network of judicial groups was set up to ensure judicial input into the design of new products and services, including:

191.The Senior President of Tribunals said that the judiciary had taken a full part in the leadership of change. He has established a Tribunals Change Network that brings together project judges, judges who advise on the Judicial Engagement Groups, leadership judges from the Tribunals Judiciary Executive Board and representatives from judicial associations. The independent Administrative Justice Council, chaired by the Senior President of Tribunals, aims to make the administrative justice system accessible, fair and effective.308 It has set up three expert panels drawn from the academic, pro-bono and advice sectors which are helping to provide external scrutiny for the tribunal reform programme.

192.However, not everyone shared the senior judiciary’s belief that there were adequate mechanisms for consultation with all levels of the judiciary. The Council of HM Circuit Judges said it was “apparent from our own inquiries that there are many differing views among the judiciary as to the impact and success of the reforms to date”; it thought that judicial associations should be involved in ongoing consultation and evaluation, as well as the Judicial Engagement Groups.309 The Legal Committee of HM Council of District Judges (Magistrates’ Courts) thought that judicial office holders had been given adequate opportunities to comment on the reform programme.310 By contrast, the Association of HM District Judges questioned whether there had been meaningful consultation with all levels of the judiciary on court closure proposals:

There is a feeling that whilst MoJ/HMCTS consult, they do not listen but proceed simply to implement the decisions taken by them before any consultation took place. Such an approach makes consultation meaningless.311

193.The quality of communication with magistrates about the reform programme has been criticised. The Magistrates Association has expressed concern about the JWOW consultation process; many magistrates were disappointed by the lack of detail in the published response, leaving them unclear as to whether the consultation process had influenced the reform programme.312 John Bache JP accepted that there were engagement mechanisms, including the Magistrates Engagement Group of which he was himself a member. However, he acknowledged that the answers given by HMCTS to the Group’s questions “are not always what we would like to hear.”313

194.The Judicial Intranet is an important means of communication with magistrates. The Magistrates’ Leadership Executive (MLE) thought that there had been difficulties in keeping this up to date, as a result of which “most magistrates have little idea what reform will look like for them and little appreciation of the value it will bring.” The MLE suggested that the magistracy “has different communication needs to the rest of the judiciary”, and that a new approach was needed to engage magistrates and persuade them of the benefits of reform.314

195.The Lord Chief Justice said that he felt “disappointed and, frankly, concerned” that magistrates felt that their communication needs were not being met. He thought that feedback from magistrates into the Judicial Ways of Working project “has profoundly influenced the product of those exercises”, and stressed the important role of the Magistrates Engagement Group. He said the Judicial Office has a new, dedicated communications team to deal with reform, and in his assessment, much information was available on the intranet. However, he accepted that “the very fact that the concern has been expressed—I appreciate that it has been expressed—leads me to conclude that we have to look at that again.”315

196.Our evidence suggests that HMCTS has struggled to explain its vision for the court and tribunal reform programme. Given the programme’s constitutional, strategic and operational significance, we recommend that the Lord Chancellor and Secretary of State for Justice takes clearer ownership of the programme and assumes the lead in communicating its vision.

197.Consultation and engagement should never be mere “lip service.” Early and effective engagement with stakeholders including judicial office holders at all levels is critical to the programme’s success, because this provides external expertise and detailed scrutiny of untested proposals.

198.We recommend that the Ministry of Justice and HMCTS increase the resources dedicated to stakeholder engagement and adopt a more rigorous approach to analysing and reacting to the feedback received.

Evaluation of the reforms

199.HMCTS has started a significant research and evaluation programme, which “continually seeks the views of the people who use the courts and tribunals system and develops insight from its findings.”316 The purpose of this evaluation will be to understand the effect of the reform programme as a whole by answering three principal questions:

200.HMCTS expected to have completed scoping work for its evaluation programme by Spring 2019, setting out detailed questions that will underpin an assessment of the three principles; what data is collected and what further information is needed; and how it will evaluate the effect of the reforms on vulnerable users.

201.The MoJ has acknowledged that evaluation is likely to raise two specific challenges:

202.Richard Goodman, Change Director at HMCTS, explained that evaluation of the programme had several levels. Overarching evaluation was being undertaken outside HMCTS by the MoJ; this would be supported by an independent advisory panel of academic experts.318 The other aspects of the evaluation “are happening all the time in the background”; for example, the independent evaluation of the fully video hearings in the Tax Tribunal that was conducted by the London School of Economics.319

203.Some who submitted evidence to the inquiry had little confidence in the MoJ’s evaluation of reforms so far, or in its proposals for future evaluation; we have already noted these concerns in relation to video hearings and video links (see Chapter 3). A typical response came from Harriet Bosnyak from Shelter:

There does not seem to have been much evaluation of what has gone on before. What has happened? Are people struggling to make it to the courts because there have been so many court closures? Are people actually using the online processes that are already there? [ … .] If we are pushing forward further reform, how do we know whether it is going to have the effect we want it to have?320

Others expressing lack of confidence in MoJ’s evaluation included the Legal Committee of HM Council of District Judges (Magistrates)321 and the Prison Reform Trust.322

204.Some witnesses expressed a degree of cynicism. Referring to HMCTS’s “Post Implementation Review” of the decision to close all but one county court in Greater Manchester, the Association of HM District Judges pointed out that no local or national impact assessments had been provided to the review panel, and reported that one participant judge had summarised the review’s apparent purpose as being to consider “what further changes might we make …… to achieve greater efficiencies from the diminishing staff and judiciary without spending any money?”323 Manchester Law Society Civil Litigation Committee had seen no evidence that significant steps had been taken to evaluate the reforms and had the impression that “pilots are run and then adopted without any significant change, irrespective of how the [legal] profession believes the pilot operated.”324 The Magistrates’ Leadership Executive thought that pilots benefited from the input of extra resources for short periods, leading to false outcomes.325 There have been reports that the published version of research conducted for the HMCTS Customer Insight Team on user experiences of the justice system omitted findings that suggest people have a more positive experience of the justice system if they attend court in person.326

“Agile” design technique

205.In developing processes for specific projects within the court and tribunal reform programme, HMCTS is using new “agile” design techniques, pioneered in the UK by the Government Digital Service. This involves iterative testing with user groups to support the design of processes, developing and revising them on an ongoing basis in response to continuous user feedback. HMCTS considers that this gives opportunities to test, refine and improve each change, rather than bringing everything together at a single point at the end.

206.Although there was support for the principle of user engagement, some witnesses raised concerns about the “agile” approach.327 Dr Joe Tomlinson cautioned that, in placing emphasis on convenience and what users appear to want, care must be taken not to overlook traditional concerns such as procedural fairness.328 Transform Justice argued that unpublished “user research” of this type, designed to help product design, “does not meet any of the guidelines essential for academic research and does not conform to the protocol on the publication of government social research.”329 The likelihood that people using HMCTS prototypes lack expert legal knowledge, and thus do not know when they need more information, was considered problematic by Amanda Finlay. She was concerned about HMCTS teams developing projects in isolation rather than working on an “end to end” process.330

207.The Senior President of Tribunals, Sir Ernest Ryder, thought it inevitable that an agile design technique would involve evaluation of the constituent parts of the programme. However, we were pleased that he emphasised the importance of evaluating the reforms by reference to access to justice principles—an approach also supported by the Administrative Justice Council.331

The views of academics

208.Dr Joe Tomlinson, then of Kings College London, provided a detailed literature review for the Committee, giving an overview of key recent pieces of academic literature relevant to our inquiry terms of reference.332 His review revealed “a dearth of concrete empirical evidence of the performance of online dispute resolution (ODR) and related technologies (such as video link hearings)”; this meant that much published material was best characterised as “sophisticated speculation and analysis but without an empirical evidence base”—although with some notable exceptions. Dr Tomlinson’s review assisted us in distilling some of the more robust research findings that we refer to in our report. We were particularly interested to hear that there is a growing literature on the evaluation of justice processes and related concepts, including access to justice, including the authoritative handbook produced by the Tilburg Institute for Interdisciplinary Studies of Civil Law and Conflict Resolution Systems.333 We also received submissions from several academics on approaches to evaluating the reforms, five of whom (in addition to Dr Tomlinson) attended an informal evidence session to discuss how the Government should ensure proper and effective evaluation of the programme.

209.Dr Natalie Byrom, on the basis of research conducted on secondment to HMCTS in 2019, makes recommendations for principles for evaluating the impact of reform on access to justice, including:

210.We regret that the Ministry of Justice’s plans for evaluating its court and tribunal reforms are not as far advanced as might have been expected at this stage in the programme. We are concerned that reliance on “agile” design techniques in some projects may distract from analysis of implications for access to justice. The Ministry could do much more to evaluate the impact of the reforms on vulnerable and excluded groups.

211.The Ministry of Justice should be as transparent as possible in its evaluation of the reforms. We recommend that the following evaluation approaches are adopted:

290 House of Commons Committee of Public Accounts. Transforming courts and tribunals, Fifty Sixth Report of Session 2017–19, 20 July 2018, HC 976, P6

291 HMCTS, Engaging our external stakeholders, November 2018

292 For example, Crown Prosecution Service (CTS0074); Victim Support (CTS0003); Women’s Aid (CTS0006); JUSTICE (CTS0068)

293 Including Centre for Justice Innovation UK (CTS0008); Criminal Justice Alliance (CTS0050); Right2Justice (CTS0054); The Criminal Bar Association (CTS0063); Transition 2 Adulthood (T2A) Alliance (CTS0037); Revolving Doors Agency (CTS0073)

294 Ms Diane Astin (CTS0066)

296 The Law Society of England and Wales (CTS0040)

297 Chartered Institute of Legal Executives (CTS0049)

298 Public Law Project (CTS0027). Similar complaints were made by other witnesses including Rhona Friedman, Sue James and Simon Mullings (CTS0085) and Transform Justice (CTS0022)

299 Leeds Law Society (CTS0011); Housing Law Practitioners Association (CTS0090); Manchester Law Society - Civil Litigation Committee (CTS0004)

301 IPSEA (CTS0041)

302 Free Representation Unit (CTS0080)

303 Citizens Advice (CTS0016)

305 Public and Commercial Services union (CTS0010)

306 Together with the Tribunals Judicial Executive Board, the Judicial Executive Board is the most senior decision-making forums for providing a judicial view on design or implementation questions relating to the reform programme.

307 Lord Chief Justice of England and Wales (CTS0078)

308 The Administrative Justice Council is the successor body to the Administrative Justice Forum which was abolished in April 2017.

309 Council of Her Majesty’s Circuit Judges (CTS0091)

310 Legal Committee of Her Majesty’s Council of District Judges (Magistrates’ Court) (CTS0032)

311 The Association of Her Majesty’s District Judges (ADJ) (CTS0084)

312 MAG0001. In evidence to this inquiry, Captain Hugh Daglish JP (CTS0001) referred to “plenty of anecdotal evidence of worry and discontent” among magistrates, many of whom fear that the reform programme will have a detrimental effect on justice

314 Magistrates` Leadership Executive (CTS0017)

316 HM Courts & Tribunals Service (CTS0064). The approach to evaluating the HMCTS reforms is set in the MoJ’s Response to the Public Accounts Committee recommendation 4: Evaluating our reforms, January 2019

318 To date, the membership of the advisory panel has not been announced.

321 Legal Committee of Her Majesty’s Council of District Judges (Magistrates’ Court) (CTS0032)

322 Prison Reform Trust (CTS0046)

323 The Association of Her Majesty’s District Judges (ADJ) (CTS0084)

324 Manchester Law Society - Civil Litigation Committee (CTS0004)

325 Magistrates` Leadership Executive (CTS0017)

326 See blog by Transform Justice (March 24, 2019) and report in Buzzfeed News (March 18, 2019). The research was published as HM Courts & Tribunals Service Citizen User Experience Research. HMCTS Customer Insight Team, 2018

327 JUSTICE (CTS0068))

328 Dr Joe Tomlinson (CTS0092)

329 Transform Justice (CTS0022)

330 Ms Amanda Finlay (CTS0055). Similar concerns were raised by JUSTICE (CTS0068), Law Centres Network (CTS0081) and Citizens Advice (CTS0016)

332 Dr Joe Tomlinson (CTS0092)

Published: 31 October 2019