- Children, Schools and Families Committee Contents


Memorandum submitted by the National Foundation for Educational Research

EXECUTIVE SUMMARY

  1.  NFER is Britain's leading independent educational research institution and is submitting this paper to the Children, Schools and Families Select Committee to inform its inquiry into school accountability.

2.  The evidence presented by the Foundation is based on work commissioned by a number of clients wishing to examine elements of the school accountability system. The findings portray a relatively positive attitude (from schools) to the new accountability mechanisms and suggest a number of areas where further attention should be focused. The key findings from our work are summarised below.

    — That schools accept the principle of accountability and would largely expect to be held accountable.

    — Questions remain about the extent of and audience for accountability measures.

    — Section 5 inspections are seen as "fit for purpose".

    — Schools value inspection as a tool for school improvement.

    — That the perceived burden of inspection has reduced on moving from Section 10 (S10) to Section 5 (S5) inspections.

    — An independent inspectorate has a positive influence on school standards.

    — Schools would welcome a greater emphasis on lesson observation and detailed feedback rather than an undue emphasis on data to aid school improvement.

    — School self-evaluation, whilst time-consuming, is seen by schools as a good way of identifying strengths and weaknesses.

    — Data-led interpretation of school performance must take into account the circumstances of the particular school: Every Child Matters has brought this more sharply into the foreground.

    — In a minority of cases, schools feel that inspection findings are not specific or can lead to direct action to address concerns raised by inspectors.

INTRODUCTION

  3.  The Children, Schools and Families Select Committee is conducting an inquiry into school accountability. This submission by the National Foundation for Educational Research (NFER) aims to provide evidence and recommendations that we hope will support the committee in its inquiry.

4.  NFER is Britain's leading independent educational research institution. The Research, Evaluation and Information Department is one of two research departments at the NFER, and specialises in providing high-quality, independent research and evaluation in all areas of education and training. The Department for Research in Assessment and Measurement is the second research department of the Foundation. It specialises in test development and research into assessment-related questions. This submission draws solely on the Foundation's experience with respect to research and evaluation concerning school accountability issues. Further information about the NFER is provided in the accompanying NFER Credentials document.

RESEARCH CONDUCTED BY THE NFER

  5.  The following sections take the lead from the structure set out in the call for evidence by this Select Committee. The evidence submitted by NFER covers only the elements pertaining to Accountability and Inspection.

ACCOUNTABILITY

Is it right in principle that schools should be held publicly accountable for their performance?

  6.  Research carried out by NFER suggests that it is indeed right that schools should be held publicly accountable for their performance. Indeed, evidence collected from school respondents across various projects indicates that school staff themselves would very largely accept this principle.

7.  Issues arise, however, when questions are raised about the extent of this accountability and to whom schools should be accountable (see below). Most school personnel would stress strongly that their accountability should be based on valid and reliable data and upon comprehensive information about the full circumstances of the school, including its geographical and socio-economic context. In other words, if the information used to judge a school's performance is comprehensive and fair, then a school would accept this judgement. Evidence supporting these statements is provided below in the findings from our research on school inspections.

What should be the fundamental purposes of an accountability system for schools and, in particular?

    to whom should schools be accountable?for what should they be held accountable?

    how should they be held to account? and

    what should be the consequences?

  8.  The accountability system should have three fundamental purposes: (1) to ensure that there is a degree of regular external accountability; (2) to ensure that areas for improvement are identified; and (3) to ensure that schools are provided with appropriate support and guidance on how to improve.

Is the current accountability system of inspection and performance reporting for schools broadly fit for purpose?

  9.  Given that a majority of schools are satisfied with the Section 5 (S5) inspection process (see below) it would seem that this process is broadly fit for purpose.

INSPECTION

  10.  Since 2006 a research team at NFER has had a unique opportunity to carry out a detailed and independent evaluation of the new inspection process. The central aim of the research has been to assess the extent to which schools feel that the new inspections have contributed to school improvement. The evaluation, commissioned by the Office for Standards in Education, Children's Services and Skills (Ofsted), commenced in February 2006 and will be completed in March 2009. The research methods included:

    — A survey of all schools inspected between October 2005 and March 2006 (subject to minor exclusions): 1,597 schools responded to this survey—a 67% response rate.

    — Case-study visits to 36 schools where interviews were conducted with headteachers, senior managers, governors and parents, usually individually, along with 243 pupils, usually in small discussion groups.

    — Follow-up interviews with headteachers or senior managers, to provide a longitudinal perspective.

    — Statistical modelling of survey responses and various school background factors, using satisfaction with, and perceived impact of, inspection as outcomes.

    — A desk-top review of key case-study school documents and test and examination results.

  11.  The key finding from this evaluation has been that, overall the vast majority of schools are satisfied with the inspection process and this process is generally perceived as contributing to school improvement. Over half of the schools surveyed (52%) were "very satisfied" with the inspection and more than a third (36%) were "quite satisfied". A minority, 10% of survey schools, were "not at all satisfied" with the S5 inspection.

  12.  A substantial majority of survey respondents (83%) thought that the actual monetary costs incurred due to inspections were minimal and were certainly less than those incurred during the previous S10 inspection process. Furthermore, three-fifths of school respondents thought that the new S5 inspection process was less stressful than the previous system. Further evidence and findings from this evaluation are presented at the relevant points below.

Is an independent inspectorate an appropriate mechanism for holding schools to account?

  13.  It is difficult to see how schools could be accountable without some form of independent inspection. The NFER believes that the withdrawal of an independent element of inspection or accountability checks would be likely to lead to a decline in academic standards in some schools.

14.  Furthermore, despite the largely negative impression of inspection experiences, based mainly upon anecdotal and personal opinions, evidence collected by the NFER clearly indicates that schools themselves are largely satisfied with, and understand the need for, independent external inspections.

What is the impact of the inspection process on school performance, including confidence, creativity and innovation?

  15.  The impact of the inspection process on school performance is largely indirect, but nonetheless important. This is because there are many drivers of school improvement, including the school's management approach, school improvement or development planning, the use of data, and the impact of specific educational initiatives. The main benefit of inspection was perceived to be that the inspection had been valuable in providing external confirmation of schools' own self evaluation (86% of survey respondents took this view). Additionally, other benefits were perceived to be that inspection boosted staff morale (42%) and, as well as providing confirmation, it also stimulated improvements (33%). In these respects inspections do contribute to confidence, creativity and innovation.

16.  Although schools were generally satisfied with the inspection, just under half made suggestions for changes. These most often related to perceptions that the (self-evaluation form) SEF should be simplified, more time should be allowed for inspectors to observe lessons, inspections should be less data-driven and that there should be more consistency across inspection teams.

Are inspectors appropriately qualified and trained to carry out inspections, particularly in the light of the need to report against Every Child Matters outcomes?

  17.  With respect to schools inspections up to the present time, the evidence collected by the NFER indicates that those who have been inspected have a very positive view of the professionalism and qualifications of inspectors.

18.  Oral feedback from the inspection team was found to be a vital part of the inspection process. Over half (60%) of the survey respondents found the oral feedback very useful, and further 32% fairly useful. The research also revealed a positive, and statistically significant, relationship between constructive oral feedback and overall satisfaction with the inspection process.

Is it appropriate for inspection reports to be placed in the public domain?

  19.  The majority of survey respondents and interviewees agreed with the inspection report recommendations and valued the contribution to school improvement in terms of the confirmation, prioritisation and clarification of areas for improvement. Over three-quarters of case-study school interviewees believed the inspection team's diagnosis, and the written report, to be fair and accurate. Over half of survey respondents found the written report helpful for identifying areas for improvement. The majority of interviewees found the written report to be useful, helpful and easy to read. A few interviewees believed the report to be too brief and generalised. However, it was perceived to be accessible and parents appreciated an independent assessment of schools.

20.  On the whole pupils liked the letter from the inspector and valued involvement in the process. Recommendations for improvement were, on the whole, considered to be helpful and sufficiently specific and follow-up interviews showed that almost all case-study schools were implementing all, or most, of their recommendations.

  21.  The area of greatest perceived impact, from the S5 recommendations, was in assessment, monitoring and tracking. Nearly two-thirds of survey respondents and just over half the case-study interviewees considered that the inspection had contributed to school improvement. The main way it had contributed was by confirming, prioritising and clarifying areas for improvement, rather than by highlighting new areas.

  22.  Above all, schools recognised that however reliable their own self evaluation was, it was useful for parents and the local communities, as well as for their own staff, to have their judgements confirmed by an external and objective body.

How often should inspections be carried out and how long and detailed should these inspections be?

  23.  There needs to be a careful balancing act here. There appears to be more support from schools for the five-year cycle with shorter inspections and concise reports, as opposed to the three-year cycle with longer inspection visits and more detailed reports. However, school respondents have told us that they would like more lesson observations and that they appreciated detailed feedback and discussion.

24.  On this basis, the current Section 5 approach, with a three-year cycle is appreciated by schools—if the visits required by this approach were to be made more detailed, then any expansions should be in the observation (which could mean more teachers or more subjects) and feedback elements.

How much notice, if any, should a school receive of an upcoming inspection?

  25.  The shorter notice system is more popular among schools than the previous Section 10 approach, with a longer period of notice.

In the context of an inspection, what is the value of:

    the school's self-assessment;the results of national tests and;

    the school's contextual value added scores.

  26.  All of these are clearly important parts of inspection considerations. School self-assessment and school self-evaluation have grown dramatically in importance in the last 10 to 15 years. So much so that they now have an official and substantial place in the inspection process, in the form of the SEF and all stakeholders now appear to appreciate the usefulness of this.

  27.  Although the majority of interviewees reported that it was time-consuming to complete the SEF, there was also a strong view that the SEF had been effective as a means of identifying school strengths and weaknesses. Inspection teams made good use of the SEF and it provided a focus for the inspection. Self evaluation generally was regarded as having improved and the SEF framework had contributed to this improvement.

  28.  In the first year's use of the SEF, school staff complained about the time that was required to fill the SEF in, but they also recognised how important this was and how useful the process is as a means of "getting to know your school". Schools welcome the fact that they can now present their own data and that the inspection team should take due account of this information. The use of self-evaluation is a crucially important part of the dialogue that takes place between the school managers and the inspectors.

  29.  The NFER has evidence to show that school self-assessments mostly tend to be in line with external assessments, suggesting that, in the main, schools are realistic and sensible in their self-judgements. Filling in the SEF required schools to make their own self-assessment of their "overall effectiveness" and a number of other categories such as "achievement and standards".

  30.  School survey respondents were asked to compare their own SEF grades with those awarded by the inspectors: two-thirds of survey respondents reported no differences between the S5 and school's SEF grades, indicating a large degree of consistency between the two sets of judgements.

  31.  The results of national tests are also clearly of importance to inspection as they provide an objective assessment of pupils' progress at certain stages in their school careers. One of the disadvantages of "raw" test results, however, is that they do not take account of the effects of prior attainment, gender and the socio-economic background of pupils. Value added and contextual value added scores can take account of these factors, and schools generally welcome their use, though there have been some concerns about the way school inspectors have used both "raw" and "value added" data.

…and how much weight should be attached to these elements in the inspection report?

  32.  A common complaint from schools (in a context of mostly positive views about the Section 5 inspections) in our research was that inspection was "too data driven". Clearly attainment data needs to feature significantly in inspection considerations because they indicate a child's academic progress and the extent to which a school is performing the function of providing qualifications and an appropriate academic education.

33.  However, many school staff would argue firstly that there has been too much emphasis on the attainment inspection grade, at the expense of other inspection grades and, secondly, that in the context of ECM and the Children's Plan, a school's provision is now about much more than academic qualifications, indeed it is about the five outcomes and creating responsible citizens and much more besides.

In an inspection, how should emphasis be balanced between educational attainment and other aspects of a school's provision, such as the Every Child Matters outcomes?

  34.  It would seem that attainment indicators are universal and should be maintained, but not to the exclusion of other indicators. The time is now right for other indicators to receive greater emphasis and for more sophisticated indicators of pupil progress and school improvement to be developed. Assessment has to become more sophisticated in the 21st century.

Should inspections be tailored to the current performance levels of the specific school being inspected and, if so, to what extent?

35.  The tradition in local authority intervention and support for schools is, of course, intervention in "inverse proportion to success". With regard to inspections there seems to be general agreement that low performing schools benefit from more regular inspections, more detailed inspections and customised support from the local authority and others. With respect to high performing schools the picture is not so clear.

OTHER ELEMENTS OF INSPECTION

  36.  In the work undertaken by NFER, a number of other factors have been identified as being important to schools and should be shared with the Committee. These are described below.

USE OF DATA

  37.  Concern was expressed over data interpretation by some schools inspected throughout the period from October 2005 to March 2006. This suggests that there may be a need for more evenness and consistency in terms of the way data is used, particularly in relation to fully understanding the school context. With the introduction of RAISE online, there should be opportunities to ensure that consistencies in data use and interpretation are further promoted and strengthened.

IMPORTANCE OF ORAL FEEDBACK AND DIALOGUE

  38.  In view of the importance that schools placed on the oral feedback, Ofsted should maintain and perhaps even enhance the central position which oral feedback has in the inspection process. Schools appeared to welcome the opportunity for, and were responsive to, dialogue, especially as these conversations provided opportunities to explain the broader school context.

USING POSITIVE TERMINOLOGY

  39.  Though it was widely accepted that inspectors had to work within the agreed standard framework of gradings, there was some dissatisfaction in schools with the terminology used to describe the "overall effectiveness" grades, especially in relation to the "satisfactory" grading. Whilst it was accepted that parents and other stakeholders should be provided with a clear, comparable, external, objective assessment of a school's performance, some school interviewees expressed a view that the terminology was too negative and too rigid. Several respondents suggested that inspectors should look for further ways of providing praise and encouragement for staff: and it might be possible to do this through the oral feedback and the lesson observation elements of the inspection process.

REFINING INSPECTION RECOMMENDATIONS

  40.  The vast majority of survey respondents agreed with the inspection recommendations, found them helpful, and felt that they were sufficiently specific. Only one in 10 schools found the recommendations "not at all helpful". Where this latter view was present, the reasons were usually along the lines of: (1) the recommendations were not specific enough; or (2) the recommendations lacked practical guidance. It might be worth bearing these two points in mind when any further advice on drafting recommendations is given to inspectors.

What is the impact on schools of league tables published by the press?

41.  The importance of terminology and function is worth re-stating here. The Department for Children, Schools and Families (DCSF) produces school and college achievement and attainment tables, free of any attempts to rank schools or colleges. The transformation of these data into league tables published by the press tend to cause more harm than good. This is particularly true of tables based on raw results. Such information can be very damaging to a school which in fact is doing well and adding value to pupils' experience by supporting them to achieve better than expected test or examination results.

CONCLUSIONS

  42.  A range of research and evaluation exercises have been carried out by NFER focusing upon the school accountability system in England. The predominant findings are about the positive impact which having an independently guided inspection system which has a clear focus on school improvement is welcomed by schools. There are undoubtedly challenges in squaring a simple and low-burden system with the increasingly complex range of activities which schools are expected to be accountable for (emphasised by the scope of Every Child Matters). The research which NFER has undertaken is broadly supportive of the changes from Section 10 to Section 5 inspections.

February 2009

SOURCESArnold, R (1998). Target Setting: School and LEA in Partnership. Slough: NFER, EMIE.

Arnold, R (2000). Education Strategic Plans in Wales: "Building Excellent Schools Together" (EMIE Report No.58). Slough: NFER, EMIE.

Arnold, R (2002). The School Improvers: Survey of LEA Advisory, Inspection, and School Improvement Services (EMIE Report 65). Slough: NFER, EMIE.

Arnold, R (2002). Good Practice in the Midlands: a Report Prepared for the Society of Midlands Education Officers. Slough: NFER, EMIE.

Arnold, R (2004). Transforming Secondary Education: the Beacon Council Scheme. Round 4 (EMIE Report No.78). Slough: EMIE at NFER.

Atkinson, M, Gulliver, C, Lamont, E and White, R (2006). The New School Funding Arrangements 2006-07: the Local Authority Perspective (LGA Research Report 2/06). Slough: NFER.

Atkinson, M, Lamont, E, Gulliver, C, White, R and Kinder, K (2005). School Funding: a Review of Existing Models in European and OECD Countries (LGA Research Report 3/05). Slough: NFER.

Atkinson, M, Springate, I, Johnson, F and Halsey, K (2007). Inter-School Collaboration: a Literature Review. Slough: NFER [online]. Available: http://www.nfer.ac.uk/publications/pdfs/downloadable/ATPreport.pdf [27 February, 2009].

Benton, T (2003). "Measuring schools' value-added performance: the importance of picking the right outcome." Paper presented at the British Educational Research Association Annual Conference, Heriot-Watt University, Edinburgh, 12 September [online]. Available: http://www.nfer.ac.uk/publications/other-publications/conference-papers/measuring-schools-value-added-performance-the-importance-of-picking-the-right-outcome.cfm [27 February, 2009].

Benton, T, Hutchison, D, Schagen, I and Scott, E (2003). Study of the Performance of Maintained Secondary Schools in England: Report for the National Audit Office. Slough: NFER. [online] Available: http://www.nfer.ac.uk/publications/other-publications/downloadable-reports/pdf_docs/NAO.pdf [27 February, 2009].

Chamberlain, T, Lewis, K, Teeman, D and Kendall, L (2006). Schools' Concerns and their Implications for Local Authorities: Annual Survey of Trends in Education 2006 (LGA Research Report 5/06). Slough: NFER.

Davies, D and Rudd, P (2001). Evaluating School Self-evaluation (LGA Research Report 21). Slough: NFER.

Derrington, C (2000). The LEA Contribution to School Improvement: a Role Worth Fighting For (LGA Research Report 9). Slough: NFER.

Easton, C, Knight, S and Kendall, L (2005). Annual Survey of Trends in Primary Education: Survey of 2004 (LGA Research Report 8/05). Slough: NFER.

Easton, C, Knight, S and Kendall, L (2005). Annual Survey of Trends in Primary Education. Digest No. 17: Survey of 2004. Slough: NFER.

Fisher, D (1999). Partnership for Progress: Support for Underachieving Schools. Slough: NFER, EMIE.

Fowler, J and Waterman, C (2005). Digest of the Education Act 2005. Slough: NFER.

Hackworth, C and Barson, A (2007). School Improvement: Beacon Scheme Round 8 (EMIE Report 89). Slough: EMIE at NFER.

Halsey, K, Judkins, M, Atkinson, M and Rudd, P (2005). New Relationship with Schools: Evaluation of Trial Local Authorities and Schools (Research Summary). [online]. Available: http://www.nfer.ac.uk/research-areas/pims-data/summaries/new-relationship-with-schools-evaluation-of-trial-leas-and-schools.cfm [27 February, 2009].

Harland, J, Kinder, K, Lord, P, Stott, A, Schagen, I, Haynes, J, Cusworth, L, White, R and Paola, R (2000). Arts Education in Secondary Schools: Effects and Effectiveness. Slough: NFER.

Hendy, J (1998). Fourth Survey of LEA Advisory and Inspection Services. Slough: NFER, EMIE.

Knight, S, Taggart, G and Kendall, L (2006). Annual Survey of Trends in Secondary Education: Report for 2005 (LGA Research Report 1/06). Slough: NFER.

Mann, P (1995). Third Survey of LEA Advisory and Inspection Services. Slough: NFER, EMIE.

McCrone, T, Rudd, P, Blenkinsop, S and Wade, P (2006). Impact of Section 5 Inspections: Maintained Schools in England [online]. Available: http://www.ofsted.gov.uk/Ofsted-home/Publications-and-research/Browse-all-by/Documents-by-type/Thematic-reports/Impact-of-Section-5-inspections-maintained-schools-in-England [27 February, 2009].

McCrone, T, Rudd, P, Blenkinsop, S, Wade, P, Rutt, S and Yeshanew, T (2007). Evaluation of the Impact of Section 5 Inspections. Slough: NFER.

National Foundation for Educational Research (2002). Evaluation of Excellence in Cities Policy: Updated June 2002. Slough: NFER.

Pritchard, K (2005). National Standards for School Improvement Professionals: the Legacy and the Future (EMIE Report No.83). Slough: EMIE at NFER.

Rudd, P, Davies, D, Jamison, J, Rickinson, M and Johnson, R (2001). Beacon Schools: Further External Evaluation of the Initiative. Final Report. Slough: NFER.

Rudd, P and Davies, D (2002). A Revolution in the Use of Data? The LEA Role in Data Collection, Analysis and Use and Its Impact on Pupil Performance (LGA Research Report 29). Slough: NFER.

Rudd, P and Dartnall, L (2003). 50 Websites for School Improvement. Slough: NFER.

Rudd, P (2007). School Inspections: International Literature Review [online]. Available: http://www.nfer.ac.uk/research-areas/pims-data/outlines/school-inspections---international-literature-review.cfm [27 February, 2009].

Saunders, L and Stradling, B with Rudd, P (2000). Raising Attainment in Secondary Schools: a Handbook for School Self-Evaluation (New Updated Edition). Slough: NFER.

Saunders, L (1999). "Value Added" Measurement of School Effectiveness: a Critical Review. Slough: NFER.

Saunders, L (1998). "Value Added" Measurement of School Effectiveness: an Overview. Slough: NFER.

Savory, C, Walker, M and Rudd, P (2005). National and Local Government Raising Standards Across Schools: a Literature Review (LGA Research Summary 5/05). Slough: NFER [online]. Available: http://www.nfer.ac.uk/research-areas/pims-data/summaries/shared-priorities-raising-standards-across-schools.cfm [27 February, 2009].

Scanlon, M (1999). The Impact of OFSTED Inspections. Slough: NFER.

Schagen, I and Benton, T (2003). "The relationships between school and pupil background factors and progress from Key Stage 1 to Key Stage 2." Paper presented at the British Educational Research Association Annual Conference, Heriot-Watt University, Edinburgh, 13 September [online]. http://www.nfer.ac.uk/publications/other-publications/conference-papers/the-relationships-between-school-and-pupil-background-factors-and-progress-from-key-stage-1-to-key-stage-2.cfm [27 February, 2009].

Schagen, I and Weston, P with Hewitt, D and Sims, D (1997). Hitting the Targets: Evaluation of Progress Towards Foundation Targets 1 and 3 by English Region. Slough: NFER.

Schagen, S, Davies, D, Rudd, P and Schagen, I (2002). The Impact of Specialist and Faith Schools on Performance (LGA Research Report 28). Slough: NFER [online]. Available: http://www.nfer.ac.uk/publications/pdfs/downloadable/specialistandfaithschools.pdf [27 February, 2009].

Sharp, C, Handscomb, G, Eames, A, Sanders, D and Tomlinson, K (2006). Advising Research-Engaged Schools: a Role for Local Authorities. Slough: NFER.

Sturman, L, Lewis, K, Morrison, J, Scott, E, Smith, P, Styles, B, Taggart, G and Woodthorpe, A (2005). General Teaching Council Survey of Teachers 2005: Final Report. Slough: NFER [online]. Available: http://www.gtce.org.uk/research/tsurvey/2005survey [27 February, 2009].

Wilkin, A, Atkinson, M, Halsey, K, Johnson, A, White, R and Kinder, K (2005). New Roles for Local Authorities in Education: Opportunities and Challenges (LGA Research Report 9/05). Slough: NFER.

Wilson, R, Sharp, C, Hemsley-Brown, J and Easton, C (2002). Using Research for School Improvement: the LEA's Role (Interim Summary). Slough: NFER.

Wray, M (2002). Local Authority Reactions to the OFSTED Inspection Process (LGA Research Report 32) [online]. Available: http://www.nfer.ac.uk/publications/other-publications/downloadable-archive-reports/local-authority-reactions-to-the-ofsted-inspection-process.cfm [27 February, 2009]





 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2010
Prepared 7 January 2010