Services for young people - Education Committee Contents


Written Evidence Submitted by New Philanthropy Capital

INTRODUCTION

NPC's Well-being Measure is an online survey-based tool that measures how young people feel and what they think about their lives. It is a useful tool for youth groups and schools for quantifying "soft outcomes" and understanding how much young people enjoy life.

It measures eight aspects of well-being: self-esteem, emotional well-being, resilience, satisfaction with friends and family, satisfaction with school and community, and overall life satisfaction.

By doing the survey at two points in time, it can measure change-and be used by charities, schools or youth groups that want to evaluate the impact of their work. All results are benchmarked against a national sample of young people.

The Well-being Measure is designed to be used with a group of ten of or more young people aged 11 to 16. It is not valid for measuring the well-being of individuals.

WHAT THE WELL-BEING MEASURE CAN DO

The tool can be used to:

(a) measure the changes in a group of children between two points in time-for example, to test whether a youth project has an impact on young people's well-being.

(b) compare a group of children to a national baseline-for example, for schools to see how their students compare to other young people in the UK.

(c) look for patterns within a group-for example to compare whether well-being differs between boys and girls in a group, or to see whether an intervention has a different impact on young people eligible for free school meals compared to those that are not.

All results are available in a downloadable report once the survey is finished. Organisations can also do their own custom analysis by using our online filtering tool.

USING THE WELL-BEING MEASURE

Organisations sign up and create their own surveys. Although surveys are based around a standard set of questions, they can be customised by selecting which areas of well-being to include and adding questions if required. Once a survey is launched, young people are given a unique code and complete it online, which usually takes around 10 to 15 minutes.

In the survey, young people are asked to respond to a series of statements about their lives, and must say how they feel on a five-point scale (ranging from strongly agree to strongly disagree). Individual responses are then aggregated to produce an overall measure of well-being in each of eight areas. The final results for the group are presented as a percentage score from 0 to 100, which reflects well-being in the context of other young people in the UK. For example, a score of 30% on self-esteem, means that 30% of the national population has lower self-esteem than your group and 70% of the national population has higher self-esteem than your group. The national population figures is drawn from a sample of 2,000 young people from across UK that have completed the survey. Over time this sample will grow, increasingly analytical capability of the tool.

Once a survey is closed, organisations receive a detailed report of their results, and can do some of their own analysis using our online filtering tool.

Organisations that want to measure change can then do a "follow-up" survey. A second report is generated once this is complete and shows how the well-being of participants has changed.

More information on the Well-being Measure is contained in the appendices. Appendix 1 shows the questions in the basic survey. Appendix 2 describes the national baseline sample.

THE CURRENT SITUATION

At present, NPC's Well-being Measure has been released to a limited number of customers. We are limiting the number so that we can test how much support organisations require to use it and make any necessary adjustments. We will be gradually building up subscribers until our full launch at the end of 2011.

Current customers include: BBC Children in Need, Action for Children, Body and Soul Charity, Wellington College, Depaul UK, Save the Children UK, ContinYou, The Outward Bound Trust, Brathay, and the Specialist Schools and Academies Trust.

COSTS

Customers pay for the amount they use and payment is based on buying "survey credits".

One credit buys a single survey of up to 200 young people. Each survey includes a report of results.

Running one survey allows users to measure the well-being of participants relative to the national baseline. Running two surveys (an "initial" survey + a "follow-up survey") allows users to measure the change in well-being, and see what impact their programme has on young people.

The costs of survey credits are below. There are discounts for multiple credits bought in the same transaction.
Number of credits CostWhat it buys
1£295 + VATA single survey of up to 200 participants
2£475 + VATAn initial and follow-up survey of up to 200 participants
10£2,000 + VAT 5 x Initial and follow-up surveys of up to 200 participants
20£3,600 + VAT 10 x Initial and follow-up surveys of up to 200 participants

Surveys of more than 200 young people require more credits. Each credit allows you an extra 200 participants.

HOW NPC'S WELL-BEING MEASURE WAS DEVELOPED

The Well-being Measure has been developed by NPC over more than three years. There were two phases in its development: creating the survey and designing the online tool. Each is described here.

Creating the survey

We began with a thorough literature review to identify all the existing work on measuring well-being. Using this as a basis we created our well-being scales using the "best bits" from other studies. Our scales all use questions adapted from work by academics and psychologists in the UK and US.

The survey was developed with five charities and in 13 schools. It was tested with over 850 young people and met stringent statistical benchmarks for validity, reliability, and internal consistency. The tests were also designed to ensure that the survey was practical to use, that young people understood it and reacted well to it. The survey went through many revisions to meet these criteria.

During the process we had a steering group of academics and practitioners to ensure the quality and robustness of our methods. You can read about the development of the survey in our 2009 report Feelings count (available on the NPC website).

This phase was entirely funded by NPC, using our charitable funds.

Designing the online tool

Having developed the survey, we wanted to make sure that it was in a practical format where it could be used by charities and schools that are not experts in evaluation methods or statistical analysis.

Working with the IT developers at Public Zone (www.publiczone.co.uk), we designed a website that allows users to sign-up, administer surveys and receive their results. Within this site, data is entered online and analysis and reporting is done automatically, making it easy to use.

The website was developed with feedback from a group of 11 organisations, including charities and schools. This group helped us to design the system, determine its requirements and test the tool. We also consulted with other experts in the field of evaluation and drew on other experience within NPC.

This phase was funded by a combination of NPC, The Private Equity Foundation and the Paul Hamlyn Foundation.

FUTURE POTENTIAL

NPC's mission is to improve the effectiveness of the charitable sector so we have designed the tool with this in mind. The data from the Well-being Measure potentially has a number of exciting applications, as described below.

Understanding the well-being of young people across the UK

NPC collects anonymous data from all users of the Well-being Measure. As more people use the tool, our database will grow and provide a richer source of information.

We collect postcode information from young people which will allow us to do detailed analysis using household level classification. For example, we will be able to compare the well-being of girls on inner city estates in south London to girls in rural Cumbria.

Making the links between soft and hard outcomes

The relationship between "soft" outcomes for young people, such as self-esteem and emotional well-being, and "hard" outcomes, such as exam results or success in the labour market, is poorly understood. Using the well-being tool, there is scope to understand that link.

We could potentially link up our database on well-being with other datasets on progression to further education or employment, and see whether there is a correlation between any of the aspects of well-being. This will help us to answer the question, "Which aspect of well-being has the greatest impact on children's employment prospects?" Is it self-esteem, relationship with peers, or resilience, and therefore which interventions are most successful?

Understanding and comparing what works

For each survey conducted using the Well-being Measure, NPC collects basic information on the type of project it is evaluating. Over time, as we refine this process and collect more data, we may be able to compare different types of intervention to see what works. For example, we could compare evaluations looking at mentoring projects with those looking at counselling to see how the outcomes differ.

Developing new measures, and measures for different groups

Our online tool could be extended to include more areas of well-being. It is currently limited to eight aspects of subjective well-being but we could add more, for example on diet, exercise or behaviour. The tool could also be extended to other groups of people, for example different age groups.

Developing and testing new measures to extend the tool would require significant investment.

APPENDIX 1:

THE WELL-BEING SURVEY

See attached the questions included in the basic survey.

AREAS OF WELL-BEING COVERED BY NPC'S WELL-BEING MEASURE
Aspect of well-beingDescription
Self-esteemA child's appraisal of his or her own worth. It is closely linked with self-confidence, and is important for a healthy, happy life.
Emotional well-being The state of a child's mental health or extent to which a child experiences depressive emotions, as well as worries and other stressful feelings. Low scores are linked to anxiety and depression.
ResilienceThe capacity to cope with stress and difficulties. It involves a positive and purposeful attitude to life and is associated with high self-esteem and interpersonal problem-solving skills. It is a particularly important protective factor to foster in children, enabling them to deal better with future negative events.
Satisfaction with friendsThe child's satisfaction or feelings about the quality of his or her close friendships both in and out of school.
Satisfaction with familyThe child's satisfaction or feelings about his or her family relationships, including the quality and quantity of time spent with parents or carers, and how well the family gets on.
Satisfaction with communityThe child's satisfaction or feelings about his or her local area and neighbours or people in the community.
Satisfaction with schoolThe child's satisfaction or feelings about his or her school environment, including how enjoyable and interesting it is, and how safe it feels.
Life satisfactionA global measure of a child's overall happiness or satisfaction, based on a single question where the child rates his or her life on a scale from 1 to 10.

APPENDIX 2:

THE NATIONAL BASELINE SAMPLE

What is the national baseline?

The national baseline is a sample of young people across the UK that have completed the well-being survey.

How is the national baseline used in NPC's Well-being Measure?

It is used in all the graphs and statistics generated in NPC's Well-being Measure to put results in context.

For example, if your results show a score of 30% on self-esteem, it means that 30% of the national population has lower self-esteem than your group and 70% of the national population has higher self-esteem than your group.

In each graph, the national baseline is adjusted to account for the age and gender of respondents, so you can be sure that your results are not biased. For example, in a graph that shows a group of young people consisting of 40% boys and 60% girls aged 11 and 12, the national baseline is adjusted to reflect this.

How is the national baseline constructed?

Currently, the national baseline is a sample based on just under 2,000 young people that have completed the survey between 2010 and 2011. This is comprised of surveys across a variety of different settings, including mainstream schools and charity projects. However, it is not fully representative in terms of geographical, demographic or social spread.

To partly correct this, the data has been weighted by age and gender in comparison to national demographic data. We use standard statistical weighting procedures.

To assess whether our sample provides a satisfactory proxy baseline, we compared it to results of a national survey of the well-being of young people age 11 to 16 in 2008 published by The Children's Society. Our sample also reported similar patterns when comparing boys and girls and different age groups within the sample.

How will you keep the national baseline up-to-date?

A condition of using NPC's Well-being Measure is that data is (anonymously) submitted to contribute to a sample of all the young people that have used it. As the well-being measure is used more and more, this sample will grow.

As the sample grows, it will enable us to learn more about the well-being of young people in the UK and increase our powers of analysis. As part of the survey, we request postcode data on individual young people which will enable us to produce a nationally-representative sample. We intend for this sample will be used to regularly update the national baseline.

June 2011


 
previous page contents next page


© Parliamentary copyright 2011
Prepared 23 June 2011