Written Evidence Submitted by New Philanthropy
NPC's Well-being Measure is an online survey-based
tool that measures how young people feel and what they think about
their lives. It is a useful tool for youth groups and schools
for quantifying "soft outcomes" and understanding how
much young people enjoy life.
It measures eight aspects of well-being: self-esteem,
emotional well-being, resilience, satisfaction with friends and
family, satisfaction with school and community, and overall life
By doing the survey at two points in time, it can
measure change-and be used by charities, schools or youth groups
that want to evaluate the impact of their work. All results are
benchmarked against a national sample of young people.
The Well-being Measure is designed to be used with
a group of ten of or more young people aged 11 to 16. It is not
valid for measuring the well-being of individuals.
The tool can be used to:
(a) measure the changes in a group of children between
two points in time-for example, to test whether a youth project
has an impact on young people's well-being.
(b) compare a group of children to a national baseline-for
example, for schools to see how their students compare to other
young people in the UK.
(c) look for patterns within a group-for example
to compare whether well-being differs between boys and girls in
a group, or to see whether an intervention has a different impact
on young people eligible for free school meals compared to those
that are not.
All results are available in a downloadable report
once the survey is finished. Organisations can also do their own
custom analysis by using our online filtering tool.
Organisations sign up and create their own surveys.
Although surveys are based around a standard set of questions,
they can be customised by selecting which areas of well-being
to include and adding questions if required. Once a survey is
launched, young people are given a unique code and complete it
online, which usually takes around 10 to 15 minutes.
In the survey, young people are asked to respond
to a series of statements about their lives, and must say how
they feel on a five-point scale (ranging from strongly agree to
strongly disagree). Individual responses are then aggregated to
produce an overall measure of well-being in each of eight areas.
The final results for the group are presented as a percentage
score from 0 to 100, which reflects well-being in the context
of other young people in the UK. For example, a score of 30% on
self-esteem, means that 30% of the national population has lower
self-esteem than your group and 70% of the national population
has higher self-esteem than your group. The national population
figures is drawn from a sample of 2,000 young people from across
UK that have completed the survey. Over time this sample will
grow, increasingly analytical capability of the tool.
Once a survey is closed, organisations receive a
detailed report of their results, and can do some of their own
analysis using our online filtering tool.
Organisations that want to measure change can then
do a "follow-up" survey. A second report is generated
once this is complete and shows how the well-being of participants
More information on the Well-being Measure is contained
in the appendices. Appendix 1 shows the questions in the basic
survey. Appendix 2 describes the national baseline sample.
At present, NPC's Well-being Measure has been released
to a limited number of customers. We are limiting the number so
that we can test how much support organisations require to use
it and make any necessary adjustments. We will be gradually building
up subscribers until our full launch at the end of 2011.
Current customers include: BBC Children in Need,
Action for Children, Body and Soul Charity, Wellington College,
Depaul UK, Save the Children UK, ContinYou, The Outward Bound
Trust, Brathay, and the Specialist Schools and Academies Trust.
Customers pay for the amount they use and payment
is based on buying "survey credits".
One credit buys a single survey of up to 200 young
people. Each survey includes a report of results.
Running one survey allows users to measure the well-being
of participants relative to the national baseline. Running two
surveys (an "initial" survey + a "follow-up survey")
allows users to measure the change in well-being, and see
what impact their programme has on young people.
The costs of survey credits are below. There are
discounts for multiple credits bought in the same transaction.
|Number of credits
||Cost||What it buys
|1||£295 + VAT||A single survey of up to 200 participants
|2||£475 + VAT||An initial and follow-up survey of up to 200 participants
|10||£2,000 + VAT
||5 x Initial and follow-up surveys of up to 200 participants
|20||£3,600 + VAT
||10 x Initial and follow-up surveys of up to 200 participants
Surveys of more than 200 young people require more credits. Each
credit allows you an extra 200 participants.
The Well-being Measure has been developed by NPC over more than
three years. There were two phases in its development: creating
the survey and designing the online tool. Each is described here.
Creating the survey
We began with a thorough literature review to identify all the
existing work on measuring well-being. Using this as a basis we
created our well-being scales using the "best bits"
from other studies. Our scales all use questions adapted from
work by academics and psychologists in the UK and US.
The survey was developed with five charities and in 13 schools.
It was tested with over 850 young people and met stringent statistical
benchmarks for validity, reliability, and internal consistency.
The tests were also designed to ensure that the survey was practical
to use, that young people understood it and reacted well to it.
The survey went through many revisions to meet these criteria.
During the process we had a steering group of academics and practitioners
to ensure the quality and robustness of our methods. You can read
about the development of the survey in our 2009 report Feelings
count (available on the NPC website).
This phase was entirely funded by NPC, using our charitable funds.
Designing the online tool
Having developed the survey, we wanted to make sure that it was
in a practical format where it could be used by charities and
schools that are not experts in evaluation methods or statistical
Working with the IT developers at Public Zone (www.publiczone.co.uk),
we designed a website that allows users to sign-up, administer
surveys and receive their results. Within this site, data is entered
online and analysis and reporting is done automatically, making
it easy to use.
The website was developed with feedback from a group of 11 organisations,
including charities and schools. This group helped us to design
the system, determine its requirements and test the tool. We also
consulted with other experts in the field of evaluation and drew
on other experience within NPC.
This phase was funded by a combination of NPC, The Private Equity
Foundation and the Paul Hamlyn Foundation.
NPC's mission is to improve the effectiveness of the charitable
sector so we have designed the tool with this in mind. The data
from the Well-being Measure potentially has a number of exciting
applications, as described below.
Understanding the well-being of young people across the UK
NPC collects anonymous data from all users of the Well-being Measure.
As more people use the tool, our database will grow and provide
a richer source of information.
We collect postcode information from young people which will allow
us to do detailed analysis using household level classification.
For example, we will be able to compare the well-being of girls
on inner city estates in south London to girls in rural Cumbria.
Making the links between soft and hard outcomes
The relationship between "soft" outcomes for young people,
such as self-esteem and emotional well-being, and "hard"
outcomes, such as exam results or success in the labour market,
is poorly understood. Using the well-being tool, there is scope
to understand that link.
We could potentially link up our database on well-being with other
datasets on progression to further education or employment, and
see whether there is a correlation between any of the aspects
of well-being. This will help us to answer the question, "Which
aspect of well-being has the greatest impact on children's employment
prospects?" Is it self-esteem, relationship with peers, or
resilience, and therefore which interventions are most successful?
Understanding and comparing what works
For each survey conducted using the Well-being Measure, NPC collects
basic information on the type of project it is evaluating. Over
time, as we refine this process and collect more data, we may
be able to compare different types of intervention to see what
works. For example, we could compare evaluations looking at mentoring
projects with those looking at counselling to see how the outcomes
Developing new measures, and measures for different groups
Our online tool could be extended to include more areas of well-being.
It is currently limited to eight aspects of subjective well-being
but we could add more, for example on diet, exercise or behaviour.
The tool could also be extended to other groups of people, for
example different age groups.
Developing and testing new measures to extend the tool would require
THE WELL-BEING SURVEY
See attached the questions included in the basic survey.
|Aspect of well-being||Description
|Self-esteem||A child's appraisal of his or her own worth. It is closely linked with self-confidence, and is important for a healthy, happy life.
|Emotional well-being ||The state of a child's mental health or extent to which a child experiences depressive emotions, as well as worries and other stressful feelings. Low scores are linked to anxiety and depression.
|Resilience||The capacity to cope with stress and difficulties. It involves a positive and purposeful attitude to life and is associated with high self-esteem and interpersonal problem-solving skills. It is a particularly important protective factor to foster in children, enabling them to deal better with future negative events.
|Satisfaction with friends||The child's satisfaction or feelings about the quality of his or her close friendships both in and out of school.
|Satisfaction with family||The child's satisfaction or feelings about his or her family relationships, including the quality and quantity of time spent with parents or carers, and how well the family gets on.
|Satisfaction with community||The child's satisfaction or feelings about his or her local area and neighbours or people in the community.
|Satisfaction with school||The child's satisfaction or feelings about his or her school environment, including how enjoyable and interesting it is, and how safe it feels.
|Life satisfaction||A global measure of a child's overall happiness or satisfaction, based on a single question where the child rates his or her life on a scale from 1 to 10.
THE NATIONAL BASELINE SAMPLE
What is the national baseline?
The national baseline is a sample of young people across the UK
that have completed the well-being survey.
How is the national baseline used in NPC's Well-being Measure?
It is used in all the graphs and statistics generated in NPC's
Well-being Measure to put results in context.
For example, if your results show a score of 30% on self-esteem,
it means that 30% of the national population has lower self-esteem
than your group and 70% of the national population has higher
self-esteem than your group.
In each graph, the national baseline is adjusted to account for
the age and gender of respondents, so you can be sure that your
results are not biased. For example, in a graph that shows a group
of young people consisting of 40% boys and 60% girls aged 11 and
12, the national baseline is adjusted to reflect this.
How is the national baseline constructed?
Currently, the national baseline is a sample based on just under
2,000 young people that have completed the survey between 2010
and 2011. This is comprised of surveys across a variety of different
settings, including mainstream schools and charity projects. However,
it is not fully representative in terms of geographical, demographic
or social spread.
To partly correct this, the data has been weighted by age and
gender in comparison to national demographic data. We use standard
statistical weighting procedures.
To assess whether our sample provides a satisfactory proxy baseline,
we compared it to results of a national survey of the well-being
of young people age 11 to 16 in 2008 published by The Children's
Society. Our sample also reported similar patterns when comparing
boys and girls and different age groups within the sample.
How will you keep the national baseline up-to-date?
A condition of using NPC's Well-being Measure is that data is
(anonymously) submitted to contribute to a sample of all the young
people that have used it. As the well-being measure is used more
and more, this sample will grow.
As the sample grows, it will enable us to learn more about the
well-being of young people in the UK and increase our powers of
analysis. As part of the survey, we request postcode data on individual
young people which will enable us to produce a nationally-representative
sample. We intend for this sample will be used to regularly update
the national baseline.