At Skateistan we talk about ‘impact’ a lot. But what do we actually mean, and how do we measure it? The challenge is, impact is multi-dimensional and there are lots of ways to understand, analyse and communicate it. But essentially we want to know what sort of difference and how much of a difference we are making. Over the past few years we worked with three volunteers who rose to this challenge and explored a unique way of looking at impact. We are very excited to show you what they found.
It’s called ‘Social Return on Investment.' What, you say? Yes, it’s a mouthful. So we shortened it to SROI. SROI is a measurement tool that helps organizations to understand and to tell the story of how social, economic and environmental value is being created by their programs. Social values are translated into monetary values so that a ratio for return on investment can be calculated. For example, an investment of $1 USD could deliver $X USD of social value. But SROI is primarily about value, rather than money. The process requires careful consideration of all levels of all program operations, from the planning of activities to their execution and evaluation.
Our volunteers, Zach Lynn, Elisa Likovic and Adèle Guinaudie, each focused on a Skate School location (Afghanistan, South Africa and Cambodia respectively) to write up a SROI report for the year 2018. Methodology was chosen from an established guide for SROI analysis developed by Social Value UK and the New Economics Foundation (Social Value UK, 2012). We were supported throughout the process by Statistical Evaluation Consultant, Samantha Chui, as well as a team of 10+ independent volunteers from ‘Statistics Without Borders’ to help us analyze and verify our data.
Below are some of the main steps followed:
Who was involved?
Identifying key stakeholders such as students, families, staff and indirect stakeholders like governments, schools, partners, and even yourselves.
How much was invested?
Identifying the monetary inputs - including in-kind donations.
What were the activities?
Understanding Skateistan’s sport and educational program activities, as well as staff training activities.
What were the changes?
Identifying major outcomes; improved wellbeing and skill development including life skills, leadership skills, and literacy/numeracy skills. As well as improved child care capacity for our staff which takes into account an improvement in employability as staff develop necessary skills to succeed and excel in the workplace. These outcomes were evidenced from qualitative and quantitative data analysis from our Skate Schools.
How did we value these changes?
Noting various assumptions and estimations from qualitative interviews with staff at our Skate Schools, and identifying suitable financial proxies to our program activities. We asked ourselves; how long will the impact last?; how much change would happen if we didn't exist?; and what services (financial proxies) can we compare our programs with?
What did we find?
Calculating the SROI ratio based on a 5-year and 10-year forecast by incorporating all of the above information.
We explored each location Skateistan works in individually, because each country’s context is very different, meaning that a comparison between them is not appropriate. For example, 2018 was the year Skateistan Cambodia moved locations and built a new Skate School, so a smaller number of programs were running than usual. In reality, the values we found are likely to be understated, given the unique nature of the services offered by our sport and educational programs. Similar services are extremely scarce and don’t necessarily offer the same quality of guidance for youth, or accessibility to such a variety of activities.
Key staff members from each Skate School were involved in the SROI process, but the analysis is not without its limitations. With more freedom in geographic locations, time and resources, more people could be involved in the valuation process. Overall, the study highlighted some lessons learned that we now want to focus more on. One of our volunteers, Elisa, explains,
An important lesson was about our understanding of long-term impact. Making forecasts for 5 or 10 years into the future got us thinking about how we can build on our programs and data collection system to stay connected with our students even when they leave Skateistan and graduate from high school. Many of our students have grown up to become educators at Skateistan themselves, but there’s also many unheard and untold stories out there. That’s why this year we are piloting an alumni project, supporting our graduating students to take action in their local communities.
We are very grateful to everyone who was involved in this research. If you are interested in reading one of our full reports on a Skate School location, please contact us at firstname.lastname@example.org. We are in discussion about third party evaluation plans for the future, and we appreciate all constructive feedback to ensure the ongoing improvement of our impact evaluations.
“I am really grateful to have taken part and contribute to such an in-depth and important study and I believe many other organisations would benefit from it.” - Adèle,