One of the most rewarding aspects of running a profitable business with a social purpose is our ability to offer pro bono time.  In our role as Service Designers, we see many of the funders that we support struggle to evaluate impact. Whether it is too time-consuming or they don’t know where to start or they need a fresh pair of eyes to spot areas they could improve.  

Last year, we offered our nonprofit customers pro bono time from our Service Design team to identify ways to better understand their impact.  Two of the funders that said ‘yes please’ were Cycling Scotland and Halifax Foundation for Northern Ireland.  Impact assessment is a huge topic but in this blog, I share the short story of how we helped Cycling Scotland to identify simple steps that they could take to find out who is benefiting from their work and what is and isn’t working.  

Cycling Scotland works with others to help create and deliver opportunities and an environment so anyone anywhere in Scotland can cycle easily and safely.  Their Cycling Friendly Development Fund provides grants for infrastructure such as cycle parking, upgrading access routes for people cycling, walking and wheeling, facilities such as changing areas, showers, lockers and drying areas.  They wanted to better understand the reach of their grants.

The key to good impact assessment is often good data.  Our first step was to analyse the data being collected. How, what and when is impact data captured throughout the funding process from application to assessment to post-award monitoring.  Below is a summary of our findings and recommendations:

How is data being captured?

• There was a significant amount of missing data required to measure impact including geographical location and sector breakdown.  Recommendation: Consider making key fields required on forms and ensure that options are provided consistently and in a way that meets your impact reporting needs
• Reasons why applications are rejected were not specific enough to pinpoint why applications ‘lacks detail or quality’. Not only did this minimise opportunities to understand why applicants weren’t fully completing their forms, but Cycling Scotland were unable to understand how their forms could be changed to improve applicant’s chances.  Recommendation: Capture more precise reasons why applications are not successful to inform areas for improvement
• Outcome data is not being captured consistently in order to provide an aggregate view or to indicate how inequalities are being addressed.  Recommendation: Ensure that what you are asking applicants to evaluate is then followed up consistently to capture how it is being met during funding and beyond
• An insufficient percentage of end of grant monitoring forms are being submitted for meaningful analysis.  Recommendation: Survey grantees to find out why forms are not submitted (time / questions being asked?) Offer support via webinars or recordings to convey the importance of collecting this information

Are there barriers preventing desired impact?

We also investigated whether there were any barriers during the assessment process which may prevent certain community groups from applying or being awarded.

• Our analysis found that assessment projects were favoured where a financial or in-kind contribution is made.  Recommendation: Review assessment criteria to make sure that it is not preventing target communities from receiving funding
• Smaller groups may not understand or be able to comply with the terms of the funding. For example ‘acceptable regulations’ is open to interpretation.  Recommendation: By crystal clear with your explanations of what is expected and offer support for those groups who made need it

During our research, we helped Cycling Scotland to generate visual representations of the data available to them.  See this simple map that we produced to show the funding hotspots and gaps.

We carried out a similar data and process review with Halifax Foundation for Northern Ireland and presented them with recommendations to strengthen their ability to understand the impact of their funds.

There is a second phase of work planned with both organisations to conduct focus groups with grantees to understand how the longer-term impact of programme facilities can be measured. This will be an opportunity to gather some rich data to understand what happens once funding has ended – as we know that once the funding has been awarded, that’s only the beginning of the journey for many.

Map displaying areas of funding hotspots and gaps for Cycling Scotland.

Simple tips to assess your impact

Here are some simple tips that can help funders to get started evaluating your own impact:

• Start simple – don’t overcomplicate
  • Define the questions that will shape your impact evaluation early, for example, ‘who are we reaching’ ‘what barriers do applicants face to receiving funding?’
• Only collect the data that you will need
  • When designing your application, assessment and monitoring processes, think of your future self and ask – is this what I will need to know later on?
  • Minimise the amount of time it takes to provide data so don’t be templated to ask for lots of information that won’t be used
• Make it easy for you grantees to provide data
  • Simplify your data capture forms
  • Make fields to capture important information required
  • Provide clear guidance to support grantees throughout the process
• Make it easy for grant staff   
  • Can requests and reminders for impact data be automated to save time chasing?
  • Can questions be captured consistently across programmes to simplify analysis?
• Remember that your impact goes further than the grants process
  • Carry out repeat surveys to know what has happened as a result of your funding
  • Maintain communication with your grantees, create feedback streams like focus groups or comment channels
• Know your target groups
  • Check if your funding is representing the groups you aim to support
  • Target specific groups to understand their specific needs
  • Analyse rejection reasons and act on them

“Using Hyphen8’s help we were able to identify considerations about the way in which we currently gather data, with actionable recommendations being made for us at the end of the first phase of the work, which we’ve started implementing. With their support, we are now moving to a second phase of work, which will investigate the longer-term impact of the projects we have funded across community groups, employers, social housing providers, and education organisations.

We have found working with Hyphen8’s team to be a very positive experience – they have provided us with insight, practical recommendations and next steps, and have been flexible in delivering the work around our busy schedule”

Peter Mills
Salesforce Administrator and Programme Manager, Cycling Scotland