A Student-Led Consultancy Report On ECHO, A Covid Business Volunteering Project In East London From Economy Of Hours.
What Is The ECHO Project?
ECHO is a non-profit organisation which connects people and businesses to share skills via mentoring. They developed an digital platform called The Match to deliver their services during the Covid-19 lockdowns. A team of five graduates from Queen Mary University of London were tasked with assessing how effective The Match had been in connecting volunteers with businesses and contributing to social return in the wider community.
Echo launched The Match in direct response to the Covid-19 crisis, recognising the challenges many small creative businesses are facing, and the potentially untapped resource of skilled furloughed workers.
1. Review of previous research
Students reviewed current literature on Social Impact Evaluation and non-profit organisations.
2. Development of Case Studies
Case studies were produced to help highlight the experiences of some key participants from The Match: volunteers and host organisations.
3. Quantitative Analysis
This then led to a deep analysis of The Match data on volunteer and host organisation characteristics.
4. Recommendations
Our recommendations for more effective social impact evaluation in small and local non-profit initiatives.
Executive Summary
The report produced analysed and evaluated an understanding of the meaning of social impact and social value when using evaluation practices in Third Sector organizations. It is clear that both terms lack an agreed definition but at the same time governments around the world in recent years have helped formalize and incentivize evaluation practices in the Third Sector.
Lack of theorization and conceptual framing of performance measurement in the Third Sector exists but ‘evaluation logics’ – specifically the Scientific, Bureaucratic and Learning Logics – can be helpfully applied through three main types of evaluation methods: Logical Framework approach (LFA); the Most Significant Change (MSC) technique; and the Social Return on Investment (SROI).
All the above evaluation methods invariably include stakeholders during the evaluation process, and stakeholder involvement is considered to have a positive impact on both the usefulness and the credibility of the evaluation.
Attention to evaluation of Third Sector organisations projects has been driven by: i) the professionalization of the industry; ii) the pressure of funders wanting to find out whether their funds are making a difference or could be spent somewhere else; and iii) entrepreneurs looking for new opportunities.
Geographical Context
The Match operates across the East London Boroughs, including Barking and Dagenham, Hackney, Newham, Tower Hamlets, Lewisham, Greenwich and Waltham Forest.
Impact of Covid-19
The Match was set up in response to the Covid-19 pandemic, so we wanted to see in what ways businesses had been affected, and their motives for signing up to this project. It’s important to note that not all businesses were adversely impacted by Covid-19. In fact, some businesses formed because of the pandemic, such as fabric companies that sell face masks.
The most common impact on businesses was the fact that they were forced to make business changes regarding their models etc. A lot of businesses found they had to deal with some clients virtually or even stop providing their services all together.
“It was amazing how quickly we found somebody through The Match. It was really nice to find someone with a specific skillset as this was one of the things we really needed.” — Sarah, The Line
Recommendations For Improvement
The outcomes of small and localised partnership projects such as Echo are typically qualitative in character and very resistant to sensible measurement (for example, as attempted in SROI calculations).
The outcomes of such projects are also largely unknown until both parties start working together, and – as the case studies show – it is often that unexpected or additional benefits are achieved beyond the expectations of the initial help offered. Thus, a learning logic is the most appropriate approach.
Within a learning logic approach, it is also appropriate for small and localised partnership projects to evaluate using the Most Significant Change method, in which the key qualitative changes to partners’ networks and working methods are identified, tracked and recorded over time.
In operationalising the Most Significant Change method, it may be helpful to map individual match journeys to help build narratives of the stages and elements that lead to different quality matches. This could eventually enable a typology of characteristics of strong, moderate and weak matches to be constructed.
In order to be able to express the strength and quality of matches between volunteers and businesses, it should be possible to complement the dominant qualitative story-telling data with some quantitative data. This could also enable interesting correlations between different variables to be run e.g. between particular skills, business needs and demographic characteristics.
This might be achieved at the participant-registration stage, by asking Likert-scaled questions (e.g. Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree) to both sets of participants on a range of items.
In order to visually demonstrate the development of relationships in the geographical area over time, build an anonymised dashboard, displaying the connections made and hours given.