To work in social impact is to be confused about how to most effectively measure the impact of the work you do. Impact measurement, sometimes referred to as monitoring and evaluation, is a hotly debated topic (later we’ll clarify the difference between the two).
Stanford Social Innovation Review and NextBillion, two of the preeminent thought leaders in social impact, have each published over 70 articles on this topic in the last few years alone. SSIR goes so far as to reject the concept completely in one of its popular articles, Ten Reasons Not to Measure Impact—and What to Do Instead. It’s a concept we’re trying to collectively clarify as a sector, and there are plenty of individuals and social enterprises dedicated to solving this puzzle as well - attend any conference on social impact, and you’ll see no shortage of talks covering tools, platforms, and frameworks to help make sense of measuring impact. In just the past 3 years, a flurry of startups like Sopact, 60 Decibels, and Proof of Impact, have all launched and grown quickly by offering solutions to the often onerous – and expensive – job of impact measurement.
While the value of data is undeniable, many also acknowledge that sometimes the emphasis on metrics goes too far. A common critique of the Gates Foundation is its obsession with measurement almost to the point of ridiculousness, which Valerie Strauss at the Washington Post characterizes by reporting, “If this tells us anything, it is that the obsession with measurement and data in school reform has reached new nutty heights.” The irreverent and insightful Nonprofit AF blog has also had plenty to say about the shortcomings of impact measurement in its current form, publishing articles including How the concept of effectiveness has screwed nonprofits and the people we serve and Why Impact-Per-Dollar is a terrible, harmful way to measure nonprofit effectiveness.
For those of us working in social impact, it can be hard to figure out how to meaningfully measure the impact of the work we do. In this guide, we’re going to help demystify the concept of impact measurement by breaking down the following:
Looking for a template for implementing measurement for your social enterprise? Skip to section 11
Simply put, impact measurement is an ongoing process that helps organizations answer the following questions:
As we touched on in the introduction, impact measurement can take many different forms depending on the organization and type of work it doesIn general, though, impact measurement tends to involve the following:
Monitoring & Evaluation is a related but distinct concept that often gets mentioned in conversations about impact measurement. Often, these terms are used interchangeably, but there are some important differences to note that help us understand why both are useful.
In general, impact measurement is an organization-level analysis to ensure there is progress being made towards the organization’s mission, which includes looking at programs, processes, and long-term outcomes. Monitoring & Evaluation (M&E) is often something that happens at the specific program, service, or intervention level and involves the continuous monitoring of a program (is it doing what it is supposed to do) and frequent evaluation (is the program actually making a positive impact). As such, a social enterprise might employ impact measurement for its public impact report about its entire portfolio of services, and it will also employ specific M&E activities for each of its products/programs/services individually.
Venture philanthropy group REDF provides a useful diagram to show this distinction in its introduction to measuring impact:
As you can see, program monitoring and performance measurement typically happen over shorter time frames, while impact and outcome evaluation is something that covers a longer time horizon.
Another way to differentiate between monitoring activities and evaluation activities is to look at whether it’s something the company is doing internally or something being done by an external party. According to the nonprofit Unite for Sight, “Often used synonymously, monitoring and evaluation in fact refer to two different activities. Monitoring involves on-going measurement of performance, examining parameters such as cost efficacy, and assessing whether things are going well…and involves the regular tracking of resources, activities, and outputs at the project, regional, national, or global level. This includes the monitoring of a country’s progress against the Sustainable Development Goals, or other standard measures of success. Evaluation, on the other hand, is often conducted by external advisers, and is focused on proving impacts rather than improving interventions. Evaluation seeks to answer questions about whether the right things are being done, and is a measure of the performance of the program design, of the implementing team, and of the implementing agency.”
In 2006, Factfulness author Hans Rosling gave one of the most popular TED Talks of all time: The best stats you've ever seen. This talk sparked the imagination of policy makers, nonprofit leaders, early impact investors, and funders about the power of data, and motivated a massive effort in the sector to collect and report on more of it.
In our popular article, the 600 year history of the social enterprise movement, we zoom out even further to tell the story of Florence Nightingale, a pioneer of the social worker movement. In her work in the late 1800’s, Florence used data and early versions of data visualization to show how nurses could improve health outcomes at hospitals. However, it took over 100 years before these improved data-informed models became mainstream.
In 1969, the Tax Reform Act gave the USA Section 501(c)3 in the Internal Revenue Service Code, which established the nonprofit organization as a separate type of legal entity, leading to a proliferation of nonprofits (and some tax fraud). To ensure that nonprofits were being truly used for social good, legislators later added specific requirements for measuring and reporting on impact. In 1994, the Interorganizational Committee on Principles and Guidelines for Social Impact Assessment (ICGP) took things a step forward by publishing the Guidelines and Principles For Social Impact Assessment. In 2000, the Global Reporting Initiative launched and created Standards to improve the way that businesses around the world reported on sustainability and social impact. A year later, in 2001, Charity Navigator launched to bring more transparency to the nonprofit sector.
In 2006, the United Nations established Principles for Responsible Investment to guide public companies on reporting on ESG. In 2007, B Labs launched its first B Impact Assessment to help companies, particularly social enterprises, measure their impact on their workers, community, environment, and customers. A few years later, in 2015, the 17 Sustainable Development Goals (along with their 169 targets) were established to create a set of targets and measurement frameworks that standardized reporting (to at least a small degree) around the world. A 2016 report from the Global Impact Investing Network (GIIN) shows that most enterprises now believe that measuring impact is critical to creating both business and impact performance.
The proliferation of data visualization tools further accelerated this trend, especially with the launch of the Tableau Foundation’s “living” impact report in 2016. In 2018, two bestselling books brought the idea of more rapid measurement and reporting into the public mainstream: Ann Mei Chang’s Lean Impact, and John Doerr’s Measure What Matters.
Every organization has unique nuances that make standardized impact measurement a real challenge. However, there are two frameworks that have become ubiquitous, and working through them is sure to benefit any organization:
Framework #1: The Impact Management Project’s Five Dimensions of Impact Framework
The Five Dimensions of Impact framework is a helpful starting point to ensure that your Impact Measurement & Management (IMM) is set up to generate a robust understanding of all of the different ways your program may be creating change during the crisis. In this framework, impact is measured along the dimensions of What, Who, How Much, Contribution, and Risk:
Framework #2: The Theory of Change
The Theory of Change (ToC) is the simplest logic model to help a social enterprise develop a high-level strategy for measuring impact, and figuring out what data points are needed. In section 11, we provide a 5-step guide to developing your own impact model, which is largely based on the Theory of Change model.
For a more detailed summary of the Theory of Change, check out the below video from Sopact (and their templates), and we also recommend reading this guide from Rank & File.
If you are looking for more information about measuring change, FSG’s report on Evaluating Complexity provides guidance on how to take a more iterative and adaptive approach to avoid becoming locked into a set of metrics and assumptions that may become obsolete as conditions change.
Unfortunately, good intentions don’t always lead to good outcomes. There has been a lot of criticism of impact measurement from multiple angles, but when you look across these failures, you start to see some notable trends:
Measurement has created the “nonprofit hunger games”
According to Vu Le, author of the fantastic blog Nonprofit AF, the requirement that donors impose on nonprofits to measure data leads to a vicious cycle of high costs and micromanagement. In his talk, The way we think about charity is dead wrong Dan Pollata also highlights how what we choose to measure can stifle the potential of social impact initiatives when it is done the wrong way.
Measuring the wrong thing creates the wrong solutions (which can be problematic in and of themselves)
Tom’s Shoes is a great example of a social enterprise that measured the wrong thing – it measures its success in terms of total volume of shoes given away as opposed to the impact of its buy-one-give-model. Focusing on “vanity metrics'' rather than metrics that meaningfully move the needle on long term impact can create a false sense of accomplishment that obfuscates more foundational problems. As we mentioned in the introduction, measurement is how we know whether what we’re doing is working. But defining “working” incorrectly is like setting your GPS for the wrong address - you’ll get there, but it’s not the place you originally meant to go.
Imprecise reporting propagates bad ideas
In the U.S.A in 1978, a documentary film called Scared Straight! received the Academy Award for best documentary. The movie followed juvenile offenders as they got a harsh preview of prison life from inmates, and it inspired a wave of similarly fear-driven youth-intervention programs and television spin offs across the country throughout the 80s and 90s. Despite their popularity, these types of programs not only didn’t work, but they also had a net negative effect according to multiple long-term studies. In other words, improper measurement and promotion led to billions being spent on a program that fell short of its goal while negatively impacting individuals and society.
There are some great organizations doing work to improve impact measurement for all. Here are some categorized by sector:
For cross-sector measurement
For the nonprofit sector
For the corporate sector
For the impact investing & social enterprise sector
If you’re interested in improving impact measurement, here are some must-read books:
We’ve collected a number of great articles on impact measurement that we think will help you learn more about this topic:
Based on a review of existing literature, the following represents 10 principles that we have collected in our Institute to help guide individuals and organizations in using impact measurement to improve performance:
Here are some great impact reports that show how measuring impact can lower costs and improve performance:
Social enterprises should embark on impact measurement with deliberate urgency. Meaning: measure, but do so in as lean and strategic a way as possible. Mark Horoszowski, CEO of MovingWorlds, provides the following guidance, “I think about impact measurement the way I think about any other big decision at our social enterprise: what is the objective, what is the most cost efficient way to operate it, and what will create the most value for our stakeholders?”
Below are 6 steps that you can take to begin and/or improve on the way you measure and report on impact at your social enterprise (download our template here):
Start by pinpointing exactly why you want to measure your impact. Is it to woo donors? To earn trust with customers? To iterate and improve on your social enterprise operating model? Most likely, it will be a combination of these, so you want to document the reasons to monitor impact in the first place to ensure you’re answering the right questions. In the table at the end of this list are six common categories that social enterprises should be reporting on.
For each objective that you have for capturing data, determine a metric (or a couple of metrics) that will accomplish that objective. One great resource is the IRIS+ Thematic Taxonomy from the Global Impact Investing Network (GIIN) which seeks to “provide a shared language for describing, assessing, communicating, and ultimately comparing impact performance.” You can browse through the taxonomy and review the metrics they suggest, and then choose the one(s) that is best for you. Taking the B Impact Assessment will also help you think about all the different metrics you can measure, and uncover the biggest areas of opportunity for improvement.
What is the leanest way you can measure the metrics you identified in step 2? Once you think through this, it is likely you will need to revisit your data points from step 2 to simplify them. Remember, you can always revisit and improve these points over time. For a great resource on this topic, take a look at Acumen Fund’s Lean Data Field Guide.
The more data you show externally, the better. It builds transparency, trust, and accountability, so as soon as you are able, you should start this process of public reporting. However, don’t create too many complex systems or costs to do this until you are in the optimization stage of your organization – otherwise you’ll divert resources away from your strategic priorities.
Decide on the most efficient tools, processes, and systems to capture, report, and verify data. The following are really useful:
Step back from your work and look at your impact data through the eyes of a neutral party. Use evaluation from a third-party as well as comparisons with available data to assess how effective you are at creating impact compared to others, and use this to foster ongoing improvement. If able, develop a Social Return on Investment calculation (great tips from Sopact on how to do that here).
Here's a template you can use to get started:
And here is a sample of a completed template to guide you:
Measuring impact is challenging, however it helps foster ongoing innovation to provide better services to more people, and to do so more efficiently.
Remember, measuring impact is not a once-a-year activity – it’s an ongoing process that should become a strategic part of your organization so that you keep improving.
Interested in learning more? Register to access our Social Impact Measurement certificate course here.
Ready to take your impact to the next level? If you're an individual looking to transition your career for greater purpose and impact, apply to the MovingWorlds Institute. If you're a social entrepreneur looking to scale your for-good business, apply to our S-GRID Social Enterprise Revenue Accelerator.