Is it worth trying to measure impact?

One problem with impact investing is that the true value of the impact (i.e. the benefits to society of certain investments) is very often unmeasured. As a result, we can never truly gauge the success of our efforts to address societal challenges. Impact thinking is based on a logic chain that is known by the abbreviation IOOI: Input, Output, Outcome, Impact.

• The input stage includes the factors of production, such as the monetary budget or the people that can be used for the project.
• At the output stage the results are the direct consequence of the inputs: how many people were involved in discussions or how many pages of analysis were produced?
• The outcome states the changes achieved in the target audience: how many learned to read, how many changed their habits?
• The impact is the change in well-being achieved as a result of specific actions. For instance, how did the lives of those who learned to read change or how is the path of their life different from what it was earlier?

Often the measuring of results is done at too early a stage in the IOOI chain, and true impact is not examined. If the impact is unknown, evaluation of whether the work performed was truly beneficial is also impossible.

Challenges

The basic idea of the chain is easy to grasp. The problems in practice are often created in the step from outcomes to impact, and the difference between the two is often unclear. Impact also may not be revealed until years later. Furthermore, impact may affect many different areas and be shared among many different beneficiaries. Its causality may be ambiguous and difficult to isolate.

There are at least three approaches to measuring impact.

1)     Measure impact with the best possible methods, even though it is difficult. This is often the process in pilot cases, where more money can perhaps be spent on evaluation than in continuous work in the field. In the first social impact bond in the UK, at Peterborough prison in 2010, an outside consultant and a university sought multiple comparison points with other prisons for each inmate involved in the intervention.

2)     Choose the outcomes carefully and trust that their impact will be positive. In many fields research findings on outcomes which lead to impact are scarce. This holds especially in cases where the impact is based on subjective sensations or is highly context-dependent and occurs over the course of a number of years. When the desired outcomes have been chosen the focus is based on achieving them. For example, the Nesta Impact Investments fund has defined 12 outcomes that companies seeking investment need to pursue. Nesta (the UK’s independent innovation charity) uses the word “impact” in reference to the extent to which output leads to outcome, not in the IOOI-sense of impact.

3)     Assume a link between outcome and impact, but think through what might challenge this link or prevent it from functioning. This approach is based on the fact that measurement of impact is often too expensive in terms of both time and money to be sensible. It is still important to try to ensure that the impact is useful also in the long term and does not, for example, lead to unintended negative impact. The European Venture Philanthropy Association (EVPA) recommends this approach.

Learning by doing

It is worthwhile trying to measure impact, even though we know that the final result will be imperfect. Perhaps the final goal can never be reached; there may always be some uncertainty. Despite this, measuring – to an extent that is sensible both in practice and economically – deepens the understanding of the topic at hand and the causes and effects within its domain. As experience accumulates, measurement will gradually become more successful.

The measurement of impact gains even more importance when private funds are enticed into solving societal challenges. Impact investing can advance the testing of new ideas. But especially when private investments stand to profit from a reduction in societal expenses (e.g. the social impact bond financing model), the metrics have to be agreed upon in contractual form before the investment takes place.

The need to measure impact creates an extra challenge for impact investing, which complicates the investment process. Therefore, it is important to try to standardize and simplify measurement. An added benefit from this is that impact investments that use relatively standardized practices and metrics are easier to resell to other investors, which brings liquidity to the market.