Our research analyzes how other leading philanthropic organizations and impact investors quantify, model or incorporate risk into their processes. We find there are three general approaches:
GiveWell makes limited adjustments for uncertainty related to execution risk (wastage, but already pre-selects investments that have rigorous impact evidence, so the only adjustment to evidence risk is ‘False monitoring results’. In example below, GiveWell discounts the estimated impact down by 9% (91% adjustment factor)
https://lh4.googleusercontent.com/7tGHViG6Ojz_BVM0VL9k8RjdoqWcacs2HC33SpdNCWbtAxUQ8WY9NS5kE2dkbE8YB9jaSYz8MhvK-eIMZ4pYjb-DlmQVizlfK6nf4S8qTJTvvNbbTnfSUZt0ZxAPmzhKUUzZG9SA4xhUDd1PDUkpvaU
The Hewlett Foundation uses an expected return model as part of their Outcomes Focused Grantmaking approach. Their model’s formula is: “The formula multiplies potential benefit of a project by likelihood of success and the grant’s contribution, and then divides the product of that calculation by cost”:
https://lh3.googleusercontent.com/xxhnK0UHqd6KP1OxagGL6L3KqPpgqargct5fLApz0A0P5p0ysG7yJFhzSTQbUWfpMuhlsXSz0M1LTyJTWaNDzDMEikMBQYrRw2DTEFt6RzQ2pJxLdoWa0okzSvFAjeXm73ygxNAFjZZ1dxFGK7LBPyY
Their formula for likelihood of success is:
https://lh4.googleusercontent.com/Dbrx1D8oYKR5CDUkUvSypSDhay5SZOzSMqHLqh7Zq3Ym3MYY-RwrgGHpmtfPAae1FN7T_iauKzk8BU9Kli7ZaSJ92R6ihEzCF-FBMU-0wI7YuuCEoRSZGxoVgkA3BSefvAbq_vG1XjnFNLoKLJkY0tY
An example they use is:
Benefits of a project in Nigeria = Double incomes for 8,000,000 people
Strategic Accuracy (Impact) x Grantee Success (Execution) x External Conditions (Unknown Risk)
Likelihood of success = 25%. They acknowledged that the theory of change relies on many moving parts working together in just the right way. Taking into account all these risks, the Foundation gave its theory of change for Nigerian governance a 25 percent probability of success.
Hewlett contribution - 10%
Cost - $30 mill
Expected Benefit = 200,000 people have incomes doubled
RiseFund uses an Impact Multiple of Money (IMM) model to assess the potential impact of their investments.
The impact valuation model is Breadth x Depth x Value x Risk Adjustment:
Impact Multiple of Money Methodology Here
https://lh5.googleusercontent.com/JpjDDx2UhFHjnP52_C5-i5pFMg1IBoVtYVuYPeb0RbMOlHCOgHiWWc9x8Uv-GWVLi03CiSOOfiC0or5FtdYu3_6MOusg3vj-XIxCX2JYpqbUU2cZU6gc0ddIEY7eUvWrdwrdoSAC3aBz4kFOOeWU324
Risk Adjustment: They include a risk adjustment that lowers the expected impact value by the level of risk. They use a 100-point risk scale which is described as: “We use a to adjust expectations of impact for different kinds of risks, including the risk of business output, the risk of product usage, and the risk of research suitability, risk of scaling, risk from change in geographies, risk of change in product design, etc. We calculate the realization risk of the specific impact pathways that we incorporate into the IMM. The Rise Fund assigns values to six risk categories and total them to arrive at an impact-probability score on a 100-point scale.
Global Innovation Fund’s Practical Impact Unit Methodology
Model:
https://lh3.googleusercontent.com/WA3_lvXf_YYNEqs-G9gIhEtWpJudPuJ7C-m_brmyl6_kzJCTZcE8mUGHg6vLizrnsYIW7Q61sPRDHv6Qtcn9Nw1V0u9XbngBJ1NdBaObkqH-1s9hCtZ-lHvKW9WI-FkVXpTgR320G8vTWPcEuAFM1ec
https://lh6.googleusercontent.com/Z9CGe0uFCGjoO2bxdfPfTDhcYWoLaNWtyYayFcbtpNRQ0h13uW-L_Nh9A-7B6hG5m4_s9Qph5X6uYV1PR2K3wrPFzTkaYZMwOHjILx2B4IARCftMEKckgAAmbiMR-fJ4ejQQvVdtSfsWau2wtQrV52A
Detailed model here Risk Assessment: “Probability of success is the converse of risk, with probability estimates necessarily educated guesses, based on identifying and assessing critical risks. Research has shown that it is possible and useful to assign numerical probabilities to risks.”
“Stage 1 (current): GIF’s current investment tests whether an innovation can achieve a critical step towards scale. For example, a grant might fund a rigorous test of an innovation’s cost-effectiveness. This could be a condition for adoption by a government agency. “
Stage 2 (scale): Success at stage 1 may be necessary, but not sufficient for the follow-on action. The government may decline to adopt the innovation; the funding round may not materialize. Even if there is follow-on support, the innovation may simply not work at scale.
Probability of success at scale = probability of success at stage 1 x probability of success at stage 2, assuming stage 1 success.
This approach has also been used by many impact investors and philanthropic funders to assess risk as a risk management strategy.
Robin Hood Foundation has an inspiring Benefit-Cost Ratio approach to assessing the potential impact of their grant investments. They do not discount their estimates by the likelihood of success of the programs. However, they do consider the potential scalability and reach of their grants which informs their risk strategy.
According to their benefit-cost methodology “We do not discount point estimates to reflect the relative risk of grant-specific estimates”
The Impact Management Project’s Five Dimensions of Impact (now hosted at Impact Frontiers) is an excellent resource that highlights best practices in impact measurement and impact due diligence. One of the five dimensions is risk and their website outlines best practices in how impact investors (and funders) could assess the risk of their investments.
They recommend reviewing potential investments along 9 types of impact risk and to use a scorecard that adds a point value for each risk.
Use a scorecard to identify potential risks that can be used to inform risk mitigation. “Once risks have been identified, assessed and prioritized, they need to be mitigated.”
https://lh5.googleusercontent.com/2csmN_k7d0MV33ph7r-vXxzFCcSZsTx7-Mf3aWI5K0tJm3uhjGCDUJB71DWvqlrK34ohJ4JrKy7cEyIYyxBHom5xAZ5BdYL1SezPufZI-m4AUGhRDuqngFyOIRO_srW8gUsPl5mtUL5zkTIwBk-hYEk
DIV uses a tiered approach to grantmaking. Those projects that have higher risk with lower levels of impact evidence receive a lower grant amount. This allows DIV to separate their portfolio by risk levels. As risk is lowered, projects are eligible for higher funding amounts.
Tiered Approaches to Impact
https://lh4.googleusercontent.com/2jOxiXayPaSV-FqF6GxmBdeKQhPaGsLUDCC3ssEhEPypp2lmLbl05Ny8V4lqb5jERDtIosZvShMPOkBCNr-9u3nURmoPpYle_KJ2uEErBAM9CYXe5Pj6EQ-_nRdsmWzQhKMDaINvaSNODKjxgmJZMac
The Livelihood Impact Fund has a separate Lab Portfolio that invests in data driven, earl stage, African-led organizations. These organizations are 1 to 5 years old and the Livelihood Impact Fund works closely with these higher risk organizations through ongoing capacity building support and multi-year, unrestricted funding.
Kiva Labs is a separate portfolio of financial service and social enterprise partners. Kiva’s flagship partners are low-risk microfinance organizations that are able to make loans and receive repayments at a 96% repayment rate. Kiva Labs was intentionally separated to focus on riskier but more innovative social enterprises around the world without comparing Kiva Labs companies to the typical microfinance partner.
Our research also finds that organizations classify risk into thematic areas, the most common of which are execution risk and impact evidence risk. The risk types outlined below can be primarily categorized into those two types:
Impact Frontier Nine Types of Risk
Open Road Alliance Risk Matrix