Different expectations by portfolio area: We aim to work with organizations dedicated to continuous learning and have a strong foundation for measuring their social impact. We organize our portfolio into three categories - laboratory grants (smaller grants), scaling grants (larger grants), and systems change (larger grants). We have higher expectations for evidence for our larger grants, scaling and systems change grants, and lower expectations for our smaller grants, Laboratory grants.

A focus on outcomes: For all grantees, we expect that they are able to collect quantitative and qualitative data that illustrates if their programs and services are impacting participants’ outcomes. For scaling grants, grantees will already have this data, and for laboratory grants, they are likely to collect that data during the grant period. Our current focus is to support organizations that can demonstrate their activities, programs, and services are increasing incomes for their participants. This means collecting outcome metrics. Outcome metrics measure the change in livelihoods participants experience after receiving a program’s products or services. Outcome data can be collected from surveys (via in-person, email, text message, phone), or through available administrative data (LinkedIn profiles, employer starting salary records, tax return records, Unemployment Insurance records, etc.). We don’t expect every participant to be surveyed, rather, representative surveys of program’s participants are a reliable way to estimate the outcomes of a program. We advise grantees to apply best practices to increase response rates and mitigate non-response bias when possible.

Impact measurement is a never-ending journey, and we respect that many grantees are working to develop their measurement approaches further. We provide capacity-building grants through our annual Learning for Action Fund. Read more about how we support grantees to build their own impact evidence here.

We don’t require outcome data for comparable non-participants, also known as the counterfactual. However, we will ask grantees if they have an estimation of what outcomes their participants would experience if they did not have access to their organization. Often we look at starting incomes before individuals receive services, or look at publicly available data to estimate comparable annual incomes for similar individuals. We assess the level of grantee evidence through the Colorado State Evidence Continuum Model. Laboratory grants might begin at Step 1 or Step 2, but we expect all grantees to measure outcomes, Step 3 by the end of the grant period:

https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2022/01/colorados-evidence-continuum-promotes-efficient-effective-public-programs