Evidence-Building Activities

The GitLab Foundation supports evidence-building activities in two ways, which we will begin piloting in June 2024:

  1. **Launching a Learning for Action Fund:** GitLab Foundation will invite existing grantees to submit proposals for stand-alone grants related to evidence-building - in particular, where the potential results could unlock dramatic program changes or scaling opportunities. This fund will enable grantees to apply for up to $50,000 in funds that can be used to bolster their impact measurement or participant feedback activities.
  2. Building Program Feedback Activities or Impact Assessments into Grants: During our standard grant development process, Program Officers and potential grantees may discuss incorporating client feedback or impact assessment activities into the proposal. This typically involves setting specific goals and budget line items, and conducting additional modeling work to ensure the minimum ROI threshold can still be met. Funds may be earmarked for grantees to work with specific consultants or vendors.

Find out more about our grantmaking strategies here.

Our Philosophy

  1. We seek to invest in and validate the most impactful ways to improve lifetime earnings.
    1. Because we desire to support the highest North Star ROI work, we look for evidence that the programs we invest in will succeed. We use research, credible data, and proofs-of-concept wherever possible to guide our decision-making.
    2. We also seek to continuously learn and validate results by working with our grantees to measure the impact that our investments have created. Evidence-building accrues long-term, collective benefits for the field that extend beyond any single organization.
  2. However, broken incentives discourage evidence-building in the nonprofit sector**.**
    1. The for-profit world has strong incentives to conduct end-user research and customer feedback activities, as an improved customer experience can lead to higher financial returns. Additionally, any changes to a product or service can be measured clearly by the change in revenues associated with that change (e.g., via A/B testing), helping to validate returns on investment easily.
    2. In contrast, the nonprofit sector lacks direct financial incentives to improve return on impact. Furthermore, conducting research or measuring a change in impact can be costly, difficult, and imprecise. Therefore, there is a broken incentive system for nonprofits to optimize the effectiveness of their programs.
  3. As a result, we have committed to investing in evidence-building activities when those investments are predicted to have a 100X ROI.
    1. We fund two distinct types of evidence-building - end-user research and impact evaluation to help our grantees uncover ways to improve their program effectiveness, drive improved outcomes, validate their impact, and improve the likelihood of accessing additional funding to scale their work.
    2. Our estimates demonstrate that cost-effective investments in customer research and impact evaluations can generate above a 100 to 1 return on investment.

Evidence to Action Drives North Star ROI

  1. Modeling Evidence-Building Investments: We only recommend investing in evidence-building when there is a clear action associated with the creation of impact evidence and when that action is predicted to improve lifetime earnings.
    1. Using educated assumptions, we model the predicted upside of evidence-building to unlock further outcomes to understand the potential impact of an investment in learning. Further outcomes could take the form of improved program design, prioritization of the most effective work, or crowding in additional funding due to evidence of effectiveness.
    2. As with our other grants, investments in evidence-building must surpass our threshold of $100 in predicted lifetime earnings gained for each $1 that we spend as a foundation.
  2. Creating the Conditions for Action: We attempt to intentionally link evidence-building activities to an action plan to ensure that results lead to impact.
    1. One effective way to ensure that evidence-building drives action is to develop an action plan with a nonprofit’s leadership in the grant proposal stage. Program Officers work with the nonprofit leaders to ensure they own and drive the work and are committed to taking action when the results come through.
    2. Another way to capture the upsides of evidence is to co-fund or showcase evidence-building work to like-minded philanthropies. The GitLab Foundation is willing to partner with other funders to co-sponsor impact evaluations to increase the likelihood that those funders will support scaling the programs that are shown to be effective.
    3. A third way we work to create conditions for action is by investing in studies that would produce evidence that meets government funding requirements - potentially unlocking much larger sources of funding and scale.
  3. Examples of Impact on ROI:
    1. Example 1: Program Feedback Investment Improves Program Design: An organization working with 1,000 students conducts a cost-effective ($20,000) and valid feedback survey and makes programmatic changes based on the feedback. If the program change led to annual earnings improvements of just 3 percent, the North Star ROI would be 373.
    2. Example 2: Demonstrating Impact Effectiveness Enables Scaling: A $300,000 investment in a rigorous impact study determines that a program measurably increases average annual incomes by 25%. This leads to additional investments from other funders, enabling the nonprofit to expand from impacting 100 people to 500 people per year after five years. This evidence-building grant has an ROI of 239.

Link to sub-page:

2024 Learning for Action Fund

Impact Measurement and Feedback Tools and Vendors

Client Feedback and Impact Studies Lead to More Effective Programs

Providing Grantee Support for Impact Evaluation, Monitoring, and Continuous Learning