1. What Makes the GitLab Foundation’s Approach Unique?

1. We hold ourselves accountable to our North Star Return on Investment (NSROI) prediction
   1. Our foundation achieves its goals once impact is measured and validated. Our North Star ROI model helps inform which investments we make, and our monitoring and evaluation process determines the Foundation's actual ROI.
   2. We celebrate when outcomes have been achieved, not when they are estimated.
   3. Our goal is to create impact that would not have occurred if it were not for our investment. Thus, we define success as improving outcomes when compared to a counterfactual. This means we don’t just compare how incomes change before and after an intervention - we also estimate how similar individuals’ incomes change during the same period.
2. We support grantees in improving their measurement and evaluation capabilities
   1. Most funders do not support their grantees in their efforts to measure their impact. We recognize the complexity, cost, and effort it takes to conduct robust impact measurement, and we support grantees to improve their practices and build evidence.
   2. By supporting the creation of impact evidence, our grantees can gain the feedback they need to design more impactful and effective programs. We learn what programs work best, and the economic mobility sector as a whole benefits from our shared learning.
   3. Generating impact evidence can be difficult due to the lack of resources to invest in data collection. However, this lack of impact evidence often prevents nonprofits from receiving financial support. We support organizations in developing their impact data strategies to build more evidence.
3. We have a strong focus on strategic learning and iteration
   1. When impact evaluation is used only as an accountability measure, grantees are more likely to report numbers that make grantmakers happy and are less likely to benefit from the process. The GitLab Foundation seeks to work with grantees to create shared learnings that are mutually beneficial to the Foundation’s strategy and that lead to improving the grantee’s products and services. The Foundation only supports research and evaluation projects when the learnings will directly inform programmatic or strategic decisions.

2. Our Impact Monitoring and Evaluation Approach

1. Impact Monitoring and Evaluation Approach: We developed an impact monitoring and evaluation strategy to support our grantees and our Foundation in maximizing impact and improving impact strategies. Read more about our grantee impact monitoring and reporting process and requirements here.
2. Balancing Accountability and Efficiency: We recognize that impact reporting can be a burden for grantees. Our approach is to tailor our reporting requirements for each grantee, understand what data they already collect, and support impact measurement as needed. Some metrics will be required, and others will be customized for the specific goals of the grant. Impact data collection will occur through a survey platform that enables user-friendly, efficient data inputs.
3. Building an Impact Measurement Plan: All grantees will share their plan to measure the impact of their programs during the grant application process; GitLab Foundation’s Director of Impact Measurement and Analytics will collaborate with the grantee to learn if there are ways that the Foundation can support improvements to their impact management processes.
4. Semi-Annual Check-ins: Most grantees will have semi-annual check-ins with Program Officers, with quantitative reporting conducted annually after enough time has elapsed for meaningful data to be collected. These check-ins will track ongoing lessons learned, roadblocks, and progress.
5. Annual Impact Report: All grants will include an annual impact report that describes the outputs and outcomes that the grant has delivered during the year. This includes quantitative and qualitative data as well as insights on lessons learned during the year. Read more about our annual reporting requirements here.

3. Taking a Right-Sized Approach to Grant Evaluation

We take a ‘right-sized’ approach to investing in impact measurement and stakeholder learning activities. Read more about our approach to evidence-building here.

1. Evaluation of the Counterfactual: Our impact evaluation approach aims to measure a program's net effect relative to what would have happened had the program not existed. We analyze this with a variety of strategies to estimate the counterfactual impact.
2. High-Rigor Approaches: High-rigor, quasi-experimental, or experimental approaches will be used when there is a strong return on investment in doing so, i.e., when generating strong impact evidence will greatly benefit the grantee, the sector, or GitLab Foundation. This is typically determined by calculating how much additional investment could be generated if strong, positive results were measured. Larger grants are more likely to require rigorous impact evaluation approaches.
3. Less Rigorous Approaches: Low-rigor approaches are used when impact data is self-reported by the grantee and without a counterfactual control group. They can be appropriate in cases where there is existing strong research showing that a similar intervention already has a significant impact, when there are high-quality, publicly available datasets to create an estimated counterfactual from similar populations (such as national college graduation rates, or regional salary data), or when it is clear that a change is directly due to the intervention itself (e.g., a new clean water well in a rural village with no other sources of clean water nearby). In these cases, tools such as simple pre- and post-intervention studies may be used.
4. Supporting Formative Evaluation: There are many reasons why an organization may not be ready for a rigorous impact evaluation. The Foundation supports organizations in their learning journey which may include [formative evaluations](https://www.evalcommunity.com/career-center/formative-evaluation/#:~:text=Formative evaluation is a process,of the program or product.). These include needs assessments or process evaluations to improve program design or program implementation.

Right-Sized Approach: Supporting grantees at the right level of impact evaluation:

                                                                        ([Image Source](<https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2022/01/colorados-evidence-continuum-promotes-efficient-effective-public-programs>))

4. Further Reading

Grantee Monitoring and Impact Reporting

Five Goals of GitLab Foundation’s Monitoring and Evaluation Approach

Expectations of Impact Evidence

Evaluation for Action

Staff Instructions for Creating the Annual Grantee Impact Survey