Staff instructions for completing an impact evidence risk assessment.

Step 1

Complete the impact evidence risk assessment during the final North Star ROI process.

Go to the North Star ROI Model TEMPLATE has a tab called ‘Impact Evidence Risk’

Step 2

  1. Use the section of the final application on ‘Measurement’ to determine the appropriate level of impact measurement capabilities and processes the grantee has. There are answers provided by the grantee to the question, “Please share how you currently collect program metrics and outcomes.” and “Is there anything else you would like to share about tracking measurement and your approach to assessing outcomes?”

  2. There are four sub-components of the impact evidence risk.

    1. **Literature Review Generalizability -**The level of external evidence associated with the type of the grantee’s intervention or program
    2. Grantee Outcome Data - The amount of outcome data the grantee has for their program participants
    3. Grantee Impact Rigor - The overall level of rigor of impact data that the grantee has
    4. Customer Data System - The strength of the grantee’s participant data tracking system

    Step 3

    1. Estimate the most appropriate score for each of the four sub-components.
      1. The literature review should not take more than 2 hours per grantee
      2. For scaling or systems change grants, this impact evidence risk template should also be completed by the grantee
    2. Highlight the cell that is most appropriate for each sub-category
    3. Record the value for each of the four sub-categories. 1 = high impact risk, 5 = low impact risk
    4. Find the average across the four sub-categories

    Untitled

    Step 4

    1. Share this score with the program officer during the next 1:1 meeting and/or at the monthly grant review meeting.
    2. If there is high impact evidence risk and this is a scaling grant, the program officer and impact measurement team members should ask the applicant for their plan to improve their impact measurement process.
    3. A score of 4 or higher (high evidence risk) can be approved for a laboratory grant if the grantee also has plans to improve their impact measurement processes.
    4. A score of 4 or higher (high evidence risk) should not be approved for a scaling grant as the impact evidence risk is too high to warrant a scaling grant.
    5. Grantees can include impact measurement-focused budget expenses to their overall budget. We have approved grants that cover the cost of a data analyst or program evaluation costs.
    6. Remind grantees of the impact evidence fund, which invites interested grantees to apply for up to $50,000 in additional grant funds that are focused on improving impact evidence and participant feedback activities.