Educate! rigorously measures impact and is proven to improve youth life outcomes.

We’re committed to doing what works: utilizing evidence to develop and run skills-based models, continuously capturing efficacy data, processing it quickly, and integrating lessons learned to optimize quality and cost.

Educate!’s Monitoring & Evaluation Team plays a critical role in our product development process from pilot to scale, increasing the rigor and scope of the evaluation along the way. As we create new solutions for youth, the M&E Team supports our research and development through Rapid Impact Assessments — experiments to test the design and delivery of new solutions. This allows us to learn, almost immediately, what is working and how to help us target the most successful elements and generate the greatest impact.

Check out Rapid Evaluation for a Reshaped World to learn more

Once we can validate a new model’s impact on youth, we work to improve its efficiency, then work to scale the proven solution. At that point, our monitoring system ensures program quality and progress towards our targets by tracking 20+ key performance indicators (KPIs) on a regular basis and feeding data back into a dashboard in real time. Our evaluation philosophy employs rigorous studies to understand our impact on a longer-term basis, including an external randomized controlled trial and quasi-experimental evaluation.

 
 

Monitoring and Learning Tools

Featured
Build-Measure-Learn Loops
Build-Measure-Learn Loops

Educate! prioritizes ongoing innovation and learning in our core program model. One of our key learning mechanisms is Build-Measure-Learn (BML) loops, a type of feedback loop in which we Build a new or different element into program design, Measure its impact, and Learn from the results. BML loops enable Educate! to test programmatic and operational components of our model and to make rapid adjustments based on the results of each loop. For example, we used a BML loop to determine that using visual aids was more effective in the classroom than writing on the chalkboard. After analyzing the results of the BML, we were able to quickly integrate this lesson and get visual aids into all of the classrooms we work in.

SMS and Smartphone Monitoring
SMS and Smartphone Monitoring

Educate! uses SMS (text messaging) and smartphone surveys as key components of our M&E system. With the goals of rapid turnaround and robust program management, in 2014 we launched a sophisticated mobile money and telecommunications system allowing for rapid turnaround, robust program management, and cost-effective data collection via SMS text messaging and smartphones. These technologies allow Educate! to gather regular information from remote parts of the country in a very cost-effective manner. The metrics that field staff provide through this system populate a web-based dashboard that updates in real time and allows Educate! to monitor our performance at scale. For example, we can track student businesses started on a trimesterly basis, and optimize quality as necessary if our scale performance is lower than targeted.

Cross-Cutting Designs
Cross-Cutting Designs

Educate! uses experiments to inform and improve program design and implementation. A cross-cutting design is an a/b test that allows us to rigorously assess the impact of new program design components. For example, in 2014 we ran a cross-cutting design to test the effectiveness of group mentorship sessions, providing some of our students with lessons only and others with additional group mentorship. Our M&E team analyzed the data and reported to the Programs and Design & Training teams, who used the data to determine how group mentoring sessions would be implemented in the future.

 

Evaluation Philosophy

Educate!’s impact measurement philosophy relies on periodic, rigorous external evaluations to measure medium- and long-term outcomes. Our model has been extensively evaluated, including through two randomized controlled trials of our programs in Uganda and Rwanda. For these evaluations, Educate! is partnering with researchers from the University of California, Berkeley; Innovations for Poverty Action; Oregon State University; and the World Bank, with funding from the Abdul Latif Jameel Poverty Action Lab (J-PAL), the United Kingdom Department for International Development (DFID), and the Global Innovation Fund (GIF), among others.

These evaluations are examining our impact on the following outcomes: 21st-century skills development, educational attainment, community involvement and leadership, as well as economic outcomes

 
 

Target Outcomes

Educate! monitors four target outcomes to determine measurable impact of our model:

 
 
livelihoods.png

Improved Livelihoods

business ownership and employment.png

Increased Business Ownership and Employment

 

community participation.png

Increased Community Participation

workforce readiness.png

Improved Workforce Readiness