Impact Evaluation

Our theory of change is rooted in the belief that an increase in health knowledge leads to an increase in health-promoting behaviors, which ultimately results in improved health outcomes.

Our evaluation team works with both Stanford faculty experts across academic disciplines and collaborators around the world to identify opportunities for improvement in digital learning models. From smaller-scale case studies to large-scale randomized evaluations, we leverage both quantitative and qualitative methods to pilot and rigorously evaluate the use of digital health content.

We are a collaborative learning lab and share research to promote global advances in health education. We also use these learnings to iterate on our own digital health content creation and development.

Our Process

We apply a multipronged approach to evaluation through which we examine not just the effectiveness of interventions as an entirety, but also unpack the mechanisms of educational models to better understand the theory of change.

We examine a continuum of outcomes to understand the mechanisms of interventions, recognizing that changes to long-term health improvements must follow immediate impact on beneficiaries’ knowledge, skills acquisition, and behavior change.

Recent publications

Feb 2023: The Use of Open-Source Online Course Content for Training in Public Health Emergencies: Mixed Methods Case Study of a COVID-19 Course Series for Health Professionals

Jan 2023: Barriers to COVID-19 vaccine acceptance to improve messages for vaccine uptake in indigenous populations in the central highlands of Guatemala: a participatory qualitative study

Dec 2022: Evaluating the impact of a linguistically and culturally tailored social media ad campaign on COVID-19 vaccine uptake among indigenous populations in Guatemala: a pre/post design intervention study

Jun 2022: Development and Field-Testing of Proposed Food-Based Dietary Guideline Messages and Images amongst Consumers in Tanzania

Jun 2022: “The Videos Gave Weight to Our Work”: Animated mHealth Videos and Tablet Technology Boost Community Health Workers’ Perceived Credibility in Khayelitsha, South Africa

Mar 2022: Digital education for health professionals: An evidence map, conceptual framework and research agenda

Sep 2021: Evaluation of a community-based mobile video breastfeeding intervention in Khayelitsha, South Africa: The Philani MOVIE cluster-randomized controlled trial

Jun 2021: Design preferences for global scale: a mixed-methods study of “glocalization” of an animated, video-based health communication intervention

Dec 2020: The Role of Non-governmental Organizations in Community-based COVID-19 Education: A Qualitative Study in South Africa and Zambia

Apr 2019: The Philani MOVIE study: a cluster-randomized controlled trial of a mobile video entertainment-education intervention to promote exclusive breastfeeding in South Africa

Randomized Trial:

Story-based Breastfeeding Promotion in South Africa

Despite progress in reducing maternal, newborn and child mortality, more than 5 million children globally die before reaching their fifth birthday. Many of these deaths are the result of preventable diseases, which could be reduced by implementing health-promoting behaviors like exclusive breastfeeding.

Read more

Narrative, story-based entertainment-education approaches to health education have emerged as potentially powerful strategies for promoting positive health behavior change. The characteristics of particularly impactful entertainment education include appealing narratives, high-quality production, persuasive and unobtrusive messages, and high potential for involvement or identification with the presenters or characters portrayed. 

To better understand how such approaches can be used to promote exclusive breastfeeding, we worked with the Philani Maternal, Child Health and Nutrition Trust to conduct a cluster randomized controlled trial measuring the effect of a story-based video intervention developed by Digital Medic.

The trial took place in the Western Cape of South Africa, an area that has one of the lowest exclusive breastfeeding rates in the world. The intervention was delivered via Philani community health workers. In the control arm, health workers provided standard of care perinatal in-home counseling, while health workers in the treatment arm provided standard of care plus the video intervention. Learn more about the study and our learnings here.

Case Study:

Vaccine Education via Social Media for Indigenous Populations in Guatemala

The COVID-19 pandemic has highlighted several barriers to vaccination for Indigenous community members in rural Guatemala, including a lack of credible health information in local Mayan languages and pervasive misinformation.


Read more

We collaborated with Wuqu’ Kawoq | Maya Health Alliance and the University of California San Francisco to 1) better understand the barriers to vaccination among Indigenous populations in rural Guatemala, 2) create and promote a series of culturally and linguistically relevant health education videos on how vaccines work as well as debunking common myths, and 3) measure the impact of these videos through social media measurement tools as well as in-person surveys. 

We will share results of this collaborative work soon; learn more about the project and see all educational videos here.

Exploratory Pilot:

Advocacy Training for Community Health Workers

Community health workers (CHWs) are trusted care providers for millions of people around the world. Despite their importance, CHWs have historically been excluded from conversations and decision-making that affect their work and communities.


Read more

To help address this gap, we created an “Advocacy Training for Community Health Workers” in collaboration with the Community Health Impact Coalition and an advisory group of CHWs from Kenya, Uganda, Liberia, South Africa, Sierra Leone, Malawi, and Guatemala. This free mobile training equips CHWs with the skills to share their stories and promote the health issues most important to their communities.

We also created a complementary Facilitator Guide to assist CHW supervisors and trainers in conducting in-person and hybrid group learning sessions. The guide is being piloted by CHWs who have completed the advocacy training. Learn more about the guide on our blog.

Our evaluation team is working to measure learner engagement with the course and facilitator guide. Through focus group discussions, we seek to understand how virtual training can impact the knowledge, attitudes, and practices of CHWs. The findings as of July 2021 are reflected in this Progress Report. Additional work is underway to assess the reach and impact of this training on CHW knowledge and beliefs.

A free advocacy training for CHWs around the world.

Current research

  • Evaluating the effectiveness of a social media campaign to promote COVID-19 vaccination among indigenous families in Guatemala with the University of California San Francisco and Wuqu’ Kawoq – Maya Health Alliance
  • Investigating the impact of interactive voice response COVID-19 vaccinator training in the Democratic Republic of Congo with Viamo
  • Assessing the impact of advocacy training for CHWs with the Community Health Impact Coalition
  • Evaluating the impact of WhatsApp delivered health information on maternal child health outcomes in India with Noora Health 
  • Collaborating with the OpenWHO to investigate the reach of online training for health workers 
  • Understanding health education barriers among migrant and refugee communities in South Africa with the Scalabrini Centre of Cape Town
  • Investigating the longterm use of breastfeeding education videos in government clinics in the Western Cape of South Africa with La Leche League
  • Identifying factors that impact viewership and engagement with health condition videos on YouTube