Impact Evaluation

Our collaboration with local organizations does not end with the creation of content. We continue to work closely with our collaborators to deploy and measure the impact of our work.

Engaging Stanford faculty experts across academic disciplines, we function as a learning lab to identify ways to improve on digital learning models. We leverage both quantitative and qualitative methods to pilot and rigorously evaluate the use of digital health content.

We apply a multipronged approach to evaluation through which we examine not just the effectiveness of interventions as an entirety, but also unpack the mechanisms of educational models to better understand the theory of change.

From smaller-scale case studies to large-scale randomization evaluations, we iterate with user feedback on content creation and development. We then examine a continuum of outcomes to understand the mechanisms of interventions, recognizing that changes to long-term health improvements must follow immediate impact on beneficiaries’ knowledge, skills acquisition, and behavior change.

The videos help the healthcare workers. It legitimizes the message that they’re sending. So it becomes a confirmation.
Nomzamo Matodlana
Program Manager - Philani Maternal, Child Health and Nutrition Project

Our Process

We apply a holistic approach to evaluation through which we not only examine the effectiveness of interventions as an entirety, but also unpack the mechanisms of educational models.

Example: The role of Digital Medic evaluation in the Philani RCT project

Content Creation: We conducted interviews with community members and key local stakeholders to understand the needs of the community

Content Distribution: We found, through focus groups, that providing community health workers with tablets loaded with our health education videos, was a feasible and acceptable way to disseminate content to communities without access to smartphones.

Knowledge & Beliefs: We measured maternal knowledge and other infant feeding practices.

Behavior: We measured exclusive breastfeeding at 1 and 5 months.

Health Outcomes: We reviewed impact on several child health indicators.

Our current research

  • Evaluating the effectiveness of a social media campaign to promote COVID-19 vaccination among indigenous families in Guatemala with the University of California San Francisco and Wuqu’ Kawoq – Maya Health Alliance
  • Investigating the impact of interactive voice response COVID-19 vaccinator training in the Democratic Republic of Congo with Viamo
  • Assessing the impact of advocacy training for CHWs with the Community Health Impact Coalition
  • Evaluating the impact of WhatsApp delivered health information on maternal child health outcomes in India with Noora Health 
  • Collaborating with the OpenWHO to investigate the reach of online training for health workers 
  • Understanding health education barriers among migrant and refugee communities in South Africa with the Scalabrini Centre of Cape Town
Randomized Trial:

Story-based Breastfeeding Promotion in South Africa

Despite progress in reducing maternal, newborn and child mortality, more than 5 million children globally die before reaching their fifth birthday. A majority of these deaths are the result of preventable diseases, the incidence of which could be reduced by implementing health-promoting maternal-child behaviors, such as exclusive breastfeeding.

Read more

Narrative, story-based entertainment-education approaches to health education have emerged as potentially powerful strategies for promoting positive health behavior change. The characteristics of particularly impactful entertainment education include appealing narratives, high-quality production, persuasive and unobtrusive messages, and high potential for involvement or identification with the presenters or characters portrayed. 

To better understand how such approaches can be used to promote exclusive breastfeeding, we worked with the Philani Maternal, Child Health and Nutrition Trust to conduct a cluster randomized controlled trial measuring the effect of a story-based video intervention developed by Digital Medic.

The trial took place in the Western Cape of South Africa, an area that has one of the lowest exclusive breastfeeding rates in the world. The intervention was delivered via Philani community health workers. In the control arm, health workers provided standard of care perinatal in-home counseling, while health workers in the treatment arm provided standard of care plus the video intervention. Learn more about the study and our learnings here.

Case Study:

Video-based Community Health Training in Rajasthan

In recent years, India has made strides in improving maternal and neonatal health; however the risks to mothers and newborns are still far greater in India than most places in the world. Community health workers serve as the first point of care for the majority of mothers and children in India, and yet they often have limited knowledge and support to address the needs of the populations they serve.

Read more

To address this gap, we collaborated with the Antara Foundation and the Government of Rajasthan to pilot video-based maternal and child health training content via our offline app to 40 community health workers serving 18 villages and a population of roughly 15,000 in rural Rajasthan.

We created the videos as part of a foundational series that covers key topics in maternal child health including recognizing danger signs in late pregnancy and delivery, nutrition for new mothers and babies, breastfeeding, and early newborn care. While the series was first created for use in South Africa, using inputs from experts and community members in India, we adapted the images, narration and content to ensure relevance and cultural appropriateness.

Exploratory Pilot:

Investigating Components of Hospital Caregiver Training

Family caregivers play an integral role in the well being of individuals in need of medical care. However, they are often ill-equipped to recognize and meet the needs of their family members, particularly after hospitalization. A growing body of work suggests that educating family caregivers is a cost-effective way to improve patient outcomes and reduce hospital readmission rates, particularly in low resource areas where follow-up care options are limited.

Read more

Noora Health is a not-for-profit organization, founded at Stanford University and based in Bangalore, India that has recognized the importance of integrating family caregivers into the health delivery process. Noora Health developed an in-hospital caregiver training model that they currently deliver in dozens of urban and rural hospitals across India. Noora trainers leverage visual tools including videos and flip charts to teach patients and their family members about best practices for follow-up care.

Starting in May 2018, Digital Medic and Noora Health entered a collaboration to better understand how the different components of the Noora Health caregiver training model contribute to improved patient practices and outcomes. Currently, we are piloting a variety of model modifications aimed at increasing the effectiveness of Noora Health’s patient and caregiver engagement. We are also exploring the potential for digital learning reinforcements after hospitalization.

Translate »
[contact-form-7 404 "Not Found"]