Despite hundreds of pilot studies, the general consensus among the mHealth community is that we lack sufficient evidence to inform scale up of programs. At the mHealth for Maternal Health technical meeting in Boston last month, participants provided insight into their idea of adequate evidence, the importance of evidence, and persistent gaps in evidence related to mHealth.
Randomized control trials (RCTs) are inarguably the gold standard for generating evidence regarding the effectiveness of new interventions. An RCT is the only type of study design that allows researchers to directly attribute outcomes to a particular intervention. This presentation by Dr. Caroline Free at the Clinical Trials Research Unit, London School of Hygiene and Tropical Medicine, provides an overview of conducting RCTs. This overview is based on evaluations of mHealth interventions, such as using text messages for smoking cessation, and systematic reviews evaluating the effectiveness of mobile phone messaging on health behavior change and health care delivery .
However, RCTs can be expensive, both in terms of time as well as money, and sometimes may not feasible due to the rapid pace of technological innovation. Further, many mHealth interventions lack control groups and very few have designs that facilitate rigorous evaluations. Technical meeting participants mentioned that RCTs would be essential in cases where implementing the mHealth tool could lead to a significant disruption in the health system, such as task shifting. However, in other settings evidence from observational studies, qualitative evaluations and evaluations using service data would also be acceptable to them.
As the success of a program is largely dependent on the details of the implementation rather than the effectiveness of the mHealth tool by itself, participants also stressed the importance of adequate operational research. Very often, the most important evidence comes from users themselves. For instance, Reid Miller, an independent consultant on mHealth and implementation science for RMNCH explains, “I take notice of a mobile solution for maternal health when there is evidence that it increases the demand, uptake, and/or correct use of evidence based health interventions. For an mHealth application that directly targets the client, an example would be evidence that it increases their adherence to a PMTCT treatment regimen or provides them health information on family planning to seek out and obtain services. When the provider is the user of the mobile application, an example would be evidence that an mHealth application helps them refer women for emergency obstetric services, and that they are more likely to obtain those services as a result of the application.”
In her presentation, Dr. Free also stressed the importance of conducting cost analysis for new interventions. Along with reliable evidence that the intervention works, policymakers and government officials are often most interested in how much implementation will cost. Having adequate cost data will enable faster adoption and scale up of effective interventions.
Dr. Free also emphasizes that one of the biggest gaps in evidence currently is the lack of trials in low-resource settings. This can largely be attributed to the high cost of conducting RCTs and the logistical challenges involved in coordinating evaluations. However, the results of trials conducted in high-resource settings may not always be applicable to low-resource settings. At the very least, health communication interventions require local adaptation. Thus, evaluations in low-income settings are needed.
In the absence of trials, groups such as the WHO’s mHealth Technical Evidence Review Group for RMNCH (mTERG) have taken an innovative approach to collecting evidence. The mTERG goes “beyond the peer-reviewed” by also including articles, reports, blogs, presentations using extended strategy, ‘sleuthing’ references and links in their assessment of the quality of evidence available for mHealth interventions. Among the applications of mHealth for health systems strengthening (Labrique et al., 2013), the current areas of focus include the use of mHealth in improving provider adherence to treatment guidelines and reducing stock-outs of essential maternal health drugs and commodities.
An overarching theme that emerged from these discussions was that mHealth should be viewed as a health systems catalyst. Consequently, rather than focusing on whether mHealth works, we should shift the focus of evidence generation to whether mHealth optimizes what we know works.
According to Reid Miller, “With critical health worker shortages in Sub-Saharan Africa and throughout the developing world, evidence demonstrating that a mobile solution results in time savings by enabling efficiencies in data collection, reporting and other day-to-day tasks, thus freeing up time for indispensable clinical services would be noteworthy. Evidence that mobile technologies overcome a critical logistical challenge in the health system that was previously insurmountable, such as managing health commodities systems, enabling health management information systems or providing training/updates for community health workers would also be worth paying attention to.”
Additional materials from the meeting including presentations and videos can be accessed here.
Do you have an opinion on the role mHealth can play to improve maternal health? What do you see as the biggest advantages of mHealth? The limitations? If you are interested in submitting a blog post for our ongoing guest blog series on mHealth for Maternal Health, please email Natalie Ramm.