Beyond Monitoring and Evaluation

Beyond Monitoring and Evaluation: 
Continuously Tracking Improved Cookstove Adoption Over Time to Achieve Lasting Success

Nearly 3 billion people in low- and middle-income countries rely on traditional cookstoves fueled by biomass, such as wood and dung, to cook and to heat their homes. Traditional cooking leads to high levels of household air pollution (HAP), including black carbon, which contributes significantly to climate change. Exposure to smoke from HAP causes 3.8 million premature deaths each year, including more than 1 million deaths in India. Improved cookstoves (ICS) that leverage fuels such as pellets and other biomass, electricity, and liquefied petroleum gas (LPG) are cleaner alternatives. Sadly, low uptake of ICS by intended users has blocked progress in the sector. A variety of barriers impede clean cookstove adoption, including stove designs that fail to meet the needs of rural women, limited financing options, equipment that is not user-friendly, a lack of after-sales service at the last mile, and a market model that is not sustainable for rural energy entrepreneurs.

We can’t fix problems we can’t see. If we do not track adoption of improved cookstoves continuously over time, stakeholders have little to no insight into ICS performance, user acceptability, or when design solutions are required to address these barriers.

Read the full report.

IoT for Development: StoveTrace

About 3 billion people rely on traditional cookstoves to prepare their food each day. To fuel their hand-made traditional stoves, people burn biomass like firewood inside their homes. This practice harms the families who live in these households and our planet as a whole.

Every year, four million people die from illnesses attributable to household air pollution. Black carbon (BC) is the second largest contributor to global warming, and emissions from traditional cooking practices are a significant global source of BC. For these reasons, changing how people cook has become a major public health and climate change priority.

The world rallied around a solution: introduce cleaner-burning, manufactured cookstoves for the rural poor. The only question that remained was how to make these “improved cookstoves” affordable for people who live on less than $2 a day…or so we thought.


IoT for Development: ColdTrace

When engineers tackle a complex problem, their first step is to gather lots of data. Take self-driving cars: what could be more challenging than teaching a car how to navigate a busy city full of obstacles? Engineers tasked with a problem like this start by figuring out how to collect enough data to help them understand the scale and scope of the challenge at hand, as well as to feed into machine learning tools. That’s why Tesla logs millions of miles of human driver data each day to seed their self-driving platform.

Getting life-saving vaccines to every child on Earth is also a massively complex problem. Vaccines have to travel across continents and through difficult conditions in low-income countries, where basic infrastructure is inconsistent or missing. Then those vaccines have to be stored in remote health clinics, some with less than 8 hours of grid power a day. And they have to be kept at safe temperatures until they’re finally administered to kids.

When something goes wrong – a vaccine fridge doesn’t work right, or a clinic loses power – the vaccines stored there can be damaged or even destroyed. Those doses are then wasted, or worse, they’re unknowingly administered to kids despite the damage. And a recent WHO – UNICEF joint statement reveals that 55% of these health facilities have fridges that either perform poorly or don’t function at all. Mothers often travel many miles to make sure their babies get vaccinated. What could be worse than leaving those children unprotected?


StoveTrace Report: Climate and Health Metrics

Transparent Climate and Health Metrics: An Open Data Dashboard and Wireless Platform for Cookstove Monitoring

Nexleaf and the World Bank collaborated on a project to monitor and evaluate cookstove use and subsequent PM2.5 reductions, to support sensor-enabled climate financing.  With support from Berkeley Air Monitoring Group and The Energy and Resources Institute, and in consultation with Scripps Institution of Oceanography, Nexleaf completed the first phase of a three phase project.

StoveTrace sensors were deployed in 79 households in four states in India, yielding unprecedented insight into user preference and behavior and spurring tremendous technological development of the platform. The investigation revealed that  the health benefits of cleaner air are less immediate and tangible, and do not hold much sway over women’s preference. In 10 additional households, prototypes of integrated PM2.5 devices were deployed, but further technological development and field testing are required to draw conclusions based on air quality data.

This work aims to scale an integrated sensor and increase the capacity of the open data platform accordingly.

Read the full report.

Coldtrace Report: Randomized Control Trial

uptime-month-boundariesRTM Increases Uptime and Decreases Freezing Duration in Vaccine Refrigerators: A Randomized Control Trial Analysis

The randomized control trial conducted by Nexleaf with the Mozambique Ministry of Health, VillageReach, PATH, and UNICEF evaluated the “uptimes” of vaccine refrigerators in rural communities. Health facilities using Nexleaf’s remote temperature monitoring (RTM) were evaluated in a controlled trial against facilities using 30 day temperature recorders (30DTR) and facilities using paper charts.

The study was conducted to learn: Do refrigerators with RTM devices capable of sending SMS alerts maintain higher uptimes than refrigerators monitored with 30DTRs or paper charts?

Health facilities were randomly selected and placed into three groups:

  • Group 1 consisted of 29 health facilities with RTM + SMS alerts; staff received training on responding to RTM alerts, and Standard Operating Procedures (SOPs) were posted in clinics.
  • Group 2 consisted of 28 health facilities monitoring fridges with 30DTR devices; staff received additional training on 30DTR usage.
  • Group 3 consisted of 26 health facilities monitoring fridges with stem thermometers; staff recorded temperatures twice daily on paper charts.

Following a ten-month evaluation period, we found that vaccine refrigerators equipped with RTM capable of sending SMS alerts, monitored by trained health care professionals, achieved the highest amount of uptime (90%) and lowest percent of total time spent in WHO freeze alarm state (0.55%). However, Group 2, refrigerators monitored by 30DTR, experienced an average of three hours less in WHO heat alarm state than those with RTM.

Read the full report.

ColdTrace Report: RTM Data for Maintenance

fridge-reportWhy Fridges Fail 2: RTM Data for Maintenance

This assessment follows up on the randomized control trial conducted in 2014-2015 with Village Reach, PATH, and the Ministry of Health in Mozambique which aimed to evaluate the impact of remote temperature monitoring (RTM) on vaccine cold chain equipment performance at the health facility level. That study showed that RTM with SMS alerts increased fridge uptime and reduced freezing, as compared to 30 day temperature loggers and paper charts. (See Appendix I of the report for more details.) However, even some fridges in the RTM experimental group had chronic problems and failed to achieve 95% uptime.

We launched this follow-up investigation to answer the question: Can maintenance + RTM lead to 95% uptime?

A cold chain expert traveled to Mozambique from September 24 until October 14, 2015. He used ColdTrace data to identify 27 fridges that were failing and worked with the Ministry of Health maintenance technician to visit or remotely diagnose and fix as many fridges as possible. The expert remained active in fridge fixes, phone calls and follow-ups through the end of November 2015.

Perhaps the most interesting finding was that distinct patterns in the RTM data (which we call “temperature data signatures”) can be used to remotely diagnose problems and enable remote fixes via phone calls to nurses in clinics. For the improperly adjusted fridges in this investigation, thermostat adjustments increased uptime by 30%. Today, the MOH technician continues to use the dashboard data to diagnose problems, call clinics to address the issues, and verify fixes by monitoring subsequent temperature data. We also learned that, without access to key spare parts/tools, even an expert technician cannot get all fridges to 95%: flat batteries on solar fridges were one such problem that the expert did not have the resources to fix. With RTM data, however, those responsible for cold chain maintenance can identify which fridges need help and determine where to focus resources.

Click here to read the full report. Or view a shortened version of Why Fridges Fail 2.