The benefits of using clean cookstoves in developing countries have long been accepted as important in improving health, creating safer homes, and minimizing the impact of cooking on climate change. Research suggests that a clean stove can reduce child pneumonia by 50%, save the equivalent of one to two tons of carbon dioxide per year, and lead to household savings on fuels — all of which have been sound reaons for organizations to develop initiatives aimed at effectively channeling clean cook stoves to low-income households. A new study conducted by professors from Harvard and MIT indicates however, that the impact of the stoves on rural households and the environment may not be as high as initially thought. Having conducted a randomized controlled trial in this area in India, the results of the study report that a major issue is both the initial and long-term adoption of the cookstoves by the intended users. More specifically, the study found that, over time, as the stoves broke down, households did not make the necessary repairs or investments in maintenance, leading them to revert back to the traditional stoves.
Cooking the family meal can be a dangerous business for poor people in developing countries. According to a study by the World Health Organization, indoor air pollution from “primitive household cooking fires” is the leading environmental cause of death in the world. In most rural homes, which lack electricity, a stove can be an open fire and the fuel as basic as wood, dried animal dung or agricultural residue, together known as “biomass” fuels. The result is nearly 2 million deaths a year, worldwide, almost as many as are caused by malaria and tuberculosis combined. What’s worse, cooking with biomass releases carbon dioxide and “black carbon” into the atmosphere and consequently is a significant contributor to climate change.

Could technology be the savior?

That seems to be the official view of the world’s development establishment. In September 2010, U.S. Secretary of State Hillary Rodham Clinton announced the creation of the Global Alliance for Clean Cookstoves. This is a public-private partnership led by the United Nations Foundation, with support from the governments of many developed countries, including the United States, and multinational enterprises such as the investment bank Morgan Stanley and the oil producer Shell. And their ambitious goal, “100 by 20,” is to get 100 million homes to adopt clean stoves by the year 2020. They’ve even secured the actress Julia Roberts as a global ambassador.

The Global Alliance isn’t modest about its goal, nor is it about the claims it makes on behalf of clean stoves. According to its web site, a clean stove reduces child pneumonia by 50 percent; saves the equivalent of one to two tons of carbon dioxide per year; and leads to savings in fuel costs that allows a stove to pay for itself. However, until recently, evidence to support these claims has been scant, and based on laboratory tests or highly controlled conditions in which stoves were frequently monitored and maintained by the researchers. That leaves open how robust the conclusions are.

Finally, the findings of an important new study, which tested the efficacy of clean stoves in real world, rather than laboratory conditions, have been released and the news isn’t good for the advocates.

In a working paper released this month, Rema Hanna, a professor of public policy at Harvard’s Kennedy School, and Esther Duflo and Michael Greenstone, economics professors at M.I.T., report the results of a randomized controlled trial on the adoption of clean stoves in India. Ms. Duflo, together with Abhijit Banerjee, also an economics professor at M.I.T., has championed the use of randomized experiments, like those used in testing new drugs, to measure the effectiveness of interventions like these.

Ms. Hanna and her colleagues worked with Gram Vikas, an award-winning nongovernmental organization, to provide 2,600 “improved” stoves to rural households in the eastern state of Orissa. Based on a public lottery, a third of the homes received a new stove at the beginning of the project, the second third got one about two years later and the final third were equipped at the end of the study period. Households were followed for four years after the initial adoption.

The study found that acceptance and, crucially, usage, of a new stove wasn’t universal even initially, and usage declined rapidly over time, as stoves broke down and households failed to make the necessary repairs or investment in maintenance.

Some households who’d voluntarily entered the lottery refused the new stove, even though it was heavily subsidized. Other households that accepted the improved stove continued to use their old stove alongside the new one – even when it was working well – before giving up on the new one altogether when it broke down.

Not surprisingly, the presumed health benefits didn’t show up either. While the study showed some health improvements for the primary cook in the household during the first year after installation, the effect vanished over time as proper use and maintenance of the new stove declined.

The researchers found, in the aggregate, no statistically valid evidence that the stoves had any measurable impact over a wide range of health outcomes, including lung function, infant birth weight, infant mortality, or even the chance of getting sick.

Lastly, and most disturbingly, the researchers found that those who adopted the new stoves actually saw a decline in their living standards compared with their neighbors who hadn’t. They spent more time maintaining their new stoves (even though, as we saw, this wasn’t enough to keep the stoves usable in many cases), which took them away from income-earning opportunities, and there was no significant reduction in fuel costs or time spent cooking.

Ms. Hanna gave me, via email, her interpretation of the results:

This isn’t to say that indoor air pollution is not a problem, or that an improved cooking stove cannot be part of the solution. But rather, we just don’t have enough evidence that the stoves systematically improve health, particularly under real world conditions where people do not regularly use the stoves, and if they do, the use often does not perfectly follow the manufacturer’s instructions. More research needs to be done….In the meantime, it may be wise to shift limited development resources to health programs (deworming, bed nets, conditional cash transfers) where there is strong evidence that they will improve lives.

This is a laudably measured conclusion in light of the damning findings of the study. But some in the news media haven’t been as circumspect and have gone instead for the jugular.

A sharply worded news article in The Washington Post was stinging enough to provoke a response from the Global Alliance. An opinion piece in the Boston Globe by Edward Glaeser, an economics professor at Harvard, was scarcely more muted, concluding bluntly: “Dumping stoves into the developing world isn’t going to alter the long-standing advantages of traditional methods.”

Mr. Glaeser is right. Evangelists for technological solutions, in their proselytizing zeal, often forget that these need to be implemented by human beings, not robots. To be effective, technology must be linked to economic or other behavioral incentives. Now, there’s a thought that should be allowed to simmer on the stove.

Vivek Dehejia is an economics professor at Carleton University in Ottawa, Canada, and a writer and commentator on India. You can follow him on Twitter @vdehejia.