PxD was founded on principles of applying rigorous research methods to develop and evaluate evidence-based programs and we continually strive to build upon this strong foundation. We seek to contribute to generalizable knowledge and publicly share research findings as much as possible, including publishing working papers and articles in peer-reviewed academic journals. We employ rigorous research methods with oversight from experienced researchers and collaborate with local partners to make sure research is conducted in a culturally and contextually appropriate manner.
Project: Farmer Promotors in Rwanda
Full citation: Martin Abel, Tomoko Harigaya, Michael Kremer, and Jessica Zhu. 2024. “How High To Raise The Bar? Causal Effect of Self-Set Goals”. Working paper. https://precisiondev.org/wp-content/uploads/2024/04/Rwanda_Goal_Setting_for_Extension_Agents_2024.04.15.pdf
Authors: Martin Abel, Tomoko Harigaya, Michael Kremer, and Jessica Zhu.
Abstract: We investigate the effect of self-set goals among decentralized agents. We elicited productivity goals from a sample of 10,187 volunteer agricultural extension workers in Rwanda and exogenously varied the salience of goals through reminders. Comparing people with the same goal ambition hence allows us to disentangle the effect of goal magnitude from that of agent type. We find that goal reminders increase a performance index by 0.08 standard deviations when they are sent on time. The relationship between goal magnitude and effectiveness of goal reminders follows an inverted U-shape. Our findings align with existing observational evidence and underscores the effectiveness of setting goals that are both ambitious and realistic.
Project: Lime trials with One Acre Fund in Kenya and Rwanda
Full citation: Fabregas, Raissa, Michael Kremer, Matthew Lowes, Robert On, and Giulia Zane. 2024. “Digital Information Provision and Behavior Change: Lessons from Six Experiments in East Africa.” National Bureau of Economic Research: Working Paper 32048. https://doi.org/10.3386/w32048
Authors: Raissa Fabregas, Michael Kremer, Matthew Lowes, Robert On, and Giulia Zane.
Link: https://www.nber.org/papers/w32048
Abstract: Mobile phone-based informational programs are widely used worldwide, though there is little consensus on how effective they are at changing behavior. We present causal evidence on the effects of six agricultural information programs delivered through text messages in Kenya and Rwanda. The programs shared similar objectives but were implemented by three different organizations and varied in content, design, and target population. With administrative outcome data for tens of thousands of farmers across all experiments, we are sufficiently powered to detect small effects in real input purchase choices. Combining the results of all experiments through a meta-analysis, we find that the odds ratio for following the recommendations is 1.22 (95% CI: 1.16, 1.29). We cannot reject that impacts are similar across experiments and for two different agricultural inputs. There is little evidence of message fatigue, but the effects diminish over time. Providing more granular information, supplementing the texts with in-person calls, or varying the messages’ framing did not significantly increase impacts, but message repetition had modest positive effects. While the overall effect sizes are small, the low cost of text messages can make these programs cost-effective.
Project: Machine learning customization for Ama Krushi service in Odisha, India
Full citation: Susan Athey, Shawn Cole, Shanjukta Nath, Jessica Zhu. 2023. “Targeting, Personalization, and Engagement in an Agricultural Advisory Service”. Working Paper. https://doi.org/10.2139/ssrn.4536641
Link: https://precisiondev.org/wp-content/uploads/2023/11/Athey-et-al-2023.pdf
Abstract: ICT is increasingly used to deliver customized information in developing countries. We examine whether individually targeting the timing of automated voice calls meaningfully increases engagement in an agricultural advisory service. We define, estimate, and evaluate a novel recommendation system that customizes contact times to individual characteristics. This system generates significant gains, up to an 8% increase over the baseline pickup rate of 0.31. Our approach, delivered at scale, is well-suited for developing country settings. We show how to optimize around resource constraints, measure equity-efficiency trade-offs when targeting vulnerable groups, and evaluate the robustness of recommendations to technology or preference shocks.
Project: Krishi Tarang service in Gujarat, India
Full citation: Cole, Shawn, Tomoko Harigaya, Grady Killeen, and Aparna Krishna. 2023. “Using Satellites and Phones to Evaluate and Promote Agricultural Technology Adoption: Evidence from Smallholder Farms in India.” Working Paper.
Link: https://precisiondev.org/wp-content/uploads/2023/07/SoilFertility_Jul21.2023.pdf
Abstract: This paper evaluates a low-cost, customized soil nutrient management advisory service in India. As a methodological contribution, we examine whether and in which settings satellite measurements may be effective at estimating both agricultural yields and treatment effects. The intervention improves self-reported fertilizer management practices, though not enough to measurably affect yields. Satellite measurements calibrated using OLS produce more precise point estimates than farmer-reported data, suggesting power gains. However, linear models, common in the literature, likely produce biased estimates. We propose an alternative procedure, using two-stage least squares. In settings without attrition, this approach obtains lower statistical power than self-reported yields; in settings with differential attrition, it may substantially increase power. We include a “cookbook” and code that should allow other researchers to use remote sensing for yield estimation and program evaluation.
Project: Coffee Krishi Taranga service in Karnataka, India
Full citation: Cole, Shawn, Tomoko Harigaya, and Vaishnavi Surendra. 2023. “Probabilistic Weather Forecasts and Farmer Decision Making in Rural India.” Working paper.
Link: https://www.vaishnavisurendra.com/pdf/Weather_CHS.pdf
Abstract: Weather-induced risk reduces farmers’ incomes, and climate change is increasing such risk. One promising intervention to mitigate risk is high-quality, probabilistic, short-to-medium-range weather forecasts, which predict weather between zero and fifteen days ahead. For forecasts to be effective, however, farmers have to understand and act on them. This paper evaluates how farmers use probabilistic forecasts and form beliefs about their accuracy in a lab-in-the-field experiment. In scenarios that mimic real-world decision making, we find that farmers update their beliefs about the (in)accuracy of forecasts following false alarms, where forecasts erroneously predict events. Farmers who experience false alarms perform worse in subsequent rounds of incentivized experimental games, and report a lower willingness-to-pay for a real-world weather forecast service in an incentive-compatible Becker-DeGroot-Marschak elicitation. Light-touch interventions to improve probability comprehension and make climate change salient have limited impact on farmer decision-making, with positive effects that are mitigated by the incidence of false alarms.
Project: ElimuLeo service in Kenya
Full citation: Walter, Torsten Figueiredo, Guthrie Gray-Lobe, and Sarah Kabay. 2023. “Can Brick Phones Bridge the Digital Learning Divide? Evidence from SMS-Based Math Practice.” EdWorkingPaper: 23-791. Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/j4kc-kq39.
Link: https://edworkingpapers.com/ai23-791
Abstract: Hardware requirements are a barrier to widespread adoption of digital learning software among low-income populations. We investigate the demand among smallholder-farming households for a simple, adaptive math learning tool that can be accessed by widely available “brick” phones, and its effect on educational outcomes. Over a quarter of invited households used the tool, with greater demand among households lacking electricity, radios, or televisions. Usage was highest when schools were out of session. Engagement lapsed without regular reminders to use the service. Using random variation in access to the service, we find evidence that the platform increased test scores, school attendance, and grade attainment. Interpretation of these estimates is complicated by potentially endogenous outcome observation.
Project: Uganda Coffee Agronomy Training (UCAT) program
Full citation: Hoffmann, Vivian, Miki Khanh Doan, and Tomoko Harigaya. 2023. “Self-Selection versus Population-Based Sampling for Evaluation of an Agronomy Training Program in Uganda.” Journal of Development Effectiveness, July 2023: 1–11. https://doi.org/10.1080/19439342.2023.2236080
Link: https://www.tandfonline.com/doi/abs/10.1080/19439342.2023.2236080
Abstract: One of the challenges in evaluating the impact of agronomy training programs, particularly on downstream impacts such as yield, is identifying a sample of farmers who are likely to participate in the training. We assess farmers’ participation in a farm business training activity before the agronomy training intervention as a sample identification mechanism. The screening activity was designed to appeal to the same group of farmers targeted by a coffee agronomy training program, while having minimal impact on the program’s goal of increasing coffee yields. A three-session training on farm business management was conducted in 22 study villages in central Uganda. Coffee agronomy training was then offered in half of these villages, based on random assignment. The results show that 52% of coffee farmers who attended the first business training session subsequently attended agronomy training, compared to 22% of those identified through a census. Applying these results to the design of a large ongoing randomised controlled trial, we find that using a self-selected sample reduces the minimum detectable effect of agronomy training on coffee yield to 15.83%, compared to 38% if population-based sampling were used.
Project: Pest Risk Information Service (PRISE) and MoA-INFO service in Kenya
Full citation: Taylor, Bryony, Henri Edouard Zefack Tonnang, Tim Beale, William Holland, MaryLucy Oronje, Elfatih Mohamed Abdel-Rahman, David Onyango, Cambria Finegold, Jessica Zhu, Stefania Pozzi, and Sean T. Murphy. 2023. “Leveraging Data, Models & Farming Innovation to Prevent, Prepare for & Manage Pest Incursions: Delivering a Pest Risk Service for Low-Income Countries.” Science and Innovations for Food Systems Transformation, January 2023: 439–53. https://doi.org/10.1007/978-3-031-15703-5_23
Link: https://link.springer.com/chapter/10.1007/978-3-031-15703-5_23
Abstract: Globally, pests (invertebrates, vertebrates, pathogens, weeds) can cause estimated annual losses of between 20% and 40%, but higher losses are disproportionately experienced by many low-income countries, as agriculture is the mainstay of the majority of the people and of national economies. Pests pose a major barrier to these countries’ ability to meet the aims of the UN Sustainable Development Goals (SDGs), particularly SDG2, “End hunger, achieve food security and improved nutrition and promote sustainable agriculture.” However, solutions, in the form of pest risk alert systems, coupled with major advances in technology, are now providing opportunities to overcome this barrier in low-income countries. In this paper, we review these systems and the advances in data availability, management and modeling and communication technology and illustrate how these can provide new and novel solutions for the development of agricultural pest and disease early warning and risk-mapping systems and contribute to improved food systems in low-income countries. In conclusion, we identify key areas for the UNFSS that will help guide governments in engaging with these developments.
Project: Weather advisory service in Pakistan
Full citation: Viviano, Davide and Jess Rudder. 2023. “Policy design in experiments with unknown interference.” Working paper.
Link: https://jrudder4.github.io/papers/Policy_Design_Nov23.pdf
Abstract: This paper studies experimental designs for estimation and inference on policies with spillover effects. Units are organized into a finite number of large clusters and interact in unknown ways within each cluster. First, we introduce a single-wave experiment that, by varying the randomization across cluster pairs, estimates the marginal effect of a change in treatment probabilities, taking spillover effects into account. Using the marginal effect, we propose a test for policy optimality. Second, we design a multiple-wave experiment to estimate welfare-maximizing treatment rules. We provide strong theoretical guarantees and an implementation in a large-scale field experiment.
Project: Digital agricultural extension insights
Full citation: Fabregas, Raissa, Tomoko Harigaya, Michael Kremer, and Ravindra Ramrattan. 2022. “Digital Agricultural Extension for Development.” Chapter in: Temina Madon, Ashok J. Gadgil, Richard Anderson, Lorenzo Casaburi, Kenneth Lee, Arman Rezaee (eds). Introduction to Development Engineering. Springer, Cham: 187–219 https://doi.org/10.1007/978-3-030-86065-3_8
Link: https://link.springer.com/chapter/10.1007/978-3-030-86065-3_8
Abstract: Providing information at scale about improved agricultural practices to smallholder farmers remains a challenge in most developing countries. Traditional dissemination methods like in-person meetings or radio programming can be costly to scale or offer too generic information. Moreover, while most agronomic recommendations focus on maximizing crop yields, farmers weigh multiple other factors when making farming decisions, such as the profitability of investments and risks. The proliferation of mobile phones has shifted these trends. Mobile agriculture extension can cost-effectively provide tailored suggestions to farmers and improve their use of information. This case study describes the use of digital extension technologies to support farmers in a number of contexts. We draw insights from various studies and the experience of Precision Development on the importance of human-centered design, monitoring, and continuous experimentation. The chapter also discusses the ecosystem of stakeholders for digital agriculture, concerns relating to privacy and financing, and how mobile services can be used to facilitate social learning.
Project: Ama Krushi service in Odisha, India
Full citation: Kasy, Maximilian, and Anja Sautmann. 2021. “Adaptive treatment assignment in experiments for policy choice.” Econometrica 89, no. 1 (2021): 113-132. https://doi.org/10.3982/ECTA17527
Link: https://drive.google.com/file/d/1SVAitUI9R5NgDbrp6R-Q2i7Cw-LfXFxx/view?usp=sharing
Abstract: Standard experimental designs are geared toward point estimation and hypothesis testing, while bandit algorithms are geared toward in-sample outcomes. Here, we instead consider treatment assignment in an experiment with several waves for choosing the best among a set of possible policies (treatments) at the end of the experiment. We propose a computationally tractable assignment algorithm that we call “exploration sampling,” where assignment probabilities in each wave are an increasing concave function of the posterior probabilities that each treatment is optimal. We prove an asymptotic optimality result for this algorithm and demonstrate improvements in welfare in calibrated simulations over both non-adaptive designs and bandit algorithms. An application to selecting between six different recruitment strategies for an agricultural extension service in India demonstrates practical feasibility.
Project: Avaaj Otalo service in Gujarat, India
Full citation: Cole, Shawn A., and A. Nilesh Fernando. 2020. “‘Mobile’izing Agricultural Advice: Technology Adoption, Diffusion, and Sustainability.” The Economic Journal 131 (663): 192–219.
Link: https://doi.org/10.1093/ej/ueaa084
Abstract: Mobile phones promise to bring the ICT revolution to previously unconnected populations. A two-year study evaluates an innovative voice-based ICT advisory service for smallholder cotton farmers in India, demonstrating significant demand for, and trust in, new information. Farmers substantially alter their sources of information and consistently adopt inputs for cotton farming recommended by the service. Willingness to pay is, on average, less than the per-farmer cost of operating the service for our study, but likely exceeds the cost at scale. We do not find systematic evidence of gains in yields or profitability, suggesting the need for further research.
Project: Sugarcane contract farming hotline experiment in Kenya
Full citation: Casaburi, Lorenzo, Michael Kremer, and Ravindra Ramrattan. 2019. “Crony Capitalism, Collective Action, and ICT: Evidence from Kenyan Contract Farming.” Working paper.
Abstract: The shift from subsistence to commercial economies creates surplus, but often induces conflict over it. Under extractive institutions and weak contract enforcement, crony capitalism may emerge and limit the benefits of modernization. We examine the relationship between a large sugar cane contract farming company and small farmers in Western Kenya, in a setting with many features of crony capitalism. We document frequent violations of the company’s contractual obligations and propose a simple theory of how farmers’ collective action problems may make it harder to enforce contracts. We then test the direct effects of an ICT-based intervention that reduces farmers’ cost of complaining, potentially addressing company’s moral hazard and farmers’ free riding problems.
Project: Meta analysis of digital agricultural advisory experiments
Full citation: Fabregas, Raissa, Michael Kremer, and Frank Schilbach. 2019. “Realizing the Potential of Digital Development: The Case of Agricultural Advice.” Science 366 (6471): 30–38. https://doi.org/10.1126/science.aay3038.
Link: https://science.sciencemag.org/content/366/6471/eaay3038
Abstract: Mobile phones are almost universally available, and the costs of information transmission are low. They are used by smallholder farmers in low-income countries, largely successfully, to optimize markets for their produce. Fabregas et al. review the potential for boosting mobile phone use with smartphones to deliver not only market information but also more sophisticated agricultural extension advice. GPS-linked smartphones could provide locally relevant weather and pest information and video-based farming advice. But how to support the financial requirements of such extension services is less obvious, given the unwieldiness of government agencies and the vested interests of commercial suppliers.
Project: Soil health cards advisory in Gujarat, India
Full citation: Cole, Shawn, and Garima Sharma. 2017. “The Promise and Challenges of Implementing ICT in Indian Agriculture.” India Policy Forum 2017-2018. New Delhi.
Link: https://precisiondev.org/wp-content/uploads/2022/07/IPF-2017-Cole-Sharma-Conf-vesion.pdf
Abstract: While agricultural productivity in the developing world has made tremendous advances in the past half century, productivity still lags well behind the developed world. One particularly promising path to improve agricultural productivity is to employ mobile phones to enable farmers to make better decisions: through advice on input choices, farming decisions, and input and output prices. This paper takes a close look at the potential of ICT to improve input decisions by assisting with the delivery of customized information about soil nutrient status (“health”). In South Asia, fertilizers are often overused or applied in inefficient proportions. Governments in India have invested heavily in soil testing, with the goal of distributing 140 million “Soil Health Cards” (SHCs) directly to farmers. Yet absent additional information, farmers may have difficulty acting on the information provided in SHCs. The primary contribution of this paper is to evaluate the prospects for ICT to assist in the delivery of information about site-specific agricultural practices. We report on results from a field experiment examining whether audio and video supplements contribute to the understanding of information in SHCs. We begin examining the reach of traditional extension services in India, and find that they fall far short of universal coverage. If extension agents are not available, many farmers turn to local agricultural sales agents for advice. We describe results from an audit study evaluating the nature and quality of advice from these agents in the field.