by Maria Guadalupe*
When the 2019 Nobel Prize in Economics was announced, it was an exceptional moment because in addition to celebrating contributions to economic science, a methodology was also recognised. The laureates Abhijit Banerjee, Esther Duflo and Michael Kremer are known for their work in randomized controlled trials (RCTs) which develop evidence-based recommendations for public policy and organisations in the fight against global poverty.
Work with RCTs has led to the identification and implementation of impactful poverty-reduction policies in areas such as skills training and agriculture. But as a methodology, RCTs are not only for the developing world. They have been used for decades in pharmaceutical companies’ clinical trials and, in this age of data, they hold enormous promise to identify business solutions and support evidence-based management.
This methodology can challenge widely held assumptions about what gets results. Finding out what we think works but in fact is a waste of resources is one more reason to adopt RCTs more systematically. This year’s Nobel winners concluded after many trials that microfinance is not the solution to poverty that many had thought and hoped, and that in many cases, people without access to micro credit (the control group) did not fare better than those who did (the treatment group).
The purpose of the RCT Lab at INSEAD is to help bring this methodology to firms, by connecting companies to INSEAD faculty and supporting the joint research (funded by the Abu Dhabi Department of Education and Knowledge). Recently we combined two RCT conferences into a single workshop. It was a collaboration of Field Days: Experiments Outside the Laboratory (8th edition) and INSEAD Randomized Controlled Trials Days (4th edition). Professor Robert Dur of Erasmus University Rotterdam and I were pleased to host leading economists and other academics conducting field experiments who came together to present and discuss their work.
RCTs: from job search to food and health
Researchers at the conference covered a wide range of topics.
A number of papers focused on job search and matching firms to employees. For example, INSEAD’s Dylan Glover discussed his joint work on how the French public employment service stimulated firms’ labour demand by reducing their recruitment costs. Starting in 2015, the researchers analysed a programme that included new free services for SMEs, notably the pre-selection of job candidates. They found this programme increased the hiring of registered jobseekers by 10 percent, leading to 18 additional workdays created by treated firms, on average.
Instead of the traditional focus on policies that assist jobseekers, this study shows that firm-based active labour market policies may have significant added value in weak labour markets. In a country of relatively high unemployment like France, it is key to find solutions that work such as this one.
Also in the French context, Professor Pierre Chandon (INSEAD) and other researchers used a large RCT in 60 French supermarkets to establish if a simplified nutrition label placed on the front of packages in four food categories could lead consumers to purchase healthier foods over a three-month period. Evaluating the effectiveness of four pre-designed labels, the one with the largest impact translated into a 2.6 percent improvement in the average nutritional quality of consumers’ shopping baskets. “Because food preferences are deeply ingrained, and because most people eat for pleasure, not nutrition, simply providing nutrition information can only lead to small improvements on what people eat,” he said. “That’s why randomized controlled trials on a large scale and over a significant amount of time are indispensable in obtaining precise and reliable measures of the effects of nutrition information.”
Selection and delegation in organisations
Oriana Bandiera, one of our keynote speakers and this year’s winner of the Yrjo Jahnsson award (which often points the way towards the Nobel), presented her work on public sector workers and the importance of employee selection for effective bureaucracies: “A well-functioning bureaucracy requires having the right people and retaining them,” she said.
Motivation in employment – does the worker do the job for love or for cash – is a problem that Bandiera has considered in several articles over the years. One paper, about Zambia’s Ministry of Health recruiting nurses for rural positions, shows how dual motivation is a possibility.
The ministry recognised that there were two types of motivation to join a career in nursing: those who are interested in a good wage and those with a “good heart”. They worried that if they emphasised career goals in their recruiting, they would attract self-interested nurses rather than pro-social, caring nurses. But they needed more than caring nurses, they needed skilled caring nurses. So this gave rise to a project where Bandiera and her co-authors tested what the effect of emphasising the career aspects of the job did to the pool of applicants, and followed these applicants over time as they were hired by the organisation.
The findings show that, in line with common wisdom, offering career incentives encourages selfish applicants, but at the same time it encourages talented applicants. As the latter are also more pro-social and more likely to be selected, we end up with better workers.
In an ongoing experiment with Michael Best, Adnan Khan and Andrea Prat, Bandiera studies how the allocation of authority in a bureaucracy affects corruption: Should one grant more decision power to those lower down in the organisation? The field setting for this study was the public procurement service in Pakistan. After procurement officers find the necessary chairs, paper, etc., these purchases must be approved by the Accountant General, an onerous task involving a lot of documentation.
To study how the allocation of authority impacts layers of an organisation, the researchers compared the effect of monetary incentives with that of a change in decision power. In the first year, buyers were granted incentives (bonuses) based on their overall purchasing savings. In the second year, they were granted autonomy – freedom from filling in all the forms and abiding by certain rules.
The researchers found that granting authority at a lower level generated more savings. Prices paid by procurement officers went down 9 percent when the buyers had autonomy; down only 3 percent when they had incentives to find good bargains. This challenges the conventional wisdom that money is always the biggest motivator: Firms can improve performance by redesigning their hierarchies. In addition, this experiment has had substantial impact on the organisation as a whole. The total savings from the better design that the researchers helped identify are US$70 million per year.
RCTs and big data: research at Uber
Incentives – a little push to buy something or spend more money on a service – also featured in the second keynote. Professor John List of the University of Chicago is also the former Chief Economist at Uber. He now holds the same role at Lyft. While at Uber, List and his team had access to reams of excellent, nationwide data. He shared results about studies conducted around apologies (and incentives) and tipping.
In a first experiment, Uber tested the effect of apologies for bad rides on their client retention rates. Looking at 3 percent of the worst rides (late pickups, too much time spent in the car, etc.), Uber representatives emailed three different kinds of apologies to riders. One with a direct apology, one said they were “idiots” for causing the rough journey and another promised the riders that they would never experience such a terrible ride again. In addition to the apology emails, they tried another set of emails with the same types of apology combined with a voucher for money off their next journey, the only kind of compensation for disgruntled riders. Refunding fares is too expensive for the firm. What the researchers found is that an apology on its own won’t get the rider back in an Uber the way an incentive (voucher) does. But List said that a well-timed promotion – like after a bad ride – is better than just any promotion.
Another incentive, tipping, was something Uber drivers were after for a long time but Travis Kalanick was opposed to on principle. After the #DeleteUber campaign ratcheted up, Uber’s founder finally agreed to a trial. Over a two-month period in 2017, an experiment that generated data from 40 million trips began. List and his co-authors found that 59 percent of riders never tip; with only 1 percent always tipping. Women drivers receive 10 to 12 percent more than men drivers, regardless of the rider’s gender. Driver earnings, however, were flat and the trip quality didn’t improve with tipping, List explained.
Data as the new oil
Over the two-day workshop, we saw a range of questions and solutions arising from RCTs. At this moment, the methodology is in focus not only due to the recent Nobel prize, but also because we live in the age of data.
As List pointed out, “Data is now the most valuable resource in the world. It’s no longer oil. But much like oil needs a refinery, data also needs a refiner, and that’s where academics come in. They lend expertise, they understand how to think about causality, they understand how to generate new data to make causal claims and to figure out underlying mechanisms.”
*Professor of Economics and holds the Goltz Fellowship in Business and Society at INSEAD and the Academic Director of the INSEAD Randomized Control Trials (RCT) Lab
**first published in: knowledge.insead.edu
By: N. Peter Kramer
