Many companies today use experiments to collect the data that guides their strategies. This is how Facebook knows if a newsfeed change has increased user engagement, or how a bank knows if its new promotional strategy has increased deposits.
But a phenomenon called interference has always made it difficult to establish this clear cause and effect. Today, researchers at the University of Texas at Austin have devised a new method that sheds light on how interference affects the results of these randomized controlled trials, allowing social and behavioral scientists to account for interference in a wide range of applications.
Anytime an experimental treatment can change the behavior of units outside of the treatment group, there is interference and it blurs causal inference – the important cause-and-effect conclusions researchers draw.
“This phenomenon occurs every time I run a random experiment and my experimental units are connected to each other in some way – in social networks, in groups of friends, in households, in networks. shopping and city streets, ”said lead researcher David Puelz, UT McCombs School of Business clinical assistant professor of finance.
In social media research, people in treatment and control groups may be friends. In e-commerce research, people in each study group may live in the same household.
“Without taking interference into account, the estimates of causal effects will be very wrong,” Puelz said. “But if properly incorporated into a statistical analysis, interference can be a blessing. We have therefore designed a method that directly integrates network interference. “
The results are published online in advance in the Journal of the Royal Statistical Society. The research team included Guillaume Basse from Stanford University, Avi Feller from the University of California-Berkeley and Panos Toulis from the Booth School of Business at the University of Chicago.
The new method can directly test interference effects, such as fallout and peer effects in social networks. For example, Facebook can test whether a News Feed change affects not only the user, but that user’s friends as well. Policymakers can test whether wearing a mask alters viral spread within and between households, and human resource divisions in companies can determine whether wellness programs alter health outcomes among groups of employees. .
The research focused on spatial interference, drawing on data from a large-scale police experiment conducted by a separate team in Medellín, Colombia, in 2015. This team measured crime in all areas. streets of the city, identified the hotspots, then asked if additional police patrolling the streets of the hotspot was affecting crime on the adjacent streets. Did the intervention reduce crime or did it just move it to another street?
To find out, the researchers constructed a graph that encoded information about the interference structure of Medellín’s street segments and all possible combinations of police missions. Their method then uses this new graph construction with a new algorithm to divide it into two relevant subsets: overflow units which have not been processed but can be linked to a processed unit, and pure control units which have not been processed or linked.
On this smaller graph, they can then perform what is called a Fisher randomization test to determine if there is a ripple effect – whether crime on streets near guarded streets differs from those far from streets. monitored.
The research confirmed previous findings that additional street patrols reduce crime on adjacent streets. But beyond crime in Medellín, the IT approach created by Puelz and his team applies to any research disrupted by network interference.
“It can also be used on Facebook and Google in the thousands of experiments they run every day, or any other company doing A / B testing and internal experiments to improve operations,” Puelz said.
Journal of the Royal Statistical Society
Computer simulation / modeling
The title of the article
An approach of graph theory for randomization tests of causal effects under general interference
Publication date of the article
Warning: AAAS and EurekAlert! are not responsible for the accuracy of any press releases posted on EurekAlert! by contributing institutions or for the use of any information via the EurekAlert system.