Study Examined Tradeoffs in Leveraging External Data Capabilities
Findings Have Implications for Firm Managers, Policymakers
Large online firms collect a lot of information to build data-enabled capabilities, enhance offerings, and raise revenue. But measuring the impact of such capabilities is challenging. In a new study, researchers collaborated with a Chinese technology company’s search team to analyze a large-scale field experiment. They sought to measure the impact of removing access to the search engine’s autocomplete application programming interface (API) on the partner company’s search product. Their findings are informative for both firm managers and policymakers.
The study was conducted by researchers at Carnegie Mellon University, Shanghai Jiao Tong University, and the University of Notre Dame. It is published in Management Science.
“Measuring the impact of these external data capabilities on product performance can provide important information for both firms and guide relevant policy,” says Ananya Sen, associate professor of information technology and management at Carnegie Mellon’s Heinz College, who coauthored the study.
One example of how online firms enhance their data-enabled capabilities occurs when leading search engines devote resources to develop query autocomplete, which generates relevant search suggestions in response to search queries before users reach the search engine results page. Increasingly, smaller players are using APIs, which allow different applications to communicate with each other to gain access to the market leader’s capabilities and improve firms’ offerings.
In this study, researchers partnered with a large technology company in China that develops a mobile app for users to gather, use, and share information. The app offers various in-app products, such as a news feed, search engine, and video and eBook streaming, each with its own dedicated product interface.
Researchers randomized access to the search engine’s autocomplete API for more than 2.3 million users over 108 days. They measured the causal effects of removing API access on two performance metrics of the company’s search product: the clickthrough rate on search suggestions and clickthrough rate on the search engine results page. Among the study’s findings:
- On average, removing access to the market leader’s autocomplete API led to a 4.6% decline in clickthrough rate on search suggestions.
- A 10% increase (decrease) in clickthrough rate on search suggestions led to a 1.85% increase (decrease) in clickthrough rate on top-slot search results.
- However, the negative effect of removing the market leader’s autocomplete API was more pronounced for heavy users than for light users, suggesting that access to the market leader’s API was more effective in helping user retention by streamlining heavy users’ experience than in ameliorating the cold-start problem for light users.
The study has practical implications for managers and policymakers, say the authors.
“Our results suggest that leveraging external data capabilities can provide an economically meaningful return for a company, especially in the early stages of product development,” explains Xiaoxia Lei, a Ph.D. student in information, technology, and innovation at Shanghai Jiao Tong University, who led the study.
“In terms of policy issues currently being debated, we believe that sharing data capabilities across search engines could be part of a multi-pronged approach adopted by regulators to tackle market concentration by using different levers,” adds Yixing Chen, assistant professor of marketing at the University of Notre Dame’s Mendoza College of Business, who coauthored the study.
Among the study’s limitations, the authors note that they examined only one setting, so their estimates require assumptions to extrapolate to other contexts.
###
Summarized from an article in Management Science, Trade-offs in Leveraging External Data Capabilities: Evidence from a Field Experiment in an Online Search Market, by Lei, X (Shanghai Jiao Tong University), Chen, Y (University of Notre Dame), and Sen, A (Carnegie Mellon University). Copyright 2024. All rights reserved..
About Heinz College of Information Systems and Public Policy
The Heinz College of Information Systems and Public Policy is home to two internationally recognized graduate-level institutions at Carnegie Mellon University: the School of Information Systems and Management and the School of Public Policy and Management. This unique colocation combined with its expertise in analytics set Heinz College apart in the areas of cybersecurity, health care, the future of work, smart cities, and arts & entertainment. In 2016, INFORMS named Heinz College the #1 academic program for Analytics Education. For more information, please visit www.heinz.cmu.edu.