Data Science

Towards Data Science: Preventing Outcome Starvation

Predictive models are rarely static — operationalized models typically have an update cadence. At Mobilewalla, for instance, our models are updated every 30–180 days. At the end of each update period, the model is revised based on assessing the fidelity of its output since the last update. This is an important component of standard model maintenance practice, and is known as the feedback loop.

A degenerate feedback loop (DFL) occurs when this prior output unfairly impacts future outcomes. My favorite explanation of DFL is in Chip Hyuen’s lecture notes from the CS329S (ML Systems Design) class she teaches at Stanford (found here). To illustrate DFLs let me quote a passage directly from there: imagine you build a system to recommend to users songs that they might like. The songs that are ranked high by the system are shown first to users. Because they are shown first, users click on them more, which makes the system more confident that these recommendations are good. In the beginning, the rankings of two songs A and B might be only marginally different, but because A was originally ranked a bit higher, A got clicked on more, which made the system rank A even higher. After a while, A’s ranking became much higher than B. Degenerate feedback loops are one reason why popular movies, books, or songs keep getting more popular, which makes it hard for new items to break into popular lists.

Read the Full Article on Towards Data Science →


Towards Data Science Inc. using Medium, provides a platform for thousands of people to exchange ideas and to expand our understanding of data science. Our audience is mixed, consisting of readers entirely new to the subject and expert professionals who want to share their inventions and discoveries.

 

Picture of Anindya Datta, Ph.D.

Anindya Datta, Ph.D.

Anindya Datta, the CEO and Chairman of Mobilewalla, is widely regarded as a front-running technologist, leader and innovator, with core contributions to the state of the art in large-scale data management and Internet technologies. Mobilewalla has pioneered audience measurement in mobile apps by applying ground-breaking data science techniques on the industry’s largest volumetric database of mobile app data. Prior to Mobilewalla, Anindya founded and ran Chutney Technologies, where he was backed by Kleiner Perkins and evolved into one of the earliest entrants in the application virtualization area. The company was acquired by Cisco Systems in 2005. Anindya has also been on the faculties of major research universities and institutes in the United States and abroad, including the Georgia Institute of Technology, the University of Arizona, the National University of Singapore and Bell Laboratories. Anindya obtained his undergraduate degree from the Indian Institute of Technology (IIT) Kharagpur, and his MS and Ph.D. degrees from the University of Maryland, College Park.