It’s unfortunately very hard to quantify the impact of recommender systems. But here’s one experiment that may update your prior on the effectiveness of targeted video recommendations.
In 2013, Facebook did a large-scale experiment where they tweaked their newsfeeds. For some of their users, they removed 10% of posts with negative contents. For others, they removed 10% of posts with positive contents. And there was also a control group. After only one week, they observed a change in users’ behaviors: the first group posted more positive contents, and the second posted more negative contents. Here’s our discussion of the paper.
I cannot stress enough the fact that Facebook’s intervention was minimal. A 2020 aligned recommender system could completely upset the entire newsfeed. Also, it would do so for longer than just a week — and other studies suggest that it takes months for users to learn even basic adaptation to tweaks of the algorithm.
In terms of healthcare in developing countries, it still seems to me that most philanthropists or small donators neglect the effectiveness of healthcare interventions in developing countries. Yet videos (not ads!) that do discuss this are not necessarily sufficiently promoted by the YouTube algorithm. As an example, this excellent video by Looking Glass Universe only has a fraction of the views of her other videos. Similarly, this video by Bill Gates has a lot less views than his videos on, say, climate change.
Note that this other post tried to quantify aligning recommender systems in terms of the common scale Scale/Tractability/Neglectedness. But the authors acknowledge themselves that they have low confidence in their estimates. But I’d argue that this goes on showing just how neglectedness this cause area is. Scientists both within and outside these companies have a hard time estimating the impact and tractability of (making progress in) aligning recommender systems (most don’t even know what “alignment” is).
It’s unfortunately very hard to quantify the impact of recommender systems. But here’s one experiment that may update your prior on the effectiveness of targeted video recommendations.
In 2013, Facebook did a large-scale experiment where they tweaked their newsfeeds. For some of their users, they removed 10% of posts with negative contents. For others, they removed 10% of posts with positive contents. And there was also a control group. After only one week, they observed a change in users’ behaviors: the first group posted more positive contents, and the second posted more negative contents. Here’s our discussion of the paper.
I cannot stress enough the fact that Facebook’s intervention was minimal. A 2020 aligned recommender system could completely upset the entire newsfeed. Also, it would do so for longer than just a week — and other studies suggest that it takes months for users to learn even basic adaptation to tweaks of the algorithm.
In terms of healthcare in developing countries, it still seems to me that most philanthropists or small donators neglect the effectiveness of healthcare interventions in developing countries. Yet videos (not ads!) that do discuss this are not necessarily sufficiently promoted by the YouTube algorithm. As an example, this excellent video by Looking Glass Universe only has a fraction of the views of her other videos. Similarly, this video by Bill Gates has a lot less views than his videos on, say, climate change.
Note that this other post tried to quantify aligning recommender systems in terms of the common scale Scale/Tractability/Neglectedness. But the authors acknowledge themselves that they have low confidence in their estimates. But I’d argue that this goes on showing just how neglectedness this cause area is. Scientists both within and outside these companies have a hard time estimating the impact and tractability of (making progress in) aligning recommender systems (most don’t even know what “alignment” is).