Sorry I think I didn’t address the measurement issue very well, and assumed your notion of user interests meant simply optimizing for views, when maybe it isn’t. I still think through user research you can learn to develop good measures. For example: surveys, cohort tests (e.g. if you discount ratings over time within a viewing session, to down weight lower agency views, do you see changes such as users searching more instead of just letting autoplay), is there a relationship between how much a user feels netflix is improving their life (in a survey) and how much they are sucked in by autoplay? Learning these higher order behavioural indicators can help give users a better long-term experience, if that’s what the company optimizes for.
Stated preference (uplifting documentaries) and revealed preference (reality TV crime shows) are different
Asking people for their preference is quite difficult—only a small fraction of Netflix users give star ratings or thumb ratings. In general, users like using software to achieve their immediate goals. It’s tough to get them to invest time and skill into making it better in the future. For most people, each app is a tiny tiny slice of their day and they don’t want to do work to optimize anything. Customization and user controls often fail because no one uses them.
If serving recommendations according to stated preferences causes people to unsubscribe more, how should we interpret that? That their true preference is to not be subscribed to Netflix? It’s unclear.
In any case, Netflix is financially incentivized to optimize for subscriptions, not viewing. So if people pay for what they want, then Netflix ought to be aligned with what they want. Netflix is only misaligned with what people want if people’s own spending is not aligned with what they want (theoretically).
Sorry I think I didn’t address the measurement issue very well, and assumed your notion of user interests meant simply optimizing for views, when maybe it isn’t. I still think through user research you can learn to develop good measures. For example: surveys, cohort tests (e.g. if you discount ratings over time within a viewing session, to down weight lower agency views, do you see changes such as users searching more instead of just letting autoplay), is there a relationship between how much a user feels netflix is improving their life (in a survey) and how much they are sucked in by autoplay? Learning these higher order behavioural indicators can help give users a better long-term experience, if that’s what the company optimizes for.
Absolutely. A few comments:
Stated preference (uplifting documentaries) and revealed preference (reality TV crime shows) are different
Asking people for their preference is quite difficult—only a small fraction of Netflix users give star ratings or thumb ratings. In general, users like using software to achieve their immediate goals. It’s tough to get them to invest time and skill into making it better in the future. For most people, each app is a tiny tiny slice of their day and they don’t want to do work to optimize anything. Customization and user controls often fail because no one uses them.
If serving recommendations according to stated preferences causes people to unsubscribe more, how should we interpret that? That their true preference is to not be subscribed to Netflix? It’s unclear.
In any case, Netflix is financially incentivized to optimize for subscriptions, not viewing. So if people pay for what they want, then Netflix ought to be aligned with what they want. Netflix is only misaligned with what people want if people’s own spending is not aligned with what they want (theoretically).