Economic benefits of mediocre local human preferences modeling.
Epistemic status: Half-baked, probably dumb.
Note: writing is mediocre because it’s half-baked.
Some vague brainstorming of economic benefits from mediocre human preferences models.
Many AI Safety proposals include understanding human preferences as one of its subcomponents [1]. While this is not obviously good[2], human modeling seems at least plausibly relevant and good.
Short-term economic benefits often spur additional funding and research interest [citation not given]. So a possible question to ask if we can get large economic benefits from a system with the following properties (each assumption can later be relaxed):
1. Can run on a smartphone in my pocket
2. Can approximate simple preference elicitations at many times a second
3. Low fidelity, has both high false-positive and false-negative rates
4. Does better on preferences with lots of training data (“in-distribution”)
5. Initially works better on simple preferences (preference elicitations takes me 15 seconds to think about an answer, say), but has continuous economic benefits from better and better models.
An *okay* answer to this question is recommender systems (ads, entertainment). But I assume those are optimized to heck already so it’s hard for an MVP to win.
I think a plausibly better answer to this is market-creation/bidding. The canonical example is ridesharing like Uber/Lyft, which sells a heterogeneous good to both drivers and riders. Right now they have a centralized system that tries to estimate market-clearing prices, but imagine instead if riders and drivers bid on how much they’re willing to pay/take for a ride from X to Y with Z other riders?
Right now, this is absurd because human preference elicitations take up time/attention for humans. If a driver has to scroll through 100 possible rides in her vicinity, the experience will be strictly worse.
But if a bot could report your preferences for you? I think this could make markets a lot more efficient, and also gives a way to price in increasingly heterogeneous preferences. Some examples:
1. I care approximately zero about cleanliness or make of a car, but I’m fairly sensitive to tobacco or marijuana smell. If you had toggles for all of these things in the app, it’d be really annoying.
2. A lot of my friends don’t like/find it stressful to make small talk on a trip, but I’ve talked to drivers who chose this job primarily because they want to talk on the job. It’d be nice if both preferences are priced in.
3. Some riders like drivers who speak their native language, and vice versa.
A huge advantage of these markets is that “mistakes” are pricey but not incredibly so. Ie, I’d rather not overbid for a trip that isn’t worth it, but the consumer/driver surplus from pricing in heterogeneous preferences at all can easily make up for the occasional (or even frequent) mispricing.
There’s probably a continuous extension of this idea to matching markets with increasingly sparse data (eg, hiring, dating).
One question you can ask is why is it advantageous to have this run on a client machine at all, instead of aggregative human preference modeling that lots of large companies (including Uber) already do?
The honest high-level answer is that I guess this is a solution in search of a problem, which is rarely a good sign...
A potentialadvantage of running it on your smartphone (imagine a plug-in app that runs “Linch’s Preferences” with an API other people can connect to) is that it legally makes the “Marketplace” idea for Uber and companies like Uber more plausible? Like right now a lot of them claim to have a marketplace except they look a lot like command-and-control economies; if you have a personalized bot on your client machine bidding on prices, then I think the case would be easier to sell.
Economic benefits of mediocre local human preferences modeling.
Epistemic status: Half-baked, probably dumb.
Note: writing is mediocre because it’s half-baked.
Some vague brainstorming of economic benefits from mediocre human preferences models.
Many AI Safety proposals include understanding human preferences as one of its subcomponents [1]. While this is not obviously good[2], human modeling seems at least plausibly relevant and good.
Short-term economic benefits often spur additional funding and research interest [citation not given]. So a possible question to ask if we can get large economic benefits from a system with the following properties (each assumption can later be relaxed):
1. Can run on a smartphone in my pocket
2. Can approximate simple preference elicitations at many times a second
3. Low fidelity, has both high false-positive and false-negative rates
4. Does better on preferences with lots of training data (“in-distribution”)
5. Initially works better on simple preferences (preference elicitations takes me 15 seconds to think about an answer, say), but has continuous economic benefits from better and better models.
An *okay* answer to this question is recommender systems (ads, entertainment). But I assume those are optimized to heck already so it’s hard for an MVP to win.
I think a plausibly better answer to this is market-creation/bidding. The canonical example is ridesharing like Uber/Lyft, which sells a heterogeneous good to both drivers and riders. Right now they have a centralized system that tries to estimate market-clearing prices, but imagine instead if riders and drivers bid on how much they’re willing to pay/take for a ride from X to Y with Z other riders?
Right now, this is absurd because human preference elicitations take up time/attention for humans. If a driver has to scroll through 100 possible rides in her vicinity, the experience will be strictly worse.
But if a bot could report your preferences for you? I think this could make markets a lot more efficient, and also gives a way to price in increasingly heterogeneous preferences. Some examples:
1. I care approximately zero about cleanliness or make of a car, but I’m fairly sensitive to tobacco or marijuana smell. If you had toggles for all of these things in the app, it’d be really annoying.
2. A lot of my friends don’t like/find it stressful to make small talk on a trip, but I’ve talked to drivers who chose this job primarily because they want to talk on the job. It’d be nice if both preferences are priced in.
3. Some riders like drivers who speak their native language, and vice versa.
A huge advantage of these markets is that “mistakes” are pricey but not incredibly so. Ie, I’d rather not overbid for a trip that isn’t worth it, but the consumer/driver surplus from pricing in heterogeneous preferences at all can easily make up for the occasional (or even frequent) mispricing.
There’s probably a continuous extension of this idea to matching markets with increasingly sparse data (eg, hiring, dating).
One question you can ask is why is it advantageous to have this run on a client machine at all, instead of aggregative human preference modeling that lots of large companies (including Uber) already do?
The honest high-level answer is that I guess this is a solution in search of a problem, which is rarely a good sign...
A potential advantage of running it on your smartphone (imagine a plug-in app that runs “Linch’s Preferences” with an API other people can connect to) is that it legally makes the “Marketplace” idea for Uber and companies like Uber more plausible? Like right now a lot of them claim to have a marketplace except they look a lot like command-and-control economies; if you have a personalized bot on your client machine bidding on prices, then I think the case would be easier to sell.
[1] https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/
[2] https://intelligence.org/2019/02/22/thoughts-on-human-models/