The web app is neat to experiment with the ideas and help us build intuitions.
That said, I think the key ideas (not the web app in particular), are the main insight here.
The current implementation is a solid first step, but I think we’re still a ways from having something that’s fun to use. My guess is that it will require some sophisticated UX / UI work to do a job that’s good enough for this to be useful in production. (If anyone reading this wants to try, let one of us know!)
I also think it’s important to figure out how to allow for negative values. This is annoying, but so it goes.
One thing I learned over the course of this, is that we probably don’t actually want big tables of utility estimates. Or, more specifically, is that we want functions that we can query as “how does X compare to Y”?, and they give us the correct amount. These can trivially convert to tables, but are subtly better. The reason for this is that they’ll handle correlations between items.
10 apples might be exactly 10 times as good as 1 Apple; 10 oranges 10x as 1 orange. We want a query of “how much better is 10 apples compared to 1 apple” to return exactly “10x”, and similar for oranges. If we tried putting them all into a common unit, like “pear equivalent”, then we wouldn’t get this property.
I’m not sure what the best format is to store this sort of data. Maybe some cluster analysis or something. There must be some clever mathematics for this somewhere, it it’s not clear to either of us.
I just wanted to give my take on some of this:
The web app is neat to experiment with the ideas and help us build intuitions.
That said, I think the key ideas (not the web app in particular), are the main insight here.
The current implementation is a solid first step, but I think we’re still a ways from having something that’s fun to use. My guess is that it will require some sophisticated UX / UI work to do a job that’s good enough for this to be useful in production. (If anyone reading this wants to try, let one of us know!)
I also think it’s important to figure out how to allow for negative values. This is annoying, but so it goes.
One thing I learned over the course of this, is that we probably don’t actually want big tables of utility estimates. Or, more specifically, is that we want functions that we can query as “how does X compare to Y”?, and they give us the correct amount. These can trivially convert to tables, but are subtly better. The reason for this is that they’ll handle correlations between items.
10 apples might be exactly 10 times as good as 1 Apple; 10 oranges 10x as 1 orange. We want a query of “how much better is 10 apples compared to 1 apple” to return exactly “10x”, and similar for oranges. If we tried putting them all into a common unit, like “pear equivalent”, then we wouldn’t get this property.
I’m not sure what the best format is to store this sort of data. Maybe some cluster analysis or something. There must be some clever mathematics for this somewhere, it it’s not clear to either of us.