There are and have been a lot startups working on similar things (AI to assist researchers), going back to IBM’s ill-fated Watson. Your demo makes it look very useful and is definitely the most impressive I’ve seen. I’m deeply suspicious of demos, however.
How can you test if your system is actually useful for researchers?
[One (albeit imperfect) way to gauge utility is to see if people are willing to pay money for it and keep paying money for it over time. However, I assume that is not the plan here. I guess another thing would be to track how much people use it over time or see if they fall away from using it. Another of course would be an RCT, although it’s not clear how it would be structured.]
This is a live product—not just a demo! You can use it at elicit.org.
More than 45K users have tried it and ~ 10K use it each month. Users say that Elicit saves them ~ 1-2 hours / week. They proactively share positive feedback on places like Twitter and with their colleagues or friends: Elicit’s growth is entirely by word of mouth.
I agree that having people pay for it is one of the greatest indicators of value. We’ll have to balance financial sustainability with the desire to make high-quality accessible.
At some point, we probably will do a more formal evaluation e.g. RCT type study.
There are and have been a lot startups working on similar things (AI to assist researchers), going back to IBM’s ill-fated Watson. Your demo makes it look very useful and is definitely the most impressive I’ve seen. I’m deeply suspicious of demos, however.
How can you test if your system is actually useful for researchers?
[One (albeit imperfect) way to gauge utility is to see if people are willing to pay money for it and keep paying money for it over time. However, I assume that is not the plan here. I guess another thing would be to track how much people use it over time or see if they fall away from using it. Another of course would be an RCT, although it’s not clear how it would be structured.]
This is a live product—not just a demo! You can use it at elicit.org.
More than 45K users have tried it and ~ 10K use it each month. Users say that Elicit saves them ~ 1-2 hours / week. They proactively share positive feedback on places like Twitter and with their colleagues or friends: Elicit’s growth is entirely by word of mouth.
I agree that having people pay for it is one of the greatest indicators of value. We’ll have to balance financial sustainability with the desire to make high-quality accessible.
At some point, we probably will do a more formal evaluation e.g. RCT type study.