I predict the value of each project in terms of “microbostroms” (or, Quality Adjusted Research Papers), more on which here, or expected microbostroms per unit of resources, and then carry out the most promising ones. See here for a rubric for that unit.
This has recently led to some fairly well received posts, such as:
so I’m probably going to continue doing this. I initially started with a simple google sheet, which you can see here. But I recently moved to a foretold community, which looks something like this:
I’m actually looking to see if this has a chance of being useful for other people, so if you or other researchers want to send me a list of projects you’re considering, I’m happy to get you set up on foretold and predict an initial estimate of their value.
Thanks for this answer Nuno! I’ve read the first 1/3-1/2 of the post you linked so far, and I think it’s a cool framework. I think estimating value as “microbostroms”/”milibostroms” or QARPs, and framing things as an order of magnitude away from each other, are a good idea.
I’m curious how much time you expect to take to predict an initial estimate of the value of 1 research project? I don’t have a list of topics yet, but if I do have one, I’m wondering how much time it would take you to predict their value.
Also, after you’re done predicting (or crowdsourcing predictions) of the value of your projects, how do you then estimate how long each research question/project would take? Do you also do forecasts for that? And do you then just divide the milibostroms by the estimated number of hours to find what’s most cost-effective to pursue?
The posts linked under “fairly well received posts” are just as proof of capabilities (i.e., my forecasting system isn’t suggesting terrible projects). They are also fairly long, so I wouldn’t suggest reading all of them.
Right now, the time I take to predict the value of a project ranges from almost instantaneous to something like 5 mins.
how do you then estimate how long each research question/project would take? Do you also do forecasts for that?
Yes, I also have an estimate for hours, but that’s a bit more tricky, and I’m not that great at it.
And do you then just divide the milibostroms by the estimated number of hours to find what’s most cost-effective to pursue?
That’s cool to hear though that it takes you 0-5 mins to predict the value of a project. I may want to book a call with you too to dig deeper about your forecasting system for projects. Could you DM me on the Forum your Calendly link (if you have one)? :)
I predict the value of each project in terms of “microbostroms” (or, Quality Adjusted Research Papers), more on which here, or expected microbostroms per unit of resources, and then carry out the most promising ones. See here for a rubric for that unit.
This has recently led to some fairly well received posts, such as:
https://www.lesswrong.com/posts/6bSjRezJDxR2omHKE/real-life-examples-of-prediction-systems-interfering-with
https://forum.effectivealtruism.org/posts/SCqRu6shoa8ySvRAa/big-list-of-cause-candidates
https://forum.effectivealtruism.org/posts/udGBF8YWshCKwRKTp/an-experiment-to-evaluate-the-value-of-one-researcher-s-work
https://forum.effectivealtruism.org/posts/qb56nicbnj9asSemx/predicting-the-value-of-small-altruistic-projects-a-proof-of
so I’m probably going to continue doing this. I initially started with a simple google sheet, which you can see here. But I recently moved to a foretold community, which looks something like this:
I’m actually looking to see if this has a chance of being useful for other people, so if you or other researchers want to send me a list of projects you’re considering, I’m happy to get you set up on foretold and predict an initial estimate of their value.
Thanks for this answer Nuno! I’ve read the first 1/3-1/2 of the post you linked so far, and I think it’s a cool framework. I think estimating value as “microbostroms”/”milibostroms” or QARPs, and framing things as an order of magnitude away from each other, are a good idea.
I’m curious how much time you expect to take to predict an initial estimate of the value of 1 research project? I don’t have a list of topics yet, but if I do have one, I’m wondering how much time it would take you to predict their value.
Also, after you’re done predicting (or crowdsourcing predictions) of the value of your projects, how do you then estimate how long each research question/project would take? Do you also do forecasts for that? And do you then just divide the milibostroms by the estimated number of hours to find what’s most cost-effective to pursue?
The posts linked under “fairly well received posts” are just as proof of capabilities (i.e., my forecasting system isn’t suggesting terrible projects). They are also fairly long, so I wouldn’t suggest reading all of them.
Right now, the time I take to predict the value of a project ranges from almost instantaneous to something like 5 mins.
Yes, I also have an estimate for hours, but that’s a bit more tricky, and I’m not that great at it.
Yes, see column H of the spreadsheet.
Ah I meant I’ve only read 1/3-1/2 of the “An experiment to evaluate the value of one researcher’s work”. I’ll try to finish reading it within the next few days.
That’s cool to hear though that it takes you 0-5 mins to predict the value of a project. I may want to book a call with you too to dig deeper about your forecasting system for projects. Could you DM me on the Forum your Calendly link (if you have one)? :)