That’s useful info, and sounds to me like a fair point. Thanks :)
But then this strikes me as tying back into the idea that “Perhaps [ALLFED] seemingly not having been funded by the EA Long-Term Future Fund, Open Phil, and various other funders is evidence that there’s some reason not to support them, which I just haven’t recognised?”
Here that question can take a more concrete form: If Open Phil chose to fund a group that’d work on alternative foods that ALLFED thinks will be less promising than the alternative foods ALLFED focuses on, but didn’t choose to fund ALLFED (at least so far), does that mean:
Open Phil are making a mistake?
ALLFED are wrong about which foods are most promising?
Perhaps because they’re wrong about the relative costs, or because there are other considerations which outweigh the cost consideration?
ALLFED are right about which foods are most promising, but there’s some other overriding reason why the other team was a better donation opportunity?
E.g., perhaps at the present margin, what’s most needed is more academic credibility and that team could get it better than ALLFED could?
There’s some alternative explanation such that Open Phil’s decisions are sound but also ALLFED is a good donation opportunity?
E.g., perhaps there’s some reason why Open Phil in particular shouldn’t fund ALLFED at this stage, even if it thought ALLFED was a good opportunity for other donors?
I don’t really know how likely each of those possible implications are (and thus I don’t have strong reason to believe 2 or 3 are the most likely implications). So this is just a confusing thing and a potential argument against donating to ALLFED, rather than a clearly decisive argument.
I’d be interested in your (or other people’s) thoughts on this—but would also understand if this is inappropriate to discuss publicly.
(Btw, I wouldn’t want readers to interpret this as a major critique or an expression of strong doubt. I’d expect to have at least some doubt or reservation with regards to basically any place I choose to donate to, work for, etc. - prioritisation is hard! - and I’m still planning to give ~4% of my income this year to ALLFED.)
Thanks, MichaelA! On neglectedness, it is true that $3 million is very large in this space. However, the Open Phil funded group decided to propose to work on alternative foods that they already had expertise in. This includes cellulosic sugar, duckweed, forest products including inner bark, mushrooms, and sprouts. With the exception of cellulosic sugar, these alternative foods are higher cost than the ones that ALLFED is prioritizing. Low cost is important for feeding nearly everyone and maintaining stability of civilization. Therefore, we don’t believe that the highest priority sun-blocking solutions (cellulosic sugar, methane single cell protein, hydrogen single cell protein, cold tolerant crops, greenhouses, seaweed, and leaf protein concentrate) are significantly less neglected now. Furthermore, the Open Phil funded project is generally not working on interventions for losing electricity/industry, so that remains highly neglected.
Hmm, I feel like you may be framing things quite differently to how I would, or something. My initial reaction to your comment is something like:
It seems usefully to conceptually separate data collection from data processing, where by the latter I mean using that data to arrive at probability estimates and decisions.
I think Bayesianism (in the sense of using Bayes’ theorem and a Bayesian interpretation of probability) and “math and technical patches” might tend to be part of the data processing, not the data collection. (Though they could also guide what data to look for. And this is just a rough conceptual divide.)
When Ozzie wrote about going with “an approach that in-expectation does a decent job at approximating the mathematical approach”, he was specifically referring to dealing with the optimizer’s curse. I’d consider this part of data processing.
Meanwhile, my intuitions (i.e., gut reactions) and what experts say are data. Attending to them is data collection, and then we have to decide how to integrate that with other things to arrive at probability estimates and decisions.
I don’t think we should see ourselves as deciding between either Bayesianism and “math and technical patches” or paying attention to my intuitions and domain experts. You can feed all sorts of evidence into Bayes theorem. I doubt any EA would argue we should form conclusions from “Bayesianism and math alone”, without using any data from the world (including even their intuitive sense of what numbers to plug in, or whether people they share their findings with seem skeptical). I’m not even sure what that’d look like.
And I think my intuitions or what domain experts says can very easily be made sense of as valid data within a Bayesian framework. Generally, my intuitions and experts are more likely to indicate X is true in worlds where X is true than where it’s not. This effect is stronger when the conditions for intuitive expertise are met, when experts’ incentives seem to be well aligned with seeking and sharing truth, etc. This effect is weaker when it seems that there are strong biases or misaligned incentives at play, or when it seems there might be.
(Perhaps this is talking past you? I’m not sure I understood your argument.)
For context, can you explain what kind of job you are looking for? How time-consuming does each application tend to be?
Thanks, Michael, that deserves an entry in https://docs.google.com/document/d/1OTCQlWE-GkY_V4V-OfJAr7Q-vJyIR8ZATpeMrLkmlAo/edit#
I am grateful for all the people in the community, who are always happy to help with minor things. Everytime I ask someone for training advice at a conference, or for a, explanation of any word on Dank EA Memes, or for career advice in the Forum, I always got really nice and detailed answers. I feel really excepted through that, especially considering, how rare these things are on the internet.
Being born into the 1%.
Just saw this comment, I’m also super late to the party responding to you!
It actually seems to me it might have been worth emphasising more, as I think a casual reader could think this post was a critique of formal/explicit/quantitative models in particular.
Totally agree! Honestly, I had several goals with this post, and I almost complete failed on two of them:
Arguing why utilitarianism can’t be the foundation of ethics.
Without talking much about AI, explaining why I don’t think people in the EA community are being reasonable when they suggest there’s a decent chance of an AGI being developed in the near future.
Instead, I think this post came off as primarily a criticism of certain kinds of models and a criticism of GiveWell’s approach to prioritization (which is unfortunate since I think the Optimizer’s Curse isn’t as big an issue for GiveWell & global health as it is for many other EA orgs/cause areas).--On the second piece of your comment, I think we mostly agree. Informal/cluster-style thinking is probably helpful, but it definitely doesn’t make the Optimizer’s Curse a non-issue.
Just found this post, coming in to comment a year late—Thanks Michael for the thoughtful post and Ozzie for the thoughtful comments!
I’m not saying that these are easy to solve, but rather, there is a mathematical strategy to generally fix them in ways that would make sense intuitively. There’s no better approach than to try to approximate the mathematical approach, or go with an approach that in-expectation does a decent job at approximating the mathematical approach.
I might agree with you about what’s (in some sense) mathematically possible (in principle). In practice, I don’t think people trying to approximate the ideal mathematical approach are going to have a ton of success (for reasons discussed in my post and quoted in Michael’s previous comment). I don’t think searching for “an approach that in-expectation does a decent job at approximating the mathematical approach” is pragmatic.
In most important scenarios, we’re uncertain what approaches work well in-expectation. Our uncertainty about what works well in-expectation is the kind of uncertainty that’s hard to hash out in probabilities. A strict Bayesian might say, “That’s not a problem—with even more math, the uncertainty can be handled....”While you can keep adding more math and technical patches to try and ground decision making in Bayesianism, pragmatism eventually pushes me in other directions. I think David Chapman explains this idea a hell of a lot better than I can in Rationalism’s Responses To Trouble.Getting more concrete:Trusting my gut or listening to domain experts might turn out to be approaches that work well in some situation. If one of these approaches works, I’m sure someone could argue in hindsight that an approach works because it approximates an idealized mathematical approach. But I’m skeptical of the merits of work done in the reverse (i.e., trying to discover non-math approaches by looking for things that will approximate idealized mathematical approaches).
Its true that this is probably most suited to a funding scheme aimed at early researchers due to the limitations mentioned by you. However, I might think that the grant success might go up if you use a model were you sort out all bad research first, because your 20 % is probably relate to the overall number of applications. Or maybe you could give people more tickets in the lottery if they have proven they can produce good research. However, this might introduce new biases.
In addition, it might still be a good approach for intermediate researchers because the overall time for the whole grant process gets reduced dramatically if you can cut out most of the peer review, which might lead to more calls for research proposals.
Concerning the Nature article and the modified lottery system: I read conflicting opinions on this. While the Nature article states that very good research can be identified easily, there are also others that state that researchers can only reliably identify bad research, but have a hard time to sort good research in any reproducible way.
Minor typo—The DOI for Evaluating use cases for human challenge trials in accelerating SARS-CoV-2 vaccine development. Clinical Infectious Diseases has a trailing e in the url which causes the link to fail
Meta: I suggest that when upvotes are used to implement a poll, the post author would also provide a comment to be downvoted by poll participants in order to balance the the author’s karma (so as to not create a norm that might be exploited by others for gaining karma).
Also, I suggest placing all the poll comments under one parent comment.
If I owe someone a small amount of money, I sometimes suggest tossing a coin: With a 50% chance we just both forget about the debt, otherwise, we double the debt. The expected money stays the same, the variance is insignificant as the amount is small and with a 50% chance, we do not need to make the hassle of exchanging coins. However, it seems that a lot of people are unwilling to do money betting. Even with amounts less than 10 Euro.
Another possible advantage of lotteries: You simply need to put in less work in decision-making. For example in the donor lottery: https://app.effectivealtruism.org/lotteries
I’m generally in favour of experimenting with different granting models and am glad to hear that funders are starting to experiment with random allocation. However, I’d be a little bit cautious about moving to a system based solely on random grant assignment. Depending on the actual grant success rate per round (currently often <20%), it seems likely that one would get awarded grants quite infrequently, which would interrupt the continuity of research. For instance, if somebody gets a random grant and makes an interesting discovery, it seems silly to then expect to wait several years for another random grant assignment to follow up on it. So I feel that random assignment is probably better used for assigning funding for early-career researchers or pilot projects.
With respect to quality control, the Nature news article linked above notes:
assessment panels spend most of their time sorting out the specific order in which to place mid-ranking ideas. Low- and high-quality applications are easy to rank, she says. “But most applications are in the midfield, which is very big.”
The current modified lottery systems just remove the low-ranking applications, but if it’s easy to pick high-ranking applications, surely they should be given funding priority?
I’m not optimistic. When will more reasonable voices with different biases enter social media? Almost the whole world is already on social media.
I love being in a community which helps me live up to my values:
When I first came across Peter Singer, I just kind of assumed I wouldn’t end up donating much because it seemed like even though clearly we ought to, no-one else did. It makes it so much easier being suddenly in a group where donating 10% is just the accepted norm.
The community also seems great at sharing knowledge to make this easier, and supporting each other. An example I particularly loved was Rob Wiblin helping me ringing for the election: he wrote up and easy how to guide, and then for people who still found it aversive (which includes me because I hate making phone calls), he invited us over to all do it together and egg each other on, and so that he could help troubleshoot.
Sorry I think I didn’t address the measurement issue very well, and assumed your notion of user interests meant simply optimizing for views, when maybe it isn’t. I still think through user research you can learn to develop good measures. For example: surveys, cohort tests (e.g. if you discount ratings over time within a viewing session, to down weight lower agency views, do you see changes such as users searching more instead of just letting autoplay), is there a relationship between how much a user feels netflix is improving their life (in a survey) and how much they are sucked in by autoplay? Learning these higher order behavioural indicators can help give users a better long-term experience, if that’s what the company optimizes for.
Many times I make mistakes. Sometimes those mistakes are bad enough that the wronged parties would have cause to demand expensive recompense, or to weaken their ties to me. Wonderfully, my friends and colleagues by and large value their relationship with me enough to overlook my flaws. From my Christian upbringing, I’d call that grace. On Thanksgiving this year, when many of us have needed little more grace from our friends than before, I’m grateful for mine.
Thanks for raising this. I appreciate specification is hard, but I think there’s a broader lens on ‘user interests’ with more acknowledgement for the behavioural side.What users want in one moment isn’t always the same as what they might endorse when in a less slippery behavioural setting or upon reflection. You might say this is a human not a technical problem. True, but we can design systems to that help us optimize for our long-term goals and that is a different task to optimizing for what we click on in a given moment. Sure it’s much harder to specify, but I think user research can be done. Thinking about the user more holistically could open up new innovations too. Imagine a person has watched several videos in a row about weight loss and rather than keeping them on the couch longer, it learns to respond with good nudges: prompts them to get up and go for a run, reminds them of their personal goals for the day (because it has such integrations), messages your running buddy, closes itself (and has nice configurable settings with good defaults), or advertises joining a local running group (right now the local running group would not afford the advert, but in a world where recommenders weight ad quality to somehow include long-term preferences of the user, that might be different). I understand the measurement frustration issue, the task is harder than just optimising for views and clicks though (not just technically, also to align to the company’s bottom line). However, I do think little steps towards better specification can help, and I’d love to read future user research on it at Netflix.