I have a paper I’ve been kicking around in the back of my head for a couple years to formalize essentially this idea via economic/financial modern portfolio theory and economic collective action problem theory—but *someone* has me working on more important problems and papers instead… Gregory. ;)
It’s an interesting point. Do you know who you’ve given it to, and would you consider sending them a separate copy of the survey once it’s been released, to gauge effectiveness / relative effectiveness?
Having done a little bit of graduate work on survey design, I’ll put on my survey-design hat and offer a bunch of suggestions. If you’d like to discuss further, please feel free to reach out directly on—twitter / @gmail.com / calendly—my username is my full name.
Before getting to the question, a few points on sample design and research ethics:
1) You might want to provide distinct links to the survey for each subset of answerers, i.e. those who you gave the book and are asking directly, or those who each other person has done so for. You may also want to link to a different survey to ask the secondary people who borrowed the book. This allows you to later consider if there are differences in effectiveness.
2) You should clarify that the data will be analyzed and shared, but will first have any personally identifying information removed. (i.e. don’t share the raw results with email addresses with ANYONE.)
3) Best-best practice would be to pre-register your analysis plan.
On to the questions! I have a bunch of language and structure nitpicks I’d suggest changing:
re: 1 - “I have’t read/finished it yet, but I plan to,” I think that’s ambiguous—people might have stopped reading, but still be impacted. I’d change to the following options:
I read it
I haven’t begun reading it yet—Please skip to the final 2 questions
I just began reading it and plan to finish—Please skip to the final 2 questions
I have read a substantial part of the book, but didn’t finish it.
I do not plan on reading it.
(Also, don’t ask for an email address here, instead say “If you have made any changes to your life based on this, please continue. ” People are more likely to feel willing to provide extra information as they move later in the survey
Relatedly, re: “2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? If so, please list each charity, with US dollar amount, on its own line.”
I’m concerned that people will find the wording or the request pushy / invasive and refuse to answer or stop the survey completely. I’d suggest splitting this into a gentler 3-part version: (Again, people who start complying are more likely to continue filling out more-invasive questions.)
2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? Yes/No
2a. If so, we would be interested in evaluating how effective giving away the book was, and would like to know the total amount you have donated. (# Box)
2b. If you are willing to provide further details please list the charities and US dollar amounts, on separate lines. (Textbox)
For the remaining questions, I’d also recommend refactoring and making the flow better;
3. Have you considered made any other life changes since reading the book? Yes / No
(If not, please skip to question 7.)
3a. If so, did you change or consider changing your: Dietary choices (See Q4) / Career Plans (See Q5) / Other (See Q6)? (Checkboxes)
4. If you changed your Dietary choices, have you become Vegan (4a) / Vegetarian (4a) / Reduced meat consumption (4b) / Other (4c) / Considered this but have not (yet) changed anything (4c)?
4a. How many months ago?
4b. How much less eggs or meat?
4c. What else did you change / consider changing?
5. Have you made or considered making career changes?(Current Q6 options)
5a. Details (text)
6. Other changes (text)
7. Put question 10 here, before asking about the follow up.
Also: Give them the option to provide email addresses instead of asking their friend themselves, if they think it would be less intrusive. Note that the email will be short, and the email addresses will not be kept beyond sending a single email asking them to take the survey. (Note: I ALSO think people are a bit more likely to send the request themselves if given this option as an alternative.)
8. Then, Q9 / If you would be willing to take another follow-up survey, please enter… (Make language include either the “haven’t read / finished yet” AND asking for a follow-up—you will know which is which based on the above.)
The arguments about pre-driven cars seem to draw a sharp line between understanding and doing. The obvious counter seems to be asking if your brain is “pre-programmed” or “self-directed”. (If this seems confused, I strongly recommend the book “Good and Real” as a way to better think about this question.)
I’m also confused about why the meaning bias is a counter-argument to specific scenarios and estimates, but that’s mostly directed toward my assumption that this claim is related to Pinker’s argument. Otherwise I don’t understand why “fertile ground for motivated reasoning” isn’t a reason to simply find outside view estimates—the AI skeptics mostly say we don’t need to worry about SAI for around 40 years, which seems consistent with investing a ton more in risk mitigation now.
Wei—a few points in response:
1) There isn’t really a lack of funds for new effective charities—there are a variety of grant programs, both those run by CEA and others, that will help such efforts get started.
2) The coordination overhead between major donors, researchers, and non-EA orgs is already prohibitively costly. (Coordination has some costs that expand super-exponentially, and there’s already a lot of groups involved.)
3) I’m unsure that there are major costs that would be avoided by coordinating, or opportunities that would be found. Small donors can give to the major charities via Givewell fairly easily, and can choose any other cause on their own.
4) Having a “give here” suggestion/priority list seems to create potentially damaging correlation between givers’ priorities—we’d probably prefer to allow donors to make their own allocation. (Though Givewell does publish recommendations for charities they don’t support that they nonetheless suggest are worth funding, so I’m not sure anyone else sees this as an issue.)
I don’t think this is quite right. The people working at OpenAI are paid well, but at the same time they are taking huge cuts in salary compared to where they could be working otherwise. (Goodfellow and Sutskever could be making millions anywhere.) And given the distribution of salary, it’s very likely that the majority of both OpenAI and Deepmind researchers are making under $200k—not a crazy amount for Deep Learning talent nowadays.
This is spot-on, and as a matter of decision theory, the question is never “which outcome matters most,” but is rather “what action has the highest impact.” This incorporates the economic issues with marginal investment, as well as the issues with constraints discussed above. I’d recommend Tiago Forte’s series explaining the “Theory of Constraints” (ToC) for a better way to formalize the intuitive model presented in the post; https://praxis.fortelabs.co/theory-of-constraints-101-table-of-contents-8bbb6627915b/
As applied to EA, this notes that we should build clear system models for interventions in order to identify how to help. The ToC model notes that effort expended to help at any point of the system other than the limiting factor is wasted—double the funding but don’t fix the logistic constraints on spending it and you’ve helped not-at-all. (In fact, you might have made the problem worse by increasing the pressure on the logistics management!)
1) I agree that there is some confusion on my part, and on the part of most others I have spoken to, about how terminal values and morality do or do not get updated.
3) I will point to a maybe forthcoming paper / idea of Eric Drexler at FHI that makes this point, which he called “pareto-topia”. Despite the wonderful virtues of the idea, I’m unclear if there is a stable game-theoretic mechanism that prevents a race to the bottom outcome when fundamentally different values are being traded off. Specifically in this case, it’s possible that different values lead to an inability to truthfully/reliably cooperate—a paved road to pareto-topia seems not to exist, and there might be no path at all.
It’s very likely that more organizations help, up to a point. The limit, as I think I failed to make clear, but is implicit, is that coordination pressure/failure always exist. They are either between organizations, or within them. Large organizations have scaling efficiencies because they can coordinate at lower cost than markets. (This is what a couple economists won nobels for recently, for work now referred to as the theory of the firm.) Those efficiencies are greatly reduced when multiple organizations are involved, but I think a few of my suggestions—specialization, referral of promising work, and coordinating bodies—might help somewhat with that.
I would (a bit weakly) agree that as of three years ago, growth of new EA organizations was probably a bit below optimal. I’m not following all of the threads of organizations closely, but from what I have seen, I would (even more weakly) guess that the rate of new organizations forming now is probably at or above the point of effective returns, at least for existential risk organizations. That’s why I think coordination is particularly useful now. Still, attempts to find anything like an optimal rate seem like a waste of time. We simply don’t understand the questions or the domain well enough to conclusively answer the question, except perhaps approximately and in retrospect. (Even if we did have such understanding or insight, I don’t think we would be able to convince anyone to follow the guidelines, given that the optimum rate is almost certainly not a Nash equilibrium.)
Agreed, but a fairly large questions were so ill-specified that I was basically trying to decide what orders of magnitude was relevant for games I not only knew nothing about, but couldn’t find clarity even after knowing the answer, over and over.
A sample example I’m making up, but is similar to some of the questions I saw: “England out-scored France in 1982 by how much?” What sport is being referred to? What series, or single game, or Olympics, or season?
Thanks for replying.
I’d agree with your points regarding limited scope for the first and second points, but I don’t understand how anyone can make prioritization decisions when we have no discounting—it’s nearly always better to conserve resources. If we have discounting for costs but not benefits, however, I worry the framework is incoherent. This is a much more general confusion I have, and the fact that you didn’t address or resolve it is unsurprising.
Re: S-Risks, I’m wondering whether we need to be concerned about value misalignment leading to arbitrarily large negative utility, given some perspectives. I’m concerned that human values are incoherent, and any given maximization is likely to cause arbitrarily large “suffering” for some values—and if there are multiple groups with different values, this might mean any maximization imposes maximal suffering on the large majority of people’s values.
For example, if 1⁄3 of humanity feels that human liberty is a crucial value, without which human pleasure is worse than meaningless, another 1⁄3 views earning reward as critical, and the last 1⁄3 views bliss/pure hedonium as optimal, we would view tiling the universe with human brains maxed out for any one of these as a hugely negative outcome for 2⁄3 of humanity, much worse than extinction.
Cool, but a fair number of the questions are vague or lack needed context.
Still, I’d agree for people that aren’t used to self-calibration with the above assessment that, if not the most valuable, it’s really up there on the list of “most valuable 4 hours of rationality training you can do.”
Great work. A few notes in descending order or importance which I’d love to see addressed at least in brief:
1) This seems not to engage with the questions about short-term versus long-term prioritization and discount rates. I’d think that the implicit assumptions should be made clearer.
2) It doesn’t seem obvious to me that, given the universalist assumptions about the value of animal or other non-human species, the long term future is affected nearly as much by the presence or absence of humans. Depending on uncertainties about the Fermi hypothesis and the viability of non-human animals developing sentience over long time frames, this might greatly matter.
3) Reducing the probability of technological existential risks may require increasing the probability of human stagnation.
4) S-risks are plausibly more likely if moral development is outstripped by growth in technological power over relatively short time frames, and existential catastrophe has a comparatively limited downside.
You might be interested in my recent paper, “Questioning EStimates of Natural Pandemic Risk”—https://www.liebertpub.com/doi/pdf/10.1089/hs.2018.0039
As I said in response to a different comment, I don’t object to making the claim that we should treat them as morally equal due to ignorance, but that’s very different from your claim that we can assume the intensities are equal.
I’m also not sure what to do with the claim that there might be different morally relevant dimensions that we cannot collapse, because if that is true, we are in a situation where 1-point of “artistic sufferring” is incommensurable with 1-billion points of “physical pain.” If so, we’re punting—because we do in fact make decisions between options on some basis, despite the supposedly “incommensurable” moral issues.
I agree that it is morally justifiable to treat them as equal absent convincing evidence, but I don’t think it’s correct to claim we should assume they are equal.
Nice find, definitely a related point!
I don’t understand how point 1 is possible—sure, given the model the maximum could be higher than all animals, or even than all humans, but this contradicts my experience. My experience is that children suffer more intensely than adults, and given the emotional complexity of many higher mammals, they are in those terms more sophisticated beings than babies, if not toddlers.
Regarding point 2, yes, that could reduce average suffering, which matters for average utilitarians, but does not mitigate experienced suffering for any other beings, which I think most other strains of utilitarianism would care about more.
I think the adult suffering from anticipation (and from uncertainty) is limited, via both contextualization and hedonic adaptation. I’m unsure how the balance of intense pleasure / pain works for children. They may experience pleasure more intensely, but I don’t see it as much. And it’s plausible that animals also experience pleasure more intensely, but I’m agnostic about that claim.