Takeaways from some reading about economic effects of human-level AI
I spent some time reading things that you might categorise as “EA articles on the impact of human-level AI on economic growth”. Here are some takeaways from reading these (apologies for not always providing a lot of context / for not defining terms; hopefully clicking the links will provide decent context).
Tom Davidson thinks that “explosive growth would require AI that substantially accelerates the automation of a very wide range of tasks in production, R&D, and the implementation of new technologies.”
Quick BOTEC of person-hours spent on EA Job Applications per annum.
I created a Guesstimate model to estimate a total of ~14,000 to 100,000 person-hours or ~7 to 51 FTE are spent per year (90% CI). This comes to an estimated USD $ 320,000 to $3,200,000 unpaid labour time.
All assumptions for my calculations are in the Guesstimate
The distribution of effort spent by candidates is heavy-tailed; a small percentage of candidates may spend 3 to 10x more time than the median candidate.
I am not very good at interpreting the guesstimate, so if someone can state this better / more accurately than would be helpful
Keen to get feedback on whether I’ve over/underestimated any variables.
I’d expect this to grow at a rate of ~5-10% per year at least.
Sources: My own experience as a recruiter, applying to EA jobs and interviewing staff at some EA orgs.
Edited the unpaid labour time to reflect Linch’s suggestions.
I’ll adjust the estimate a bit higher.
In the Guesstimate I do discount the hours to say that 75% of the total hours are unpaid (trial week hours cone to 5% of the total hours).
I did not review the model, but only 75% of hours being unpaid seems much too low based on my experience having gone through the job hiring process (including later stages) with 10-15 EA orgs.
Okay, so I used a different method to estimate the total manhours and my new estimate is something like 60%. I basically assumed that 50% of Round 2 −4 in the application process is paid, and 100% of the work trial.
I expect that established / longtermist orgs are disproportionately likely to pay for work tests, compared to new or animal / GH&D orgs.
I think Josh was claiming that 75% was “too low”, as in the total % of unpaid hours being more like 90% or something.
When I applied to a bunch of jobs, I was paid for ~30 of the ~80 hours I spent (not counting a long CEA work trial — if you include that, it’s more like 80 out of 130 hours). If you average Josh and I, maybe you get back to an average of 75%?
*****
This isn’t part of your calculation, but I wonder what fraction of unique applicants to EA jobs have any connection to the EA community beyond applying for one job?
In my experience trying to hire for one role with ~200 applicants, ~1/3 of them neither had any connection to EA in their resumes nor provided further information in their applications about what drew them to EA. This doesn’t mean there wasn’t some connection, but a lot of people just seemed to be looking for any job they could find. (The role was more generic than some and required no prior EA experience, so maybe drew a higher fraction of outside applicants.)
Someone having no other connection to the EA community doesn’t mean we should ignore the value of their time, and the people who apply to the most jobs are likely to have the strongest connections, so this factor may not be too important, but it could bear consideration for a more in-depth analysis.
I was wondering if the repugnant conclusion could be responded by an argument of the following form:
Considering planet earth and a given happiness distribution of its citizens with total happiness h, there is simply not enough space or resources or whatsoever to let an arbitrary large number of people n live with an average amount of happiness epsilon, such that n * epsilon > h. At even larger scales, the observable universe is finite and thus for the same reason as above n does not need to exist.
What do you think of such an argument?
I am not sure, whether the nature of the repugnant conclusion is really affected by such an argument. Can you help me to understand?
The repugnant conclusion is presented as an objection to certain views in population axiology. The claim is that a possible world containing sufficiently many morally relevant beings just above neutrality is intrinsically better than a possible world with arbitrarily many beings arbitrarily happy. The claim is not that these worlds could become actual, so empirical considerations of the sort you describe aren’t relevant for assessing the force of the objection.
Put differently, theories like total utilitarianism imply that the “repugnant” world would be better if it existed, and the objection is that this implication is implausible. The implausibility would remain even if it was shown that the “repugnant” world cannot exist.
Thank you very much, you put it words, what I could not. Your answer gave me not only the assurance that my doubts were justified, but also some confidence to ask more questions of that kind.Thank you.
Variant of Chinese room argument? This seems ironclad to me, what am I missing:
My claims:
Claim: AI feelings are unknowable: Maybe an advanced AI can have positive and negative sensations. But how would we ever know which ones are which (or how extreme they are?
Corollary: If we cannot know which are which, we can do nothing that we know will improve/worsen the “AI feelings”; so it’s not decision-relevant
Justification I: As we ourselves are bio-based living things, we can infer from the apparent sensations and expressions of bio-based living things that they are happy/suffering. But for non-bio things, this analogy seems highly flawed. If a dust cloud converges on a ‘smiling face’, we should not think it is happy.
Justification II. (Related) AI, as I understand it, is coded to learn to solve problems and maximise things, optimize certain outcomes or do things it “thinks” will yield positive feedback.
We might think then, that the AI ‘wants’ to solve these problems, and things that bring it closer to the solution make it ‘happier’. But why should we think this? For all we know, it may feel pain when it gets closer to the objective, and pleasure when it avoids this.
Does it tell us it makes it happy to come closer to the solution? That may merely because we programmed it to learn how to come to a solution, and one thing it ‘thinks’ will help is telling us it gets pleasure from doing so, even though it actually gains pain.
A colleague responded:
If we get the AI through a search process (like training a neural network) then there’s a reason to believe that AI would feel positive sensations (if any sensations at all) from achieving its objective since an AI that feels positive sensations would perform better at its objective than an AI that feels negative sensations. So, the AI that better optimizes for the objective would be more likely to result from the search process. This feels analogous to how we judge bio-based living things in that we assume that humans/animals/others seek to do those things that make them feel good, and we find that the positive sensations of humans are tied closely to those things that evolution would have been optimizing for. A version of a human that felt pain instead of pleasure from eating sugary food would not have performed as well on evolution’s optimization criteria.
OK but this seems only if we:
Knew how to induce or identify “good feelings”
Decided to induce these and tie them in as a reward for getting close to the optimum.
But how on earth would we know how to do 1 (without biology at least) and why would we bother doing so? Couldn’t the machine be just as good an optimizer without getting a ‘feeling’ reward from optimizing?
TL;DR silly critics to long-termism, can you convince me to keep donating to EA funds?
See positive arguments and introduction below. The following are supposed to be naive Critics/questions:
Utilitarianism critics. Even with the newer formulation of long-termism,
Why should I care about people that won’t exist? Say we go extinct, then what? (like in the movie “Her”, this could be the smart choice). I’m more on the pro-abortion side of the discussion here. Why should X-risks be costly because of their opportunity cost and not because of the immediate suffering?
Do we care only about humans? Isn’t out there any silly argument like “we should cultivate X insect because there will be a lot of them and a lot of lives means a lot of happiness”?
Definitely not clear what is good in the long run (nor what’s good now). True or false?
Privilege/anti-capitalism critics,
This is more of a feeling. Is long-termism supporting capitalism? Doesn’t the immediate solution of the world’s most pressing problem require a systemic change? Is it the only discourse that can adapt to the current state of affairs instead of demanding an urgent change?
On the same line, how are the preferences of non-privileged (e.g. poor) people taken into account when defining what is “effective”?
Does this view incentivize the acceptance of suffering in the present moment and in future present moments?
Is this view trying to solve the existential crisis arising when imagining a future with a sustainable but not growing (e.g. population) society?
Other
The expected discounted return seems more appropriate because the probabilities of changing our preferences grow with time. True or false?
How something that feels that bad (ignoring current suffering to prioritize maybe future happiness) could be right?
Is this a justification to natural selfishness (e.g. not giving up all we “can” give up)?
Introduction:
Hi everyone! I have read some of the posts on long-termism. Maybe one hour of reading, which is obviously not enough given the time devoted and depth achieved by people around here. However, I still feel the idea is terrible, which is why I compiled the previous naive list to have each point refuted. I definitely don’t like seeing the funds giving money to already privileged people (arguably more than me), but I trust there is a good reason behind. Please add a “IMO” before each phrase.
The most compelling pro arguments for long-termism investing are: 1- it makes sense for good investments to be counter-intuitive (because they are the most neglected and they wouldn’t be neglected if they were intuitive) and 2- long-termism is more about the coincidence between the correctness of globally good actions and the correctness of those actions in the long term future.
Any Too Long Didn’t Read—like answer will be very appreciated
1.1.: You might want to have a look at group of positions in metaethics called person affecting views, some of which include future people and some of which don’t. The ones that do often don’t care about increasing/decreasing the number of people in the future, but about improving the lives of future people that will exist anyway. That’s compatible with longtermism—not all longtermism is about extinction risk. (See trajectory change and s-risk.)
1.2.: No, we don’t just care about humans. In fact, I think it’s quite likely that most of the value or disvalue will come from non-human minds. (Though I’m thinking digital minds rather than animals.) But we can’t influence how the future will go if we’re not around, and many x-risk scenarios would be quite bad full stop and not just bad for humans.
1.3.: You might want to have a look at cluelessness (EA forum and GPI website should have links) or the recent 80,000 Hours podcast with Alexander Berger. Predicting the future and how we can influence it is definitely extremely hard, but I don’t think we’re decisively in bad enough of a position where we can—with a good conscience—just throw our hands up and conclude there’s definitely nothing to be done here.
2.
2.1 + 2.2.: Don’t really want to write anything on this right now
2.3.: Definite no. It just argues that trade-offs must be made, and some bads are worse even than current suffering. Or rather: The amount of bad we can avert is greater even than if we focus on current suffering
2.4: Don’t understand what you’re getting at.
3.
3.1.: Can’t parse the question
3.2.: I think many longtermists struggle with this. Michelle Hutchinson wrote a post on the EA forum recently on what still keeps her motivated. You can find it by searching her name ont he EA forum.
3.3.: No. Longtermism per se doesn’t say anything about how much to personally sacrifice. You can believe in longtermism + think that you should give away your last penny and work every waking hour in a job you don’t like. You can not be a longtermist and think you should live a comfortable, expensive life because that’s what’s most sustainable. Some leanings on this question might correlate with whether you’re a longtermist or not, but in principle, this question is orthogonal.
Sorry if the tone is brash. If so, that’s unintentional, and I tend to be really slow otherwise, but I appreciate that you’re thinking about this. (Also, I’m writing this as sleep procrastination, and my guilt is driving my typing speed)
Hi—my intuitions fall in the other direction here, so I’m keen to explain why. Implicit IMOs in front of everything here.
1: 1.1: I have a younger brother. My parents could have stopped at one, and my family would broadly still be happy, but my brother is generally happy and leads a good life. Similarly, if they’d had a third child they probably would have been happy and great too, and I would have loved them. All else being equal I wish that youngest sibling could have existed. IMO these two sentiments aren’t meaningfully distinct.
1.2: We don’t only care about humans. Sure, the argument for making more humans would apply to insects or something as well. However, most of the things that would kill all the humans would also kill everything else, so for me not letting that happen is still much more of a priority.
1.3: True on the specifics, false more generally. I don’t know exactly what the world should look like, but I’m pretty sure people being happy is good, more people being happy is better, and everything being unrecoverably dead is neutral at most.
2:
2.1: If we weren’t potentially about to all die I’d be more willing to think about this, but we have to survive the next century or two first. Whether capitalism makes things better or worse for now depends much more on whether it makes us more or less likely to all die, than on anything else (again, for now).
2.2: I’m pretty sure non-privileged people also want to be alive and happy.
2.3: Possibly, and I’m ok with that. I’d rather live a worse life if it means my grandkids are more likely to survive and have happy ones. Although it’s definitely better for everyone to be happier now, I feel like it doesn’t amount to much if we all die in the next century.
2.4: If I can choose between a surviving but stable society, and a growing one, I would choose the growing one. But both are better than an empty rock, so the priority now is not dying either way.
3:
3.1: I’m pretty sure we’ll continue to want to be alive and happy, so false. People can’t decide what their preferences are, and work to fulfil them, if they don’t exist.
3.2: Our moral intuitions were built for very different-looking societies to where we are today. We like sugar and sex because we were supposed to go for fruit and reproduction; our moral intuitions aren’t hugely different. IMO this is in a similar category to people caring more about saving one child than eight of them.
I should clarify 3.3. For me, longtermism is partly the acknowledgement of much vaster moral stakes—so long as there are things we can do to help, they’re no less important to do as short-termist interventions. (The usual arguments about it not being helpful to demand too much of people still apply though).
I know this is a really mainstream opinion, but I recently watched a recording of the musical Hamilton and I really liked it.
I think Hamilton (the character, not the historical figure which I know very little about) has many key flaws (most notably selfishness, pride, and misogyny(?)) but also virtues/attitudes that are useful to emulate.
I especially found the Non-stop song(lyrics) highly relatable/aspirational, at least for a subset of EA research that looks more like “reading lots and synthesize many thoughts quickly” and less like “think very deeply about a narrow research topic and come up with a fleeting novel insight.” Especially the refrain:
Why do you write like you’re running out of time? Write day and night like you’re running out of time? Every day you fight, like you’re running out of time Keep on fighting, in the meantime-
And
Why do you always say what you believe? Why do you always say what you believe? Every proclamation guarantees Free ammunition for your enemies (Awww!)
Why do you write like it’s going out of style? (Hey) Write day and night like it’s going out of style? (Hey) Every day you fight like it’s going out of style Do what you do
And
How do you write like you’re running out of time? (Running out of time?) Write day and night like you’re running out of time? (Running out of time?) Every day you fight, like you’re running out of time Like you’re running out of time Are you running out of time? Awwww!
How do you write like tomorrow won’t arrive? How do you write like you need it to survive? How do you write every second you’re alive? Every second you’re alive? Every second you’re alive?
I love Hamilton! I wrote my IB Extended Essay on it!
I also really love and relate to Non-Stop but in the obsessive, perfectionist way. I like + appreciate your view on it, which seems quite different in that it is more focused on how Hamilton’s brain works rather than on how hard he works.
Thanks! I don’t see nearly as much perfectionism (like none of the lyrics I can think of talks about rewriting things over and over), but I do think there’s an important element of obsession to Hamilton/Non-Stop, which I relate pretty hard to. Since I generate a lot of my expected impact from writing, and it’s quite hard to predict which of my ideas are the most useful/promising in advance, I do sometimes feel a bunch of internal pressure to write/think faster, produce more, etc, like a bit of a race against the clock to produce as much as I can (while maintaining quality standards).
So in the context of this song, I especially admire people (including the character Hamilton, some journalists, some bloggers, and coworkers) who manage to produce a lot of output without (much) sacrificing quality.
Near term AI risk: simulated social status. Social status is a big downside to playing video games all day / abusing drugs. GPT3 and friends are probably capable of giving you believable simulated social status, far beyond what current video games provide. This could be a big theraputic boon as well as well as a threat to people’s happiness and contribution to society.
I’m (still!!!) thinking about my BA thesis research question and I think my main uncertainty/decision point is what specific policy debate to investigate. I’ve narrowed it down to two so far—hopefully I don’t expand—and really welcome thoughts.
Context: I am examining the relationship between narratives deployed by experts on Twitter and the Biden Administration’s policymaking process re: COVID-19 vaccine diplomacy. Specifically, I want to examine a debate on an issue wherein EA-aligned experts have generally coalesced around one stance.
However, the U.S. public’s trust in experts has varied. It may have peaked last year and now be declining
Vaccine diplomacy (along with all health policy) is not solely an issue of ‘following the science’
This is not to say that data or rationality is not important. In fact, I would be extremely interested in investigating whether the combination of scientific evidence + thematic framing is more effective than either alone.
However, that would be an experimental study which is not something I am interested in.
This suggests I might want to investigate the presence of scientific vs thematic elements in expert narratives. Not sure though… It’s not what I’m immediately drawn to
Evidence/science alone is insufficient. Experts need to be able to tell stories/persuade/make a moral or emotional appeal. (Extrapolated from the claim that narratives can be influential in policymaking)
At the very least, experts should make clear that no decision is value-neutral and the specific values they are prioritizing in their recommendation
Now that I think about it, the fact that I’m ‘not sure’ about this re: COVID-19 might mean this would make for a good RQ? Or maybe I’m just not thinking of the relevant literature right now.
The two debates below, including general thoughts
The COVID-19 TRIPS Waiver (waiving IP)
What most excites me about this: The Biden Admin did a strong ‘about-face’ on this and the discourse around this was very rich (involved many actors with strong opinions, and entwines with debates around vaccine sharing etc.).
Main hesitation: I don’t know how to think about experts as an actor here. Should they be considered a coalition, per the Advocacy Coalition Framework? Or should I look at a specific set of aligned expert organizations/individuals? Or should I look at all experts on Twitter?
But ACF emphasizes long-term policymaking and shared beliefs—and it seems like there was no singular expert consensus on whether the TRIPS waiver would be a net good. Now that I think about it, this might be due to a lack of transparency over what is being [morally] prioritized...
But why focus on aligned orgs/individuals? How can I justify that? How generalizable is that even?
But if I include all experts, including experts who might have other avenues to policy influence (e.g. big think tanks or former officials), then why not also examine non-expert narratives?
Specifically, the rationale behind examine Twitter is that it provides a highly-accessible advocacy platform to people who do not otherwise have much visibility/leverage
Also, looking at a wide range of Tweets helps get a sense of the general narrative
What most excites me about this: There is an explicit non-epistemic debate here (prioritizing children domestically vs the global poor), and that is what I care the most about. There still remains a scientific/epistemic component, too: “Are children safe without vaccines?”
Additionally, there is an added controversial non-epistemic element of anti-maskers
Main hesitation: But the Biden administration hasn’t really ‘made a policy’ on this. So what policy process would I be examining?
This also straddles the line between domestic and international, in that the debate is primarily about picking between the two (in contrast to the first debate), which could be tricky
These both seem like great options! Of the two, I think the first has more to play with as there is a pretty clear delineation between the epistemic vs. moral elements of the second, whereas I think debates about the first have those all jumbled up and it’s thus more interesting/valuable to untangle them. I don’t totally understand your hesitation so I’m afraid I can’t offer much insight there, but with respect to long-term policymaking/shared beliefs, it does seem like the fault lines mapped onto fairly clear pro-free-market vs. pro-redistributive ideologies that drew the types of advocates one would have predicted given that divide.
*edit 3: After reading more on Epistemic Communities, I think I’m back where I started. *edit 4: I am questioning, now, whether I need a framework of how experts influence policymaking at all … Maybe I should conceptualize my actors more broadly but narrow the topic to, say, the use of evidence in narratives?
I really appreciate your response, Ian! I think it makes sense that the more convoluted status of the first debate would make it a more valuable question to investigate.
My hesitation was not worded accessibly or clearly—it was too grounded in the specific frameworks I’m struggling to apply—so let me reword: it doesn’t seem accurate to claim that there was one expert consensus (i.e. primarily pro-/anti-waiver). Given that, I am not sure a) how to break down the category of ‘expert’ - although you provide one suggestion, which is helpful—and b) how strongly I can justify focusing on experts, given that there isn’t a clear divide between “what experts think” and “what non-experts think.”
Non-TL;DR:
My main concern with investigating the debate around the TRIPS waiver is that there doesn’t seem to be a clear expert consensus. I’m not even sure there’s a clear EA-aligned consensus, although the few EAs I saw speak on this (e.g. Rob Wiblin) seemed to favor donating over waiving IP (which seems like a common argument from Europe). Given that, I question
the validity of investigating ‘expert narratives’ because ‘experts’ didn’t really agree there
However, I don’t know if it would be in/valid (per the theories I want to draw from, e.g. Advocacy Coalition Framework (ACF) or Epistemic Communities), so that would be one of my next steps.
This particular description worries me: “Advocacy coalitions are all those defined by political actors who share certain ideas and who coordinate among themselves in a functional way to suggest specific issues to the government and influence in the decision-making process.”
This would be subverted by your suggestion, though, as I note in point 3!
the validity of investigating expert narratives specifically instead of the general public—if experts didn’t coalesce around a specific stance, what’s my justification for investigating them specifically instead of getting a sense of the public generally? ACF explicitly notes that “common belief systems bind members of a coalition together.” Given that the pro-/anti-waiver coalitions are defined by common beliefs held by both experts and non-experts (e.g. pro-free-market), how can I justify exclusively focusing on experts?
This is probably not a valid concern, now that I think about it. After all, my thesis hinges upon the idea that experts help inform policymakers + policymaking, so it makes sense to focus on their narratives rather than looking at the public as a whole...
However, it seems like focusing exclusively on two expert groups is valid at least within the Epistemic Community framework, so perhaps this would work if it turns out that certain kinds of experts advocated for the same stance.
whom I should focus on—without being able to lump all experts together, how should I break them down?
Perhaps I could subdivide experts into coalitions—e.g. experts for the waiver and experts against the waiver? (This is akin to the fault lines you mention)
I still feel kind of iffy about investigating experts specifically here, instead of the general public, particularly because I could use the same coalitional divide (pro-/anti-waiver)
Or should I focus on EA-aligned experts specifically?
But I don’t know how to justify this… It doesn’t seem like the smartest research practice
Suggestion: use an expert lens, but make the division you’re looking at [experts connected to/with influence in the Biden administration] vs. [“outside” experts].
Rationale: The Biden administration thinks of and presents itself to the public as technocratic and guided by science, but as with any administration politics and access play a role as well. As you noted, the Biden administration did a clear about-face on this despite a lack of a clear consensus from experts in the public sphere. So why did that happen, and what role did expert influence play in driving it? Put another way, which experts was the administration listening to, and what does that suggest for how experts might be able to make change during the Biden administration’s tenure?
Hmm! Yes, that’s interesting—and aligns with the fact that many different policy influencers weighed in, ranging from former to current policymakers. Thank you very much for this!
I think something I’m worried about is how I can conceptualize [inside experts] vs. [outside experts] … It seems like a potentially arbitrary divide and/or a very complex undertaking given the lack of transparency into the policy process (i.e. who actually wields influence and access to Biden and Katherine Tai, on this specific issue?).
It also complicates the investigation by adding in the element of access as a factor, rather than purely thinking about narrative strategies—and I very much want to focus on narratives. On one hand, I think that could be interesting—e.g. looking at narrative strategies across levels of access. On the other, I’m uncertain that looking at narrative strategies would add much compared to just analyzing the stances of actors within the sphere of influence.
What do you think of this alternate RQ: “How did pro/anti-waiver coalitions use evidence in their narratives?”
Moves away from the focus on experts but still gets to the scientific/epistemic component.
(I’m also wondering whether I am being overly concerned with theoretically justifying things!)
(I’m also wondering whether I am being overly concerned with theoretically justifying things!)
I think I would agree with this. It seems like you’re trying to demonstrate your knowledge of a particular framework or set of frameworks through this exercise and you’re letting that constrain your choices a lot. Maybe that will be a good choice if you’re definitely going into academia as a political scientist after this, but otherwise, I would structure the approach around how research happens most naturally in the real world, which is that you have a research question that would have concrete practical value if it were answered, and then you set out to answer it using whatever combination of theories and methods makes sense for the question.
[This is an excerpt from a longer post I’m writing]
Suppose someone’s utility function is
U = f(C) + D
Where U is what they’re optimizing, C is their personal consumption, f is their selfish welfare as a function of consumption (log is a classic choice for f), and D is their amount of donations.
Suppose that they have diminishing utility wrt (“with respect to”) consumption (that is, df(C)/dC is strictly monotonically decreasing). Their marginal utility wrt donations is a constant, and their marginal utility wrt consumption is a decreasing function. There has to be some level of consumption where they are indifferent between donating a marginal dollar and consuming it. Below this level of consumption, they’ll prefer consuming dollars to donating them, and so they will always consume them. And above it, they’ll prefer donating dollars to consuming them, and so will always donate them. And this is why the GWWC pledge asks you to input the C such that dF(C)/d(C) is 1, and you pledge to donate everything above it and nothing below it.
This is clearly not what happens. Why? I can think of a few reasons.
The above is what you get if the selfish and altruistic parts of you “negotiate” once, before you find out how high your salary is going to be. If instead, you negotiate every year to spend some fair share of your resources on altruistic and selfish resources, you get something like what we see.
People aren’t scope sensitive about donations, and so donations also have diminishing marginal returns (because small ones are disproportionately good at making people think you’re good).
When you’re already donating a lot, other EAs will be less likely to hold consumption against you (perhaps because they want to incentivize rich and altruistic people to hang out in EA without feeling judged for only donating 90% of their $10M annual expenditure or whatever).
When you’re high income, expensive time-money tradeoffs like business class flights start looking better. And it’s often pretty hard to tell which purchases are time-money tradeoffs vs selfish consumption, and if your time is valuable enough, it’s not worth very much time to try to distinguish between these two categories.
Early-career people want to donate in order to set themselves up for a habit of donating later (and in order to signal altruism to their peers, which might be rational on both a community and individual level).
As you get more successful, your peers will be wealthier, and this will push you towards higher consumption. (You can think of this as just an expense that happens as a result of being more successful.)
I think that it seems potentially pretty suboptimal to have different levels of consumption at different times in your life. Like, suppose you’re going to have a $60k salary one year and a $100k salary the next. It would be better from both an altruistic and selfish perspective to concentrate your donations in the year you’ll be wealthier; it seems kind of unfortunate if people are unable to make these internal trades.
EDIT: Maybe a clearer way of saying my main point here: Suppose you’re a person who likes being altruistic and likes consuming things. Suppose you don’t know how much money you’re going to make next year. You’ll be better off in expectation from both a selfish and altruistic perspective if you decide in advance how much you’re going to consume, and donate however much you have above that. Doing anything else than this is Pareto worse.
The GWWC pledge is akin to a flat tax, as opposed to a progressive tax—which gives you a higher tax rate when you earn more.
I agree that there are some arguments in favour of “progressive donations”.
One consideration is that extremely high “donation rates”—e.g. donating 100% of your income above a certain amount—may affect incentives to earn more adversely, depending on your motivations. But in a progressive donation rate system with a more moderate maximum donation rate that would probably not be as much of a problem.
Below this level of consumption, they’ll prefer consuming dollars to donating them, and so they will always consume them. And above it, they’ll prefer donating dollars to consuming them, and so will always donate them. And this is why the GWWC pledge asks you to input the C such that dF(C)/d(C) is 1, and you pledge to donate everything above it and nothing below it.
Wait the standard GWWC pledge is a 10% of your income, presumably based on cultural norms like tithing which in themselves might reflect an implicit understanding that (if we assume log utility) a constant fraction of consumption is equallycostly to any individual, so made for coordination rather than single-player reasons.
Yeah but this pledge is kind of weird for an altruist to actually follow, instead of donating more above the 10%. (Unless you think that almost everyone believes that most of the reason for them to do the GWWC pledge is to enforce the norm, and this causes them to donate 10%, which is more than they’d otherwise donate.)
[epistemic status: I’m like 80% sure I’m right here. Will probably post as a main post if no-one points out big holes in this argument, and people seem to think I phrased my points comprehensibly. Feel free to leave comments on the google doc here if that’s easier.]
I think a lot of EAs are pretty confused about Shapley values and what they can do for you. In particular Shapley values are basically irrelevant to problems related to coordination between a bunch of people who all have the same values. I want to talk about why.
So Shapley values are a solution to the following problem. You have a bunch of people who can work on a project together, and the project is going to end up making some total amount of profit, and you have to decide how to split the profit between the people who worked on the project. This is just a negotiation problem.
One of the classic examples here is: you have a factory owner and a bunch of people who work in the factory. No money is made by this factory unless there’s both a factory there and people who can work in the factory, and some total amount of profit is made by selling all the things that came out of the factory. But how should the profit be split between the owner and the factory workers? The Shapley value is the most natural and mathematically nice way of deciding on how much of the profit everyone gets to keep, based only on knowing how much profit would be produced given different subsets of the people who might work together, and ignoring all other facts about the situation.
Let’s talk about why I don’t think it’s usually relevant. The coordination problem EAs are usually interested in is: Suppose we have a bunch of people, and we get to choose which of them take which roles or provide what funds to what organizations. How should these people make the decision of what to do?
As I said, the input to the Shapley value is the coalition value function, which, for every subset of the people you have, tells you how much total value would be produced in the case where just that subset tried to work together.
But if you already have this coalition value function, you’ve already solved the coordination problem and there’s no reason to actually calculate the Shapley value! If you know how much total value would be produced if everyone worked together, in realistic situations you must also know an optimal allocation of everyone’s effort. And so everyone can just do what that optimal allocation recommended.
Another way of phrasing this is that step 1 of calculating the Shapley value is to answer the question “what should everyone do” as well as a bunch of other questions of the form “what should everyone do, conditioned on only this subset of EAs existing”. But once you’ve done step 1, there’s no reason to go on to step 2.
A related claim is that the Shapley value is no better than any other solution to the bargaining problem. For example, instead of allocating credit according to the Shapley value, we could allocate credit according to the rule “we give everyone just barely enough credit that it’s worth it for them to participate in the globally optimal plan instead of doing something worse, and then all the leftover credit gets allocated to Buck”, and this would always produce the same real-life decisions as the Shapley value.
--
So I’ve been talking here about what you could call global Shapley values, where we consider every action of everyone in the whole world. And our measure of profit or value produced is how good the whole world actually ends up being. And you might have thought that you could apply Shapley values in a more local sense. You could imagine saying “let’s just think about the value that will be produced by this particular project and try to figure out how to divide the impact among the people who are working on this project”. But any Shapley values that are calculated in that way are either going to make you do the wrong thing sometimes, or rely on solving the same global optimization problem as we were solving before.
Let’s talk first about how the purely local Shapley values sometimes lead to you making the wrong decision. Suppose that some project that requires two people in order to do and will produce $10,000 worth of value if they cooperate on it. By symmetry, the Shapley value for each of them will be $5,000.
Now let’s suppose that one of them has an opportunity cost where they could have made $6,000 doing something else. Clearly, the two people should still do the $10,000 project instead of the $6,000 project. And so if they just made decisions based on the “local Shapley value”, they’d end up not doing the project. And that would end up making things overall worse. The moral of the story here is that the coalition profit function is measured in terms of opportunity cost, which you can’t calculate without reasoning globally. So in the case where one of the people involved had this $6,000 other thing they could have done with their time, the amount of total profit generated from the project is now actually only $4,000. Probably the best way of thinking about this is that you had to pay a $6,000 base salary to the person who could have made $6,000 doing something else. And then you split the $4k profit equally. And so one person ends up getting $8k and the other one ends up getting $2k.
--
I think a lot of EAs are hoping that you can use Shapley values to get around a lot of these problems related to coordination and figuring out counterfactual impact and all this stuff. And I think you just basically can’t at all.
I think Shapley values are more likely to be relevant to cases where people have different values, because in this case you have more like a normal negotiation problem, but even here, I think people overstate their relevance. Shapley values are just a descriptive claim about what might happen in the world rather than a normative claim about what should happen. In particular, they assume that everyone has equal bargaining power to start with which doesn’t seem particularly true.
I think the main way that Shapley values are relevant to coordination between people with different values is that they’re kind of like a Schelling fair way of allocating stuff. Maybe you want to feel cooperative with other people and maybe you don’t want to spend a lot of time going back and forth about how much everyone has to pay, and Shapley values are maybe a nice, fair solution to this. I haven’t thought this through properly yet.
In conclusion, Shapley values are AFAICT not relevant to figuring out how to coordinate between people who have the same goals.
But if you already have this coalition value function, you’ve already solved the coordination problem and there’s no reason to actually calculate the Shapley value! If you know how much total value would be produced if everyone worked together, in realistic situations you must also know an optimal allocation of everyone’s effort. And so everyone can just do what that optimal allocation recommended.
This seems correct
A related claim is that the Shapley value is no better than any other solution to the bargaining problem. For example, instead of allocating credit according to the Shapley value, we could allocate credit according to the rule “we give everyone just barely enough credit that it’s worth it for them to participate in the globally optimal plan instead of doing something worse, and then all the leftover credit gets allocated to Buck”, and this would always produce the same real-life decisions as the Shapley value.
This misses some considerations around cost-efficiency/prioritization. If you look at your distorted “Buck values”, you come away that Buck is super cost-effective; responsible for a large fraction of the optimal plan using just one salary. If we didn’t have a mechanistic understanding of why that was, trying to get more Buck would become an EA cause area.
In contrast, if credit was allocated according to Shapley values, we could look at the groups whose Shapley value is the highest, and try to see if they can be scaled.
The section about “purely local” Shapley values might be pointing to something, but I don’t quite know what it is, because the example is just Shapley values but missing a term? I don’t know. You also say “by symmetry...”, and then break that symmetry by saying that one of the parts would have been able to create $6,000 in value and the other $0. Needs a crisper example.
Re: coordination between people who have different values using SVs, I have some stuff here, but looking back the writting seems too corny.
Lastly, to some extent, Shapley values are a reaction to people calculating their impact as their counterfactual impact. This leads to double/triple counting impact for some organizations/opportunities, but not others, which makes comparison between them more tricky. Shapley values solve that by allocating impact such that it sums to the total impact & other nice properties. Then someone like OpenPhilanthropy or some EA fund can come and see which groups have the highest Shapley value (perhaps highest Shapley value per unit of money/ressources) and then try to replicate them/scale them. People might also make better decisions if they compare Shapley instead of counterfactual values (because Shapley values mostly give a more accurate impression of the impact of a position.)
So I see the benefits of Shapley values as fixing some common mistakes arising from using counterfactual values. This would make impact accounting slightly better, and coordination slightly better to the extent it relies on impact accounting for prioritization (which tbh might not be much.)
I’m not sure to what extent I agree with the claim that people are overhyping/misunderstanding Shapley values. It seems a plausible.
Summary for myself. Note: Pretty stream-of-thought.
Proving too much
The set of all possible futures is infinite which somehow breaks some important assumptions longtermists are apparently making.
Somehow this fails to actually bother me
...the methodological error of equating made up numbers with real data
This seems like a cheap/unjustified shot. In the world where we can calculate the expected values, it would seems fine to compare (wide, uncertain) speculative interventions with harcore GiveWell data (note that the next step would probably be to get more information, not to stop donating to GiveWell charities)
Sometimes, expected utility is undefined (Pasadena game)
The Pasadena game also fails to bother me, because the series hasn’t (yet) showed that longtermism bets are “Pasadena-like”
(Also, note that you can use stochastic dominance to solve many expected value paradoxes, e.g, to decide between two universes with infinite expected value, or with undefined expected value.)
...mention of E.T. Jaynes
Yeah, I’m also a fan of E.T. Jaynes, and I think that this is a cheap shot, not an argument.
Subject, Object, Instrument
This section seems confused/bad. In particular, there is a switch from “credences are subjective” to “we should somehow change our credences if this is useful”. No, if one’s best guess is that “the future is vast in size”, then considering that one can change one’s opinions to better attain goals doesn’t make it stop being one’s best guess
Overall: The core of this section seems to be that expected values are sometimes undefined. I agree, but this doesn’t deter me from trying to do the most good by seeking more speculative/longtermist interventions. I can use stochastic dominance when expected utility fails me.
Then, using our figure of one quadrillion lives, the expected good done by Shivani contributing $10,000 to [preventing world domination by a repressive global political regime] would, by the lights of utilitarian axiology, be 100 lives. In contrast, funding for the Against Malaria Foundation, often regarded as the most cost-effective intervention in the area of short-term global health improvements, on average saves one life per $3500. (Nuño: italics and bold from the OP, not from original article)
I agree that the paragraph just intuitively looks pretty bad, so I looked at the context:
Now, the argument we are making is ultimately a quantitative one: that the expected impact one can have on the long-run future is greater than the expected impact one can have on the short run. It’s not true, in general, that options that involve low probabilities of high stakes systematically lead to greater expected values than options that involve high probabilities of modest payoffs: everything depends on the numbers. (For instance, not all insurance contracts are worth buying.) So merely pointing out that one might be able to influence the long run, or that one can do so to a nonzero extent (in expectation), isn’t enough for our argument. But, we will claim, any reasonable set of credences would allow that for at least one of these pathways, the expected impact is greater for the long-run.
Suppose, for instance, Shivani thinks there’s a 1% probability of a transition to a world government in the next century, and that $1 billion of well-targeted grants — aimed (say) at decreasing the chance of great power war, and improving the state of knowledge on optimal institutional design — would increase the well-being in an average future life, under the world government, by 0.1%, with a 0.1% chance of that effect lasting until the end of civilisation, and that the impact of grants in this area is approximately linear with respect to the amount of spending. Then, using our figure of one quadrillion lives to come, the expected good done by Shivani contributing $10,000 to this goal would, by the lights of a utilitarian axiology, be 100 lives. In contrast, funding for Against Malaria Foundation, often regarded as the most cost-effective intervention in the area of short-term global health improvements, on average saves one life per $3500
Yeah, this is in the context of a thought experiment. I’d still do this with distributions rather than with point estimates, but ok.
The Credence Assumption
Ok, so the OP wants to argue that expected value theory breaks ⇒ the tool is not useful ⇒ we should abandon credences ⇒ longtermism somehow fails.
But I think that “My best guess is that I can do more good with more speculative interventions” is fairly robust to that line of criticism; it doesn’t stop being my best guess just because credences are subjective.
E.g., if my best guess is that ALLFED does “more good” (e.g., more lives saved in expectation) than GiveWell charities, pointing out that actually the expected value is undefined (maybe the future contains both infinite amounts of flourishing and suffering) doesn’t necessarily change my conclusion if I still think that donating to ALLFED is stochastically dominant.
Cox Theorem requires that probabilities be real numbers
The OP doesn’t buy that. Sure, a piano is not going to drop on his head, but he might e.g., make worse decisions on account of being overconfident because he has not been keeping track of his (numerical) predictions and thus suffers from more hindsight bias than someone who kept track.
But what alternative do we have?
One can use e.g., upper and lower bounds on probabilities instead of real valued numbers: Sure, I do that. Longtermism still doesn’t break.
Instead of relying on explicit expected value calculations, we should rely on evolutionary approaches
The Poverty of Longtermism
“In 1957, Karl Popper proved it is impossible to predict the future of humanity, but scholars at the Future of Humanity Institute insist on trying anyway”
I’m becoming concerned that the title “EA-aligned organisation” is doing more harm than good. Obviously it’s pointing at something real and you can expect your colleagues to be familiar with certain concepts, but there’s no barrier to calling yourself an EA-aligned organisation, and in my view some are low or even negative impact. The fact that people can say “I do ops at an EA org” and be warmly greeted as high status even if they could do much more good outside EA rubs me the wrong way. If people talked about working at a “high-impact organisation” instead, that would push community incentives in a good way I think.
I have exactly the opposite intuition (which is why I’ve been using the term “EA-aligned organization” throughout my writing for CEA and probably making it more popular in the process).
“EA-aligned organization” isn’t supposed to mean “high-impact organization”. It’s supposed to mean “organization which has some connection to the EA community through its staff, or being connected to EA funding networks, etc.”
This is a useful concept because it’s legible in a way impact often isn’t. It’s easy to tell whether an org has a grant from EA Funds/Open Phil, and while this doesn’t guarantee their impact, it does stand in for “some people at the community vouch for their doing interesting work related to EA goals”.
I really don’t like the term “high-impact organization” because it does the same sneaky work as “effective altruist” (another term I dislike). You’re defining yourself as being “good” without anyone getting a chance to push back, and in many cases, there’s no obvious way to check whether you’re telling the truth.
Consider questions like these:
Is Amazon a high-impact organization? (80K lists jobs at Amazon on their job board, so… maybe? I guess certain jobs at Amazon are “high-impact”, but which ones? Only the ones 80K posts?)
Is MIRI a high-impact organization? (God knows how much digital ink has been spilled on this one)
It seems like there’s an important difference between MIRI and SCI on the one hand, Amazon and Sunrise on the other. The first two have a long history of getting support, funding, and interest from people in the EA movement; they’ve given talks at EA Global. This doesn’t necessarily make them most impactful than Amazon and Sunrise, but it does mean that working at one of those orgs puts you in the category of “working at an org endorsed by a bunch of people with common EA values”.
*****
The fact that people can say “I do ops at an EA org” and be warmly greeted as high status even if they could do much more good outside EA rubs me the wrong way.
I hope this doesn’t happen very often; I’d prefer that we greet everyone with equal warmth and sincere interest in their work, as long as the work is interesting. Working at an EA-aligned org really shouldn’t add much signaling info to the fact that someone has chosen to come to your EA meetup or whatever.
That said, I sympathize with theoretical objections like “how am I supposed to know whether someone would do more good in some other job?” and “I’m genuinely more interested in hearing about someone’s work helping to run [insert org] than I would if they worked in finance or something, because I’m familiar with that org and I think it does cool stuff”.
Terms that seem to have some of the good properties of “EA-aligned” without running into the “assuming your own virtue” problem:
“Longtermist” (obviously not synonymous with “EA-aligned”, but it accurately describes a subset of orgs within the movement)
“Impact-driven” or something like that (indicating a focus on impact without insisting that the focus has led to more impact)
“High-potential” or “promising” (indicating that they’re pursuing a cause area that looks good by standard EA lights, without trying to assume success — still a bit self-promotional, though)
Actually referring to the literal work being done, e.g. “Malaria prevention org”, “Alternative protein company”
...but when you get at the question of what links together orgs that work on malaria, alternative proteins, and longtermist research, I think “EA-aligned” is a more accurate and helpful descriptor than “high-impact”.
Oh, I would have thought it’s the other way around—sometimes people don’t want to be known as EA-aligned because that can have negative connotations (being too focused on numbers, being judgmental of “what’s worthy”, slightly cult-like etc). I think “high-impact organisation” may be a good idea as well.
Running a literal school would be awesome, but seems too consuming of time and organisational resources to do right now.Assuming we did want to do that eventually, what could be a suitable smaller step? Founding an organisation with vetted staff, working full-time on promoting analytical and altruistic thinking to high-schoolers—professionalising in this way increases the safety and reputability of these programs. Its activities should be targeted to top schools, and could include, in increasing order of duration:
One-off outreach talks at top schools
Summer programs in more countries, and in more subjects, and with more of an altruistic bent (i.e. variations on SPARC and Eurosparc)
Recurring classes in things like philosophy, econ, and EA. Teaching by visitors could be arranged by liaising to school teachers, similarly to how external teachers are brought in for chess classes.
After-school, or weekend, programs for interested students
I’m not confident this would go well, given the various reports from Catherine’s recap and Buck’s further theorising. But targeting the right students, and bringing the right speakers, gives it a chance of success. If you get to (3-4), all is going well, and the number of interested teachers and students are rising, it would be very natural for the org to scale into a school proper.
Below, I briefly discuss some motivating reasons, as I see them, to foster more interdisciplinary thought in EA. This includes ways EA’s current set of research topics might have emerged for suboptimal reasons.
More EA-relevant interdisciplinary research : why?
The ocean of knowledge is vast. But the knowledge commonly referenced within EA and longtermism represents only a tiny fraction of this ocean.
I argue that EA’s knowledge tradition is skewed for reasons including but not-limited-to the epistemic merit of those bodies of knowledge. There are good reasons for EA to focus in certain areas:
Direct relevance (e.g. if you’re trying to do good, it seems clearly relevant to look into philosophy a bunch; if you’re trying to do good effectively, it seems clearly relevant to look into economics (among others) a bunch; if you came to think that existential risks are a big deal, it is clearly relevant to look into bioengineering, international relations, etc. a bunch; etc.)
Evidence of epistemic merit (e.g. physics has more evidence for epistemic merit than psychology, which in return has more evidence for epistemic merit than astrology; in other words, beliefs gathered from different fields are are likely to pay more/less rent, or are likely to be more/less explanatory virtuous)
However, some of the reasons we’ve ended up with our current foci may not be as good:
The, in parts arbitrary, way academic disciplines have been carved up
Inferential distances between knowledge traditions that hamper the free diffusion of knowledge between disciplines and schools of thought
Having a skewed knowledge basis is problematic. There is a significant likelihood that we are missing out on insights or perspectives that might critically advance our undertaking. We don’t know what we don’t know. We have all the reasons to expect that we have blindspots.
***
I am interested in the potential value and challenges of interdisciplinary research.
Neglectedness
(Academic) incentives make it harder for transdisciplinary thought to flourish, resulting in what I expect to be an undersupply thereof. One way of thinking about why we would see an undersupply of interdisciplinry thought is in terms of “market inefficiencies”. For one, individual actors are incentivised (because it’s less risky) to work on topics that are already recognised as interesting by the community (“exploitation”), as opposed to venturing into new bodies of knowledge that might or might not prove insightful (“exploration”). What is “already recognized as valuable by the community”, however, will only in part be determined by epistemic considerations, and in another part be shaped by path-dependencies.
For two, “markets” are insufficiently liquid and thus tend to fail where we cannot easily specify what we want. I’d argue that this is the case for DS/ET work. This is generally true for intellectual work, but is likely even more true for DS/ET work due to the relatively siloed structure of academia that adds additional “transaction costs” to attempts of communicating across disciplinary boundaries.
One way to reduce these inefficiencies is by improving the interfaces between the disciplines. “Domain scanning” and “episetmic translation” are precisely about creating such interfaces. Their purpose is to identify knowledge that is concretely relevant to a given target domain and make that knolwege accessible to thinkers entrenched in the “vocabulary” of that target domain. A useful interface between political philosophy and computer science, for example, might require a mathematical formalization of central ideas such as justice.
Challenges
At the same time, doing interdisciplinary well is callenging. For example, interdisciplinary research can only be as valuable as a researcher’s ability to identify knowledge relevant to their target domain; or as a research community’s quality assurance/error correction mechanisms. Phenomena like citogenesis or motivatiogensis are examples of manifestations of these difficulties.
There have been various attempts at overcoming these incentive barriers, for example the Santa Fe Institute whose organizational structure completely disregards scientific disciplines; -ARPAs have a similar flavour; the field of cybernetics which proposed an inherently transdisciplinary view on regulatory systems; or the recent surge in the literature on “mental models” (e.g. here or here).
A closer inspection of such examples—in how far they were successful and how they went about it—might bear some interesting insights. I don’t have the capacity to properly puruse such case studies in the near future, but it’s definteily something on my list of potentially promising (side) projects.
If readers are aware of other examples of innovative approaches trying to solve this problem that might make for insightful case studies, I’d love to hear them.
I think RAND is a good case study for interdisciplinary approaches to problem solving, though I’m biased. The key there, as in industry and most places other than academia, but unlike Santa Fe and the ARPAs, is a focus on solving concrete specific problems regardless of the tools used.
Also, big +1 to cybernetics, which is an interesting case study for 2 reasons, first because of what worked, and second because of how it was supplanted / coopted into narrow disciplines, and largely fizzled out as its own thing.
Our World in Data has created two great posts this year, highlighting how the often proposed dichotomy between economic growth & sustainability is false.
the average income in the world is int.-$16 per day
Which is far below what we’d think of as the poverty line in developed countries. This means that mere redistribution of what we have is insufficient—we’d all end up poor and unable to continue developing much further because we’re too occupied with mere survival. In How much economic growth is necessary to reduce global poverty substantially?, he writes:
I found that $30 per day is, very approximately, the level below which people are considered poor in high-income countries.
As you will see in our writing there are several important cases in which an increased consumption of specific products gets into unavoidable conflict with important environmental goals; in such cases we aim to emphasize that we all as individuals, but also entire societies, should strongly consider to reduce the consumption of these products – and thereby reduce the use of specific resources and forgo some economic growth – to achieve these environmental goals. We believe a clear understanding of which specific reductions in production and consumption are necessary to reduce our impact on the environment is a much more forceful approach to reducing environmental harm than an unspecific opposition to economic growth in general.
So for discussions on how to approach individual “consumption” or policymaking around it, we could start a list of specific products to avoid. Would somebody be up for compiling this? It would be a resource I’d link to quite regularly. You can apparently just extract them from the 13 links Max Roser put just above the paragraph cited above. It would make for a great, short and crisp EA Forum post, too.
An organiser from Stanford EA asked me today how community building grants could be made more attractive. I have two reactions:
Specialised career pathways. To the extent that this can be done without compromising effectiveness, community-builders should be allowed to build field-specialisations, rather than just geographic ones. Currently, community-builders might hope to work at general outreach orgs like CEA and 80k. But general orgs will only offer so many jobs. Casting the net a bit wider, many activities of Forethought Foundation, SERI, LPP, and FLI are field-specific outreach. If community-builders take on some semi-specialised kinds of work in AI, or policy, or econ, (in connection with these orgs or independently) then this would aid their prospects of working for such orgs or returning to a more mainstream pathway.
“Owning it”.To the extent that community building does not offer a specialised career pathway, the fact that it’s a bold move should be incorporated into the branding. The Thiel Fellowship offers $100k to ~2 dozen students per year, to drop out of their programs to work on a startup that might change the world. Not everyone will like it, but it’s bold, it’s a round, and reasonably-sized number, with a name attached, and a dedicated website. Imagine a “Macaskill fellowship” that offers $100k for a student from a top university to pause their studies and spend one year focusing on promoting prioritisation and long-term thinking—it’d be a more attractive path.
If so, what/how? I don’t think full-time monitoring makes sense (first rule of comms: do everything with a full comms strategy in mind!) but I wonder if a list or Airtable would still be useful for organizations to pull from or something...
My hope is that people who see EA-relevant press will post it here (even in Shortform!).
I also track a lot of blogs for the EA Newsletter and scan Twitter for any mention of effective altruism, which means I catch a lot of the most directly relevant media. But EA’s domain is the entire world, so no one person will catch everything important. That’s what the Forum is for :-)
I’m not sure whether you’re picturing a project specific to stories about EA or one that covers many other topics. In the case of the former, me and others at CEA know about nearly everything (though we don’t have it in a database; no one ever asks). In the case of the latter, the “database” in question would probably just be… Google? I’m having trouble picturing the scenario where an org needs to pull from a list of articles they wouldn’t find otherwise. (But I’m open to being convinced!)
I think David Nash does something similar with his EA Updates (here is the most recent one). While most of the links are focused on EA Forum and posts by EA/EA-adj orgs, he features occasional links from other venues.
I enjoyed learning about the Henry Spira award. It is given by the Johns Hopkins School of Public Health to “honor animal activists in the animal welfare, protection, or rights movements who work to achieve progress through dialogue and collaboration.”
The criteria for the award are based on Peter Singer’s summary of the methods Spira used in his own advocacy. Many of them seem like strong guiding principles for EA work in general:
Understands public opinion, and what people outside of the animal rights/welfare movement are thinking.
Selects a course of action based upon public opinion, intensity of animal suffering, and the opportunities for change.
Sets goals that are achievable and that go beyond raising public awareness. Is willing to bring about meaningful change one step at a time.
Is absolutely credible. Doesn’t rely on exaggeration or hype to persuade
Is willing to work with anyone to make progress. Doesn’t “divide the world into saints and sinners.”
Seeks dialogue and offers realistic solutions to problems.
Is courageous enough to be confrontational if attempts at dialogue and collaboration fail.
Avoids bureaucracy and empire-building.
Tries to solve problems without going through the legal system, except as a last resort.
Asks “Will it work?” whenever planning a course of action. Stays focused on the practical realities involved in change, as well as ethical and moral imperatives.
See the bottom of this page for a list of winners (most recently in 2011; I don’t know whether the award is still “active”).
These models predicted growth followed by collapse. The first part has been proven correct, but there is little evidence for the second. Acting like past observations of growth are evidence of future collapse seems like an unusual example of Goodman’s New Riddle of Induction in the wild.
To clarify—“little evidence” implies that you think observations of current conditions aligning with model predictions, e.g. “Previous studies that attempted to do this found that the model’s worst-case scenarios accurately reflected real-world developments,” are weak?
Maybe someone should make ~1 Anki card each for lots of EA Wiki entries, then share that Anki deck on the Forum so others can use it?
Specifically, I suggest that someone:
Read/skim many/most/all of the EA Wiki entries in the “Cause Areas” and “Other Concepts” sections
Anki cards based on entries in the other sections (e.g., Organisations) would probably be less useful
Make 1 or more Anki card for many/most of those entries
In many cases, these cards might take forms like “The long reflection refers to… [answer]”
In many other cases, the cards could cover other insights, concepts, questions, etc. raised in the body of the entry
Making such cards seems less worthwhile for cases in which either:
the entry mainly exists as a tag (without itself having much content)
the entry is about a quite well-known thing and doesn’t really say much that’s not well-known in its body (e.g., the International relations tag)
Export the file for the resulting deck and share it on the Forum and maybe elsewhere
Other people can then either use the whole deck or pick and choose which parts of the deck to use (e.g., deleting cards when they come up, if the person feels those cards aren’t relevant to their interests and plans)
I think this could also be done gradually and/or by multiple people, rather than in one big batch by one person. It could also be done for the LessWrong Wiki.
If someone does make this deck, I would very likely use some/many of the cards myself and also promote the deck to a bunch of other people.
(I also currently feel that this would be a sufficiently useful action to have taken that I’d be inclined to reward the person with some token amount of my own money to signal my appreciation / to compensate them for their time / because me saying that now might incentivise them. I’d only do this if the cards the person makes actually seems good. Feel free to contact me if you want to discuss that.)
Turning the EA Wiki into a (huge) Anki deck is on my list of “Someday/Maybe” tasks. I think it might be worth waiting a bit until the Wiki is in a more settled state, but otherwise I’m very much in favor of this idea.
There is an Anki deck for the old LW wiki. It’s poorly formatted and too coarse-grained (one note per article), and some of the content is outdated, but I still find it useful, which suggests to me that a better deck of the EA Wiki would provide considerable value.
The EA community has collected and developed a very large set of ideas that aren’t widely known outside of EA, such that “getting up to speed” can take a similar amount of effort to a decent fraction of a bachelor’s degree
But the community is relatively small and new (compared to e.g. most academic fields), so we have relatively little in the way of textbooks, courses, summaries, etc.
This means it can take a lot of effort and time to get up to speed, lots of EAs have substantial “gaps” in their “EA knowledge”, lots of concepts are misinterpreted or conflated or misapplied, etc.
The EA Wiki is a good step towards having good resources to help people get up to speed
A bunch of research indicates retrieval practice, especially when spaced and interleaved, can improve long-term retention and can also help with things like application of concepts (not just memory)
And Anki provides such spaced, interleaved retrieval practice
I’m being lazy in not explaining the jargon or citing my sources, but you can find some explanation and sources here: Augmenting Long-term Memory
If one person makes an Anki deck based on the EA Wiki entries, it can then be used and/or built on by other people, can be shared with participants in EA Fellowships, etc.
Possible reasons not to do this:
“There’s a lot of stuff it’d be useful for people to know that isn’t on EA Wiki entries. Why not make Anki cards on those things instead? Isn’t this a bit insular?”
I think we can and should do both, rather than one or the other
Same goes for having Anki cards based on EA sources vs Anki cards based on non-EA sources
Personally, I’d guess ~25% of my Anki cards are based on EA sources, ~70% are based on non-EA sources but are about topics I see as important for EA reasons, and 5% are random personal life stuff
I think there are a lot of people in the EA community for whom engaging with the EA Wiki entries to the extent required to make this deck would be worthwhile just for themselves
I also think there are even more people in the EA community for whom using all or a subset of these cards will be worthwhile
(Though there are also of course people for whom these things aren’t true)
“Many of the entries probably won’t actually be that well-suited to Anki cards, or aren’t on very important things”
Agreed
But many will be
The card-maker(s) can skip entries, and the card-users can delete some cards from their own copy of the deck
Takeaways from some reading about economic effects of human-level AI
I spent some time reading things that you might categorise as “EA articles on the impact of human-level AI on economic growth”. Here are some takeaways from reading these (apologies for not always providing a lot of context / for not defining terms; hopefully clicking the links will provide decent context).
Economic growth models that (essentially) take growth to be driven by population seem to fit / explain historic growth data fairly well.
(in at least most cases, I think) these predict that “AI robots” will lead to explosive growth (30%+ GWP growth for 10 years).
Things other than AI robots that could cause (effective) population to dramatically increase include: whole brain emulation, artificial wombs, genetic engineering, cultural evolution (naturally favouring cultures that tend to have lots of children), behavioural changes driven by something other than cultural evolution.
Ben Garfinkel cast some doubt on some of the data (I think especially pre-1500 CE) that is used to fit e.g. the Roodman model, but Tom Davidson doesn’t think this is a decisive blow for these kinds of models.
Tom Davidson thinks that “explosive growth would require AI that substantially accelerates the automation of a very wide range of tasks in production, R&D, and the implementation of new technologies.”
Tom Davidson gives ~60% as his probability of explosive growth this century given early enough “virtual AI” (though he says this is extremely rough!).
He thinks the main ways virtual AI doesn’t lead to explosive growth are bottlenecks like getting raw materials or conducting physical experiments, or that the economic models that imply explosive growth from AI robots are wrong
Tom Davidson’s (toy) mechanism for AI replacing human labour is
AI becomes good enough to do any task a top quality human worker could do working remotely
This allows us to quickly develop the level of robotics for AI robots capable of explosive growth by replacing human workers at ~almost all tasks
Energy payback times for solar panels are currently ~1 year at best (financial payback times are longer). If historic experience curves continue to hold, this could fall to months if solar panels supply a large fraction of current demand.
Terrestrial solar power could provide up to 1000x current civilizational energy usage, so there’s room for a new “AI economy” to grow its energy usage significantly (depending on future pre-AI-economy energy usage, of course).
Combining experience curves, evidence regarding feasibility of technological capabilities from nature, and availability of solar energy suggests that leaving earth will be feasible, which will in turn make settling other stars in the galaxy feasible, which will in turn (maybe) make settling other galaxies feasible.
Roodman’s model is arguably not useful for forecasting the arrival of AI, because it basically assumes powerful AI (it assumes that population is output bottlenecked).
If you’re interested in more on this topic, I’d highlight Holden Karnofsky’s recent blog series and Tom Davidson’s recent Open Phil report as good places to start.
In case it’s useful for other people, here’s the main stuff I (at least partially) read / listened to:
Experience curves, large populations, and the long run (from Carl Shulman’s blog)
Terrestrial solar energy could eventually support extremely large economies and populations (from Carl Shulman’s blog)
Roadmap for the “most important century” series (Holden Karnofsky’s Cold Takes blog post series)
Phil Trammell on Economic Growth under Transformative AI (Hear This Idea podcast)
Does Economic History Point Toward a Singularity? (Ben Garfinkel EA Forum post)
Could Advanced AI Drive Explosive Economic Growth? (report by Tom Davidson of Open Phil; see also accompanying blog post)
Quick BOTEC of person-hours spent on EA Job Applications per annum.
I created a Guesstimate model to estimate a total of ~14,000 to 100,000 person-hours or ~7 to 51 FTE are spent per year (90% CI). This comes to an estimated USD $ 320,000 to $3,200,000 unpaid labour time.
All assumptions for my calculations are in the Guesstimate
The distribution of effort spent by candidates is heavy-tailed; a small percentage of candidates may spend 3 to 10x more time than the median candidate.
I am not very good at interpreting the guesstimate, so if someone can state this better / more accurately than would be helpful
Keen to get feedback on whether I’ve over/underestimated any variables.
I’d expect this to grow at a rate of ~5-10% per year at least.
Sources: My own experience as a recruiter, applying to EA jobs and interviewing staff at some EA orgs.
Edited the unpaid labour time to reflect Linch’s suggestions.
I think
As a normal distribution between $20-30 is too low, many EA applicants counterfactually have upper middle class professional jobs in the US.
I also want to flag that you are assuming that the time is
but many EA orgs do in fact pay for work trials. “trial week” especially should almost always be paid.
Hi Linch, thanks for the input!
I’ll adjust the estimate a bit higher. In the Guesstimate I do discount the hours to say that 75% of the total hours are unpaid (trial week hours cone to 5% of the total hours).
I did not review the model, but only 75% of hours being unpaid seems much too low based on my experience having gone through the job hiring process (including later stages) with 10-15 EA orgs.
Okay, so I used a different method to estimate the total manhours and my new estimate is something like 60%. I basically assumed that 50% of Round 2 −4 in the application process is paid, and 100% of the work trial.
I expect that established / longtermist orgs are disproportionately likely to pay for work tests, compared to new or animal / GH&D orgs.
I think Josh was claiming that 75% was “too low”, as in the total % of unpaid hours being more like 90% or something.
When I applied to a bunch of jobs, I was paid for ~30 of the ~80 hours I spent (not counting a long CEA work trial — if you include that, it’s more like 80 out of 130 hours). If you average Josh and I, maybe you get back to an average of 75%?
*****
This isn’t part of your calculation, but I wonder what fraction of unique applicants to EA jobs have any connection to the EA community beyond applying for one job?
In my experience trying to hire for one role with ~200 applicants, ~1/3 of them neither had any connection to EA in their resumes nor provided further information in their applications about what drew them to EA. This doesn’t mean there wasn’t some connection, but a lot of people just seemed to be looking for any job they could find. (The role was more generic than some and required no prior EA experience, so maybe drew a higher fraction of outside applicants.)
Someone having no other connection to the EA community doesn’t mean we should ignore the value of their time, and the people who apply to the most jobs are likely to have the strongest connections, so this factor may not be too important, but it could bear consideration for a more in-depth analysis.
Dear forum,
I was wondering if the repugnant conclusion could be responded by an argument of the following form:
Considering planet earth and a given happiness distribution of its citizens with total happiness h, there is simply not enough space or resources or whatsoever to let an arbitrary large number of people n live with an average amount of happiness epsilon, such that n * epsilon > h. At even larger scales, the observable universe is finite and thus for the same reason as above n does not need to exist.
What do you think of such an argument?
I am not sure, whether the nature of the repugnant conclusion is really affected by such an argument. Can you help me to understand?
The repugnant conclusion is presented as an objection to certain views in population axiology. The claim is that a possible world containing sufficiently many morally relevant beings just above neutrality is intrinsically better than a possible world with arbitrarily many beings arbitrarily happy. The claim is not that these worlds could become actual, so empirical considerations of the sort you describe aren’t relevant for assessing the force of the objection.
Put differently, theories like total utilitarianism imply that the “repugnant” world would be better if it existed, and the objection is that this implication is implausible. The implausibility would remain even if it was shown that the “repugnant” world cannot exist.
Thank you very much, you put it words, what I could not. Your answer gave me not only the assurance that my doubts were justified, but also some confidence to ask more questions of that kind.Thank you.
Related—The Upper Limit of Value
And thank you as well for the short, but helpful answer. The relevance of the thought of mine for philosophy gives also confidence to that thinking.
Btw we have a some friends in common of which I am aware: EdoArad → (Shay ben moshe) → Amit → Arne
^^
Variant of Chinese room argument? This seems ironclad to me, what am I missing:
My claims:
Claim: AI feelings are unknowable: Maybe an advanced AI can have positive and negative sensations. But how would we ever know which ones are which (or how extreme they are?
Corollary: If we cannot know which are which, we can do nothing that we know will improve/worsen the “AI feelings”; so it’s not decision-relevant
Justification I: As we ourselves are bio-based living things, we can infer from the apparent sensations and expressions of bio-based living things that they are happy/suffering. But for non-bio things, this analogy seems highly flawed. If a dust cloud converges on a ‘smiling face’, we should not think it is happy.
Justification II. (Related) AI, as I understand it, is coded to learn to solve problems and maximise things, optimize certain outcomes or do things it “thinks” will yield positive feedback.
We might think then, that the AI ‘wants’ to solve these problems, and things that bring it closer to the solution make it ‘happier’. But why should we think this? For all we know, it may feel pain when it gets closer to the objective, and pleasure when it avoids this.
Does it tell us it makes it happy to come closer to the solution? That may merely because we programmed it to learn how to come to a solution, and one thing it ‘thinks’ will help is telling us it gets pleasure from doing so, even though it actually gains pain.
A colleague responded:
OK but this seems only if we:
Knew how to induce or identify “good feelings”
Decided to induce these and tie them in as a reward for getting close to the optimum.
But how on earth would we know how to do 1 (without biology at least) and why would we bother doing so? Couldn’t the machine be just as good an optimizer without getting a ‘feeling’ reward from optimizing?
Please tell me why I’m wrong.
TL;DR silly critics to long-termism, can you convince me to keep donating to EA funds?
See positive arguments and introduction below. The following are supposed to be naive Critics/questions:
Utilitarianism critics. Even with the newer formulation of long-termism,
Why should I care about people that won’t exist? Say we go extinct, then what? (like in the movie “Her”, this could be the smart choice). I’m more on the pro-abortion side of the discussion here. Why should X-risks be costly because of their opportunity cost and not because of the immediate suffering?
Do we care only about humans? Isn’t out there any silly argument like “we should cultivate X insect because there will be a lot of them and a lot of lives means a lot of happiness”?
Definitely not clear what is good in the long run (nor what’s good now). True or false?
Privilege/anti-capitalism critics,
This is more of a feeling. Is long-termism supporting capitalism? Doesn’t the immediate solution of the world’s most pressing problem require a systemic change? Is it the only discourse that can adapt to the current state of affairs instead of demanding an urgent change?
On the same line, how are the preferences of non-privileged (e.g. poor) people taken into account when defining what is “effective”?
Does this view incentivize the acceptance of suffering in the present moment and in future present moments?
Is this view trying to solve the existential crisis arising when imagining a future with a sustainable but not growing (e.g. population) society?
Other
The expected discounted return seems more appropriate because the probabilities of changing our preferences grow with time. True or false?
How something that feels that bad (ignoring current suffering to prioritize maybe future happiness) could be right?
Is this a justification to natural selfishness (e.g. not giving up all we “can” give up)?
Introduction:
Hi everyone! I have read some of the posts on long-termism. Maybe one hour of reading, which is obviously not enough given the time devoted and depth achieved by people around here. However, I still feel the idea is terrible, which is why I compiled the previous naive list to have each point refuted. I definitely don’t like seeing the funds giving money to already privileged people (arguably more than me), but I trust there is a good reason behind. Please add a “IMO” before each phrase.
The most compelling pro arguments for long-termism investing are: 1- it makes sense for good investments to be counter-intuitive (because they are the most neglected and they wouldn’t be neglected if they were intuitive) and 2- long-termism is more about the coincidence between the correctness of globally good actions and the correctness of those actions in the long term future.
Any Too Long Didn’t Read—like answer will be very appreciated
1.
1.1.: You might want to have a look at group of positions in metaethics called person affecting views, some of which include future people and some of which don’t. The ones that do often don’t care about increasing/decreasing the number of people in the future, but about improving the lives of future people that will exist anyway. That’s compatible with longtermism—not all longtermism is about extinction risk. (See trajectory change and s-risk.)
1.2.: No, we don’t just care about humans. In fact, I think it’s quite likely that most of the value or disvalue will come from non-human minds. (Though I’m thinking digital minds rather than animals.) But we can’t influence how the future will go if we’re not around, and many x-risk scenarios would be quite bad full stop and not just bad for humans.
1.3.: You might want to have a look at cluelessness (EA forum and GPI website should have links) or the recent 80,000 Hours podcast with Alexander Berger. Predicting the future and how we can influence it is definitely extremely hard, but I don’t think we’re decisively in bad enough of a position where we can—with a good conscience—just throw our hands up and conclude there’s definitely nothing to be done here.
2.
2.1 + 2.2.: Don’t really want to write anything on this right now
2.3.: Definite no. It just argues that trade-offs must be made, and some bads are worse even than current suffering. Or rather: The amount of bad we can avert is greater even than if we focus on current suffering
2.4: Don’t understand what you’re getting at.
3.
3.1.: Can’t parse the question
3.2.: I think many longtermists struggle with this. Michelle Hutchinson wrote a post on the EA forum recently on what still keeps her motivated. You can find it by searching her name ont he EA forum.
3.3.: No. Longtermism per se doesn’t say anything about how much to personally sacrifice. You can believe in longtermism + think that you should give away your last penny and work every waking hour in a job you don’t like. You can not be a longtermist and think you should live a comfortable, expensive life because that’s what’s most sustainable. Some leanings on this question might correlate with whether you’re a longtermist or not, but in principle, this question is orthogonal.
Sorry if the tone is brash. If so, that’s unintentional, and I tend to be really slow otherwise, but I appreciate that you’re thinking about this. (Also, I’m writing this as sleep procrastination, and my guilt is driving my typing speed)
Hi—my intuitions fall in the other direction here, so I’m keen to explain why. Implicit IMOs in front of everything here.
1:
1.1: I have a younger brother. My parents could have stopped at one, and my family would broadly still be happy, but my brother is generally happy and leads a good life. Similarly, if they’d had a third child they probably would have been happy and great too, and I would have loved them. All else being equal I wish that youngest sibling could have existed. IMO these two sentiments aren’t meaningfully distinct.
1.2: We don’t only care about humans. Sure, the argument for making more humans would apply to insects or something as well. However, most of the things that would kill all the humans would also kill everything else, so for me not letting that happen is still much more of a priority.
1.3: True on the specifics, false more generally. I don’t know exactly what the world should look like, but I’m pretty sure people being happy is good, more people being happy is better, and everything being unrecoverably dead is neutral at most.
2:
2.1: If we weren’t potentially about to all die I’d be more willing to think about this, but we have to survive the next century or two first. Whether capitalism makes things better or worse for now depends much more on whether it makes us more or less likely to all die, than on anything else (again, for now).
2.2: I’m pretty sure non-privileged people also want to be alive and happy.
2.3: Possibly, and I’m ok with that. I’d rather live a worse life if it means my grandkids are more likely to survive and have happy ones. Although it’s definitely better for everyone to be happier now, I feel like it doesn’t amount to much if we all die in the next century.
2.4: If I can choose between a surviving but stable society, and a growing one, I would choose the growing one. But both are better than an empty rock, so the priority now is not dying either way.
3:
3.1: I’m pretty sure we’ll continue to want to be alive and happy, so false. People can’t decide what their preferences are, and work to fulfil them, if they don’t exist.
3.2: Our moral intuitions were built for very different-looking societies to where we are today. We like sugar and sex because we were supposed to go for fruit and reproduction; our moral intuitions aren’t hugely different. IMO this is in a similar category to people caring more about saving one child than eight of them.
3.3: No.
I should clarify 3.3. For me, longtermism is partly the acknowledgement of much vaster moral stakes—so long as there are things we can do to help, they’re no less important to do as short-termist interventions. (The usual arguments about it not being helpful to demand too much of people still apply though).
I know this is a really mainstream opinion, but I recently watched a recording of the musical Hamilton and I really liked it.
I think Hamilton (the character, not the historical figure which I know very little about) has many key flaws (most notably selfishness, pride, and misogyny(?)) but also virtues/attitudes that are useful to emulate.
I especially found the Non-stop song(lyrics) highly relatable/aspirational, at least for a subset of EA research that looks more like “reading lots and synthesize many thoughts quickly” and less like “think very deeply about a narrow research topic and come up with a fleeting novel insight.” Especially the refrain:
And
And
I love Hamilton! I wrote my IB Extended Essay on it!
I also really love and relate to Non-Stop but in the obsessive, perfectionist way. I like + appreciate your view on it, which seems quite different in that it is more focused on how Hamilton’s brain works rather than on how hard he works.
Hello fellow Zhang!
Thanks! I don’t see nearly as much perfectionism (like none of the lyrics I can think of talks about rewriting things over and over), but I do think there’s an important element of obsession to Hamilton/Non-Stop, which I relate pretty hard to. Since I generate a lot of my expected impact from writing, and it’s quite hard to predict which of my ideas are the most useful/promising in advance, I do sometimes feel a bunch of internal pressure to write/think faster, produce more, etc, like a bit of a race against the clock to produce as much as I can (while maintaining quality standards).
So in the context of this song, I especially admire people (including the character Hamilton, some journalists, some bloggers, and coworkers) who manage to produce a lot of output without (much) sacrificing quality.
Near term AI risk: simulated social status. Social status is a big downside to playing video games all day / abusing drugs. GPT3 and friends are probably capable of giving you believable simulated social status, far beyond what current video games provide. This could be a big theraputic boon as well as well as a threat to people’s happiness and contribution to society.
I’m (still!!!) thinking about my BA thesis research question and I think my main uncertainty/decision point is what specific policy debate to investigate. I’ve narrowed it down to two so far—hopefully I don’t expand—and really welcome thoughts.
Context: I am examining the relationship between narratives deployed by experts on Twitter and the Biden Administration’s policymaking process re: COVID-19 vaccine diplomacy. Specifically, I want to examine a debate on an issue wherein EA-aligned experts have generally coalesced around one stance.
Motivating questions/insights:
COVID-19 policymakers solicited help from experts
However, the U.S. public’s trust in experts has varied. It may have peaked last year and now be declining
Vaccine diplomacy (along with all health policy) is not solely an issue of ‘following the science’
This is not to say that data or rationality is not important. In fact, I would be extremely interested in investigating whether the combination of scientific evidence + thematic framing is more effective than either alone.
However, that would be an experimental study which is not something I am interested in.
This suggests I might want to investigate the presence of scientific vs thematic elements in expert narratives. Not sure though… It’s not what I’m immediately drawn to
Evidence/science alone is insufficient. Experts need to be able to tell stories/persuade/make a moral or emotional appeal. (Extrapolated from the claim that narratives can be influential in policymaking)
At the very least, experts should make clear that no decision is value-neutral and the specific values they are prioritizing in their recommendation
Now that I think about it, the fact that I’m ‘not sure’ about this re: COVID-19 might mean this would make for a good RQ? Or maybe I’m just not thinking of the relevant literature right now.
The two debates below, including general thoughts
The COVID-19 TRIPS Waiver (waiving IP)
What most excites me about this: The Biden Admin did a strong ‘about-face’ on this and the discourse around this was very rich (involved many actors with strong opinions, and entwines with debates around vaccine sharing etc.).
Main hesitation: I don’t know how to think about experts as an actor here. Should they be considered a coalition, per the Advocacy Coalition Framework? Or should I look at a specific set of aligned expert organizations/individuals? Or should I look at all experts on Twitter?
But ACF emphasizes long-term policymaking and shared beliefs—and it seems like there was no singular expert consensus on whether the TRIPS waiver would be a net good. Now that I think about it, this might be due to a lack of transparency over what is being [morally] prioritized...
But why focus on aligned orgs/individuals? How can I justify that? How generalizable is that even?
But if I include all experts, including experts who might have other avenues to policy influence (e.g. big think tanks or former officials), then why not also examine non-expert narratives?
Specifically, the rationale behind examine Twitter is that it provides a highly-accessible advocacy platform to people who do not otherwise have much visibility/leverage
Also, looking at a wide range of Tweets helps get a sense of the general narrative
Delaying child vaccinations (viz. the WHO’s recommendation)
What most excites me about this: There is an explicit non-epistemic debate here (prioritizing children domestically vs the global poor), and that is what I care the most about. There still remains a scientific/epistemic component, too: “Are children safe without vaccines?”
Additionally, there is an added controversial non-epistemic element of anti-maskers
Main hesitation: But the Biden administration hasn’t really ‘made a policy’ on this. So what policy process would I be examining?
This also straddles the line between domestic and international, in that the debate is primarily about picking between the two (in contrast to the first debate), which could be tricky
*edited for clarity—was in a rush when I posted!
These both seem like great options! Of the two, I think the first has more to play with as there is a pretty clear delineation between the epistemic vs. moral elements of the second, whereas I think debates about the first have those all jumbled up and it’s thus more interesting/valuable to untangle them. I don’t totally understand your hesitation so I’m afraid I can’t offer much insight there, but with respect to long-term policymaking/shared beliefs, it does seem like the fault lines mapped onto fairly clear pro-free-market vs. pro-redistributive ideologies that drew the types of advocates one would have predicted given that divide.
*edit 3: After reading more on Epistemic Communities, I think I’m back where I started.
*edit 4: I am questioning, now, whether I need a framework of how experts influence policymaking at all … Maybe I should conceptualize my actors more broadly but narrow the topic to, say, the use of evidence in narratives?
I really appreciate your response, Ian! I think it makes sense that the more convoluted status of the first debate would make it a more valuable question to investigate.
My hesitation was not worded accessibly or clearly—it was too grounded in the specific frameworks I’m struggling to apply—so let me reword: it doesn’t seem accurate to claim that there was one expert consensus (i.e. primarily pro-/anti-waiver). Given that, I am not sure a) how to break down the category of ‘expert’ - although you provide one suggestion, which is helpful—and b) how strongly I can justify focusing on experts, given that there isn’t a clear divide between “what experts think” and “what non-experts think.”
Non-TL;DR:
My main concern with investigating the debate around the TRIPS waiver is that there doesn’t seem to be a clear expert consensus. I’m not even sure there’s a clear EA-aligned consensus, although the few EAs I saw speak on this (e.g. Rob Wiblin) seemed to favor donating over waiving IP (which seems like a common argument from Europe). Given that, I question
the validity of investigating ‘expert narratives’ because ‘experts’ didn’t really agree there
However, I don’t know if it would be in/valid (per the theories I want to draw from, e.g. Advocacy Coalition Framework (ACF) or Epistemic Communities), so that would be one of my next steps.
This particular description worries me: “Advocacy coalitions are all those defined by political actors who share certain ideas and who coordinate among themselves in a functional way to suggest specific issues to the government and influence in the decision-making process.”
This would be subverted by your suggestion, though, as I note in point 3!
the validity of investigating expert narratives specifically instead of the general public—if experts didn’t coalesce around a specific stance, what’s my justification for investigating them specifically instead of getting a sense of the public generally? ACF explicitly notes that “common belief systems bind members of a coalition together.” Given that the pro-/anti-waiver coalitions are defined by common beliefs held by both experts and non-experts (e.g. pro-free-market), how can I justify exclusively focusing on experts?
This is probably not a valid concern, now that I think about it. After all, my thesis hinges upon the idea that experts help inform policymakers + policymaking, so it makes sense to focus on their narratives rather than looking at the public as a whole...
However, it seems like focusing exclusively on two expert groups is valid at least within the Epistemic Community framework, so perhaps this would work if it turns out that certain kinds of experts advocated for the same stance.
whom I should focus on—without being able to lump all experts together, how should I break them down?
Perhaps I could subdivide experts into coalitions—e.g. experts for the waiver and experts against the waiver? (This is akin to the fault lines you mention)
I still feel kind of iffy about investigating experts specifically here, instead of the general public, particularly because I could use the same coalitional divide (pro-/anti-waiver)
Or should I focus on EA-aligned experts specifically?
But I don’t know how to justify this… It doesn’t seem like the smartest research practice
Suggestion: use an expert lens, but make the division you’re looking at [experts connected to/with influence in the Biden administration] vs. [“outside” experts].
Rationale: The Biden administration thinks of and presents itself to the public as technocratic and guided by science, but as with any administration politics and access play a role as well. As you noted, the Biden administration did a clear about-face on this despite a lack of a clear consensus from experts in the public sphere. So why did that happen, and what role did expert influence play in driving it? Put another way, which experts was the administration listening to, and what does that suggest for how experts might be able to make change during the Biden administration’s tenure?
Hmm! Yes, that’s interesting—and aligns with the fact that many different policy influencers weighed in, ranging from former to current policymakers. Thank you very much for this!
I think something I’m worried about is how I can conceptualize [inside experts] vs. [outside experts] … It seems like a potentially arbitrary divide and/or a very complex undertaking given the lack of transparency into the policy process (i.e. who actually wields influence and access to Biden and Katherine Tai, on this specific issue?).
It also complicates the investigation by adding in the element of access as a factor, rather than purely thinking about narrative strategies—and I very much want to focus on narratives. On one hand, I think that could be interesting—e.g. looking at narrative strategies across levels of access. On the other, I’m uncertain that looking at narrative strategies would add much compared to just analyzing the stances of actors within the sphere of influence.
What do you think of this alternate RQ: “How did pro/anti-waiver coalitions use evidence in their narratives?”
Moves away from the focus on experts but still gets to the scientific/epistemic component.
(I’m also wondering whether I am being overly concerned with theoretically justifying things!)
I think I would agree with this. It seems like you’re trying to demonstrate your knowledge of a particular framework or set of frameworks through this exercise and you’re letting that constrain your choices a lot. Maybe that will be a good choice if you’re definitely going into academia as a political scientist after this, but otherwise, I would structure the approach around how research happens most naturally in the real world, which is that you have a research question that would have concrete practical value if it were answered, and then you set out to answer it using whatever combination of theories and methods makes sense for the question.
Thanks! I’ll take a break from thinking about the theory—ironically, I am fairly confident I don’t want to go into academia.
Again, appreciate your thoughts on this. Hope I’ll hear from you again if I post another Shortform about my thesis!
[This is an excerpt from a longer post I’m writing]
Suppose someone’s utility function is
U = f(C) + D
Where U is what they’re optimizing, C is their personal consumption, f is their selfish welfare as a function of consumption (log is a classic choice for f), and D is their amount of donations.
Suppose that they have diminishing utility wrt (“with respect to”) consumption (that is, df(C)/dC is strictly monotonically decreasing). Their marginal utility wrt donations is a constant, and their marginal utility wrt consumption is a decreasing function. There has to be some level of consumption where they are indifferent between donating a marginal dollar and consuming it. Below this level of consumption, they’ll prefer consuming dollars to donating them, and so they will always consume them. And above it, they’ll prefer donating dollars to consuming them, and so will always donate them. And this is why the GWWC pledge asks you to input the C such that dF(C)/d(C) is 1, and you pledge to donate everything above it and nothing below it.
This is clearly not what happens. Why? I can think of a few reasons.
The above is what you get if the selfish and altruistic parts of you “negotiate” once, before you find out how high your salary is going to be. If instead, you negotiate every year to spend some fair share of your resources on altruistic and selfish resources, you get something like what we see.
People aren’t scope sensitive about donations, and so donations also have diminishing marginal returns (because small ones are disproportionately good at making people think you’re good).
When you’re already donating a lot, other EAs will be less likely to hold consumption against you (perhaps because they want to incentivize rich and altruistic people to hang out in EA without feeling judged for only donating 90% of their $10M annual expenditure or whatever).
When you’re high income, expensive time-money tradeoffs like business class flights start looking better. And it’s often pretty hard to tell which purchases are time-money tradeoffs vs selfish consumption, and if your time is valuable enough, it’s not worth very much time to try to distinguish between these two categories.
Early-career people want to donate in order to set themselves up for a habit of donating later (and in order to signal altruism to their peers, which might be rational on both a community and individual level).
As you get more successful, your peers will be wealthier, and this will push you towards higher consumption. (You can think of this as just an expense that happens as a result of being more successful.)
I think that it seems potentially pretty suboptimal to have different levels of consumption at different times in your life. Like, suppose you’re going to have a $60k salary one year and a $100k salary the next. It would be better from both an altruistic and selfish perspective to concentrate your donations in the year you’ll be wealthier; it seems kind of unfortunate if people are unable to make these internal trades.
EDIT: Maybe a clearer way of saying my main point here: Suppose you’re a person who likes being altruistic and likes consuming things. Suppose you don’t know how much money you’re going to make next year. You’ll be better off in expectation from both a selfish and altruistic perspective if you decide in advance how much you’re going to consume, and donate however much you have above that. Doing anything else than this is Pareto worse.
The GWWC pledge is akin to a flat tax, as opposed to a progressive tax—which gives you a higher tax rate when you earn more.
I agree that there are some arguments in favour of “progressive donations”.
One consideration is that extremely high “donation rates”—e.g. donating 100% of your income above a certain amount—may affect incentives to earn more adversely, depending on your motivations. But in a progressive donation rate system with a more moderate maximum donation rate that would probably not be as much of a problem.
Wait the standard GWWC pledge is a 10% of your income, presumably based on cultural norms like tithing which in themselves might reflect an implicit understanding that (if we assume log utility) a constant fraction of consumption is equally costly to any individual, so made for coordination rather than single-player reasons.
Yeah but this pledge is kind of weird for an altruist to actually follow, instead of donating more above the 10%. (Unless you think that almost everyone believes that most of the reason for them to do the GWWC pledge is to enforce the norm, and this causes them to donate 10%, which is more than they’d otherwise donate.)
I thought you were making an empirical claim with the quoted sentence, not a normative claim.
Ah, fair.
[epistemic status: I’m like 80% sure I’m right here. Will probably post as a main post if no-one points out big holes in this argument, and people seem to think I phrased my points comprehensibly. Feel free to leave comments on the google doc here if that’s easier.]
I think a lot of EAs are pretty confused about Shapley values and what they can do for you. In particular Shapley values are basically irrelevant to problems related to coordination between a bunch of people who all have the same values. I want to talk about why.
So Shapley values are a solution to the following problem. You have a bunch of people who can work on a project together, and the project is going to end up making some total amount of profit, and you have to decide how to split the profit between the people who worked on the project. This is just a negotiation problem.
One of the classic examples here is: you have a factory owner and a bunch of people who work in the factory. No money is made by this factory unless there’s both a factory there and people who can work in the factory, and some total amount of profit is made by selling all the things that came out of the factory. But how should the profit be split between the owner and the factory workers? The Shapley value is the most natural and mathematically nice way of deciding on how much of the profit everyone gets to keep, based only on knowing how much profit would be produced given different subsets of the people who might work together, and ignoring all other facts about the situation.
Let’s talk about why I don’t think it’s usually relevant. The coordination problem EAs are usually interested in is: Suppose we have a bunch of people, and we get to choose which of them take which roles or provide what funds to what organizations. How should these people make the decision of what to do?
As I said, the input to the Shapley value is the coalition value function, which, for every subset of the people you have, tells you how much total value would be produced in the case where just that subset tried to work together.
But if you already have this coalition value function, you’ve already solved the coordination problem and there’s no reason to actually calculate the Shapley value! If you know how much total value would be produced if everyone worked together, in realistic situations you must also know an optimal allocation of everyone’s effort. And so everyone can just do what that optimal allocation recommended.
Another way of phrasing this is that step 1 of calculating the Shapley value is to answer the question “what should everyone do” as well as a bunch of other questions of the form “what should everyone do, conditioned on only this subset of EAs existing”. But once you’ve done step 1, there’s no reason to go on to step 2.
A related claim is that the Shapley value is no better than any other solution to the bargaining problem. For example, instead of allocating credit according to the Shapley value, we could allocate credit according to the rule “we give everyone just barely enough credit that it’s worth it for them to participate in the globally optimal plan instead of doing something worse, and then all the leftover credit gets allocated to Buck”, and this would always produce the same real-life decisions as the Shapley value.
--
So I’ve been talking here about what you could call global Shapley values, where we consider every action of everyone in the whole world. And our measure of profit or value produced is how good the whole world actually ends up being. And you might have thought that you could apply Shapley values in a more local sense. You could imagine saying “let’s just think about the value that will be produced by this particular project and try to figure out how to divide the impact among the people who are working on this project”. But any Shapley values that are calculated in that way are either going to make you do the wrong thing sometimes, or rely on solving the same global optimization problem as we were solving before.
Let’s talk first about how the purely local Shapley values sometimes lead to you making the wrong decision. Suppose that some project that requires two people in order to do and will produce $10,000 worth of value if they cooperate on it. By symmetry, the Shapley value for each of them will be $5,000.
Now let’s suppose that one of them has an opportunity cost where they could have made $6,000 doing something else. Clearly, the two people should still do the $10,000 project instead of the $6,000 project. And so if they just made decisions based on the “local Shapley value”, they’d end up not doing the project. And that would end up making things overall worse. The moral of the story here is that the coalition profit function is measured in terms of opportunity cost, which you can’t calculate without reasoning globally. So in the case where one of the people involved had this $6,000 other thing they could have done with their time, the amount of total profit generated from the project is now actually only $4,000. Probably the best way of thinking about this is that you had to pay a $6,000 base salary to the person who could have made $6,000 doing something else. And then you split the $4k profit equally. And so one person ends up getting $8k and the other one ends up getting $2k.
--
I think a lot of EAs are hoping that you can use Shapley values to get around a lot of these problems related to coordination and figuring out counterfactual impact and all this stuff. And I think you just basically can’t at all.
I think Shapley values are more likely to be relevant to cases where people have different values, because in this case you have more like a normal negotiation problem, but even here, I think people overstate their relevance. Shapley values are just a descriptive claim about what might happen in the world rather than a normative claim about what should happen. In particular, they assume that everyone has equal bargaining power to start with which doesn’t seem particularly true.
I think the main way that Shapley values are relevant to coordination between people with different values is that they’re kind of like a Schelling fair way of allocating stuff. Maybe you want to feel cooperative with other people and maybe you don’t want to spend a lot of time going back and forth about how much everyone has to pay, and Shapley values are maybe a nice, fair solution to this. I haven’t thought this through properly yet.
In conclusion, Shapley values are AFAICT not relevant to figuring out how to coordinate between people who have the same goals.
This seems correct
This misses some considerations around cost-efficiency/prioritization. If you look at your distorted “Buck values”, you come away that Buck is super cost-effective; responsible for a large fraction of the optimal plan using just one salary. If we didn’t have a mechanistic understanding of why that was, trying to get more Buck would become an EA cause area.
In contrast, if credit was allocated according to Shapley values, we could look at the groups whose Shapley value is the highest, and try to see if they can be scaled.
The section about “purely local” Shapley values might be pointing to something, but I don’t quite know what it is, because the example is just Shapley values but missing a term? I don’t know. You also say “by symmetry...”, and then break that symmetry by saying that one of the parts would have been able to create $6,000 in value and the other $0. Needs a crisper example.
Re: coordination between people who have different values using SVs, I have some stuff here, but looking back the writting seems too corny.
Lastly, to some extent, Shapley values are a reaction to people calculating their impact as their counterfactual impact. This leads to double/triple counting impact for some organizations/opportunities, but not others, which makes comparison between them more tricky. Shapley values solve that by allocating impact such that it sums to the total impact & other nice properties. Then someone like OpenPhilanthropy or some EA fund can come and see which groups have the highest Shapley value (perhaps highest Shapley value per unit of money/ressources) and then try to replicate them/scale them. People might also make better decisions if they compare Shapley instead of counterfactual values (because Shapley values mostly give a more accurate impression of the impact of a position.)
So I see the benefits of Shapley values as fixing some common mistakes arising from using counterfactual values. This would make impact accounting slightly better, and coordination slightly better to the extent it relies on impact accounting for prioritization (which tbh might not be much.)
I’m not sure to what extent I agree with the claim that people are overhyping/misunderstanding Shapley values. It seems a plausible.
Notes on: A Sequence Against Strong Longtermism
Summary for myself. Note: Pretty stream-of-thought.
Proving too much
The set of all possible futures is infinite which somehow breaks some important assumptions longtermists are apparently making.
Somehow this fails to actually bother me
...the methodological error of equating made up numbers with real data
This seems like a cheap/unjustified shot. In the world where we can calculate the expected values, it would seems fine to compare (wide, uncertain) speculative interventions with harcore GiveWell data (note that the next step would probably be to get more information, not to stop donating to GiveWell charities)
Sometimes, expected utility is undefined (Pasadena game)
The Pasadena game also fails to bother me, because the series hasn’t (yet) showed that longtermism bets are “Pasadena-like”
(Also, note that you can use stochastic dominance to solve many expected value paradoxes, e.g, to decide between two universes with infinite expected value, or with undefined expected value.)
...mention of E.T. Jaynes
Yeah, I’m also a fan of E.T. Jaynes, and I think that this is a cheap shot, not an argument.
Subject, Object, Instrument
This section seems confused/bad. In particular, there is a switch from “credences are subjective” to “we should somehow change our credences if this is useful”. No, if one’s best guess is that “the future is vast in size”, then considering that one can change one’s opinions to better attain goals doesn’t make it stop being one’s best guess
Overall: The core of this section seems to be that expected values are sometimes undefined. I agree, but this doesn’t deter me from trying to do the most good by seeking more speculative/longtermist interventions. I can use stochastic dominance when expected utility fails me.
The post also takes issue with the following paragraph from The Case For Strong Longtermism:
I agree that the paragraph just intuitively looks pretty bad, so I looked at the context:
Yeah, this is in the context of a thought experiment. I’d still do this with distributions rather than with point estimates, but ok.
The Credence Assumption
Ok, so the OP wants to argue that expected value theory breaks ⇒ the tool is not useful ⇒ we should abandon credences ⇒ longtermism somehow fails.
But I think that “My best guess is that I can do more good with more speculative interventions” is fairly robust to that line of criticism; it doesn’t stop being my best guess just because credences are subjective.
E.g., if my best guess is that ALLFED does “more good” (e.g., more lives saved in expectation) than GiveWell charities, pointing out that actually the expected value is undefined (maybe the future contains both infinite amounts of flourishing and suffering) doesn’t necessarily change my conclusion if I still think that donating to ALLFED is stochastically dominant.
Cox Theorem requires that probabilities be real numbers
The OP doesn’t buy that. Sure, a piano is not going to drop on his head, but he might e.g., make worse decisions on account of being overconfident because he has not been keeping track of his (numerical) predictions and thus suffers from more hindsight bias than someone who kept track.
But what alternative do we have?
One can use e.g., upper and lower bounds on probabilities instead of real valued numbers: Sure, I do that. Longtermism still doesn’t break.
Some thought experiment which looks like The Whispering Earring.
Instead of relying on explicit expected value calculations, we should rely on evolutionary approaches
The Poverty of Longtermism
“In 1957, Karl Popper proved it is impossible to predict the future of humanity, but scholars at the Future of Humanity Institute insist on trying anyway”
Come on
Yeah, this is just fairly bad
Lesson of the 20th Century
This is going to be an ad-hitlerium, isn’t it
No, an ad-failures of communism
At this point, I stopped reading.
I’m becoming concerned that the title “EA-aligned organisation” is doing more harm than good. Obviously it’s pointing at something real and you can expect your colleagues to be familiar with certain concepts, but there’s no barrier to calling yourself an EA-aligned organisation, and in my view some are low or even negative impact. The fact that people can say “I do ops at an EA org” and be warmly greeted as high status even if they could do much more good outside EA rubs me the wrong way. If people talked about working at a “high-impact organisation” instead, that would push community incentives in a good way I think.
I have exactly the opposite intuition (which is why I’ve been using the term “EA-aligned organization” throughout my writing for CEA and probably making it more popular in the process).
“EA-aligned organization” isn’t supposed to mean “high-impact organization”. It’s supposed to mean “organization which has some connection to the EA community through its staff, or being connected to EA funding networks, etc.”
This is a useful concept because it’s legible in a way impact often isn’t. It’s easy to tell whether an org has a grant from EA Funds/Open Phil, and while this doesn’t guarantee their impact, it does stand in for “some people at the community vouch for their doing interesting work related to EA goals”.
I really don’t like the term “high-impact organization” because it does the same sneaky work as “effective altruist” (another term I dislike). You’re defining yourself as being “good” without anyone getting a chance to push back, and in many cases, there’s no obvious way to check whether you’re telling the truth.
Consider questions like these:
Is Amazon a high-impact organization? (80K lists jobs at Amazon on their job board, so… maybe? I guess certain jobs at Amazon are “high-impact”, but which ones? Only the ones 80K posts?)
Is MIRI a high-impact organization? (God knows how much digital ink has been spilled on this one)
Is SCI a high-impact organization?
Is the Sunrise Movement a high-impact organization?
It seems like there’s an important difference between MIRI and SCI on the one hand, Amazon and Sunrise on the other. The first two have a long history of getting support, funding, and interest from people in the EA movement; they’ve given talks at EA Global. This doesn’t necessarily make them most impactful than Amazon and Sunrise, but it does mean that working at one of those orgs puts you in the category of “working at an org endorsed by a bunch of people with common EA values”.
*****
I hope this doesn’t happen very often; I’d prefer that we greet everyone with equal warmth and sincere interest in their work, as long as the work is interesting. Working at an EA-aligned org really shouldn’t add much signaling info to the fact that someone has chosen to come to your EA meetup or whatever.
That said, I sympathize with theoretical objections like “how am I supposed to know whether someone would do more good in some other job?” and “I’m genuinely more interested in hearing about someone’s work helping to run [insert org] than I would if they worked in finance or something, because I’m familiar with that org and I think it does cool stuff”.
Terms that seem to have some of the good properties of “EA-aligned” without running into the “assuming your own virtue” problem:
“Longtermist” (obviously not synonymous with “EA-aligned”, but it accurately describes a subset of orgs within the movement)
“Impact-driven” or something like that (indicating a focus on impact without insisting that the focus has led to more impact)
“High-potential” or “promising” (indicating that they’re pursuing a cause area that looks good by standard EA lights, without trying to assume success — still a bit self-promotional, though)
Actually referring to the literal work being done, e.g. “Malaria prevention org”, “Alternative protein company”
...but when you get at the question of what links together orgs that work on malaria, alternative proteins, and longtermist research, I think “EA-aligned” is a more accurate and helpful descriptor than “high-impact”.
Oh, I would have thought it’s the other way around—sometimes people don’t want to be known as EA-aligned because that can have negative connotations (being too focused on numbers, being judgmental of “what’s worthy”, slightly cult-like etc). I think “high-impact organisation” may be a good idea as well.
EA Highschool Outreach Org (see Catherine’s and Buck’s posts, my comment on EA teachers)
Running a literal school would be awesome, but seems too consuming of time and organisational resources to do right now.Assuming we did want to do that eventually, what could be a suitable smaller step? Founding an organisation with vetted staff, working full-time on promoting analytical and altruistic thinking to high-schoolers—professionalising in this way increases the safety and reputability of these programs. Its activities should be targeted to top schools, and could include, in increasing order of duration:
One-off outreach talks at top schools
Summer programs in more countries, and in more subjects, and with more of an altruistic bent (i.e. variations on SPARC and Eurosparc)
Recurring classes in things like philosophy, econ, and EA. Teaching by visitors could be arranged by liaising to school teachers, similarly to how external teachers are brought in for chess classes.
After-school, or weekend, programs for interested students
I’m not confident this would go well, given the various reports from Catherine’s recap and Buck’s further theorising. But targeting the right students, and bringing the right speakers, gives it a chance of success. If you get to (3-4), all is going well, and the number of interested teachers and students are rising, it would be very natural for the org to scale into a school proper.
Below, I briefly discuss some motivating reasons, as I see them, to foster more interdisciplinary thought in EA. This includes ways EA’s current set of research topics might have emerged for suboptimal reasons.
More EA-relevant interdisciplinary research : why?
The ocean of knowledge is vast. But the knowledge commonly referenced within EA and longtermism represents only a tiny fraction of this ocean.
I argue that EA’s knowledge tradition is skewed for reasons including but not-limited-to the epistemic merit of those bodies of knowledge. There are good reasons for EA to focus in certain areas:
Direct relevance (e.g. if you’re trying to do good, it seems clearly relevant to look into philosophy a bunch; if you’re trying to do good effectively, it seems clearly relevant to look into economics (among others) a bunch; if you came to think that existential risks are a big deal, it is clearly relevant to look into bioengineering, international relations, etc. a bunch; etc.)
Evidence of epistemic merit (e.g. physics has more evidence for epistemic merit than psychology, which in return has more evidence for epistemic merit than astrology; in other words, beliefs gathered from different fields are are likely to pay more/less rent, or are likely to be more/less explanatory virtuous)
However, some of the reasons we’ve ended up with our current foci may not be as good:
Founder effects
The, in parts arbitrary, way academic disciplines have been carved up
Inferential distances between knowledge traditions that hamper the free diffusion of knowledge between disciplines and schools of thought
Having a skewed knowledge basis is problematic. There is a significant likelihood that we are missing out on insights or perspectives that might critically advance our undertaking. We don’t know what we don’t know. We have all the reasons to expect that we have blindspots.
***
I am interested in the potential value and challenges of interdisciplinary research.
Neglectedness
(Academic) incentives make it harder for transdisciplinary thought to flourish, resulting in what I expect to be an undersupply thereof. One way of thinking about why we would see an undersupply of interdisciplinry thought is in terms of “market inefficiencies”. For one, individual actors are incentivised (because it’s less risky) to work on topics that are already recognised as interesting by the community (“exploitation”), as opposed to venturing into new bodies of knowledge that might or might not prove insightful (“exploration”). What is “already recognized as valuable by the community”, however, will only in part be determined by epistemic considerations, and in another part be shaped by path-dependencies.
For two, “markets” are insufficiently liquid and thus tend to fail where we cannot easily specify what we want. I’d argue that this is the case for DS/ET work. This is generally true for intellectual work, but is likely even more true for DS/ET work due to the relatively siloed structure of academia that adds additional “transaction costs” to attempts of communicating across disciplinary boundaries.
One way to reduce these inefficiencies is by improving the interfaces between the disciplines. “Domain scanning” and “episetmic translation” are precisely about creating such interfaces. Their purpose is to identify knowledge that is concretely relevant to a given target domain and make that knolwege accessible to thinkers entrenched in the “vocabulary” of that target domain. A useful interface between political philosophy and computer science, for example, might require a mathematical formalization of central ideas such as justice.
Challenges
At the same time, doing interdisciplinary well is callenging. For example, interdisciplinary research can only be as valuable as a researcher’s ability to identify knowledge relevant to their target domain; or as a research community’s quality assurance/error correction mechanisms. Phenomena like citogenesis or motivatiogensis are examples of manifestations of these difficulties.
There have been various attempts at overcoming these incentive barriers, for example the Santa Fe Institute whose organizational structure completely disregards scientific disciplines; -ARPAs have a similar flavour; the field of cybernetics which proposed an inherently transdisciplinary view on regulatory systems; or the recent surge in the literature on “mental models” (e.g. here or here).
A closer inspection of such examples—in how far they were successful and how they went about it—might bear some interesting insights. I don’t have the capacity to properly puruse such case studies in the near future, but it’s definteily something on my list of potentially promising (side) projects.
If readers are aware of other examples of innovative approaches trying to solve this problem that might make for insightful case studies, I’d love to hear them.
I think RAND is a good case study for interdisciplinary approaches to problem solving, though I’m biased. The key there, as in industry and most places other than academia, but unlike Santa Fe and the ARPAs, is a focus on solving concrete specific problems regardless of the tools used.
Also, big +1 to cybernetics, which is an interesting case study for 2 reasons, first because of what worked, and second because of how it was supplanted / coopted into narrow disciplines, and largely fizzled out as its own thing.
Our World in Data has created two great posts this year, highlighting how the often proposed dichotomy between economic growth & sustainability is false.
In The economies that are home to the poorest billions of people need to grow if we want global poverty to decline substantially, Max Roser points out that given our current wealth,
Which is far below what we’d think of as the poverty line in developed countries. This means that mere redistribution of what we have is insufficient—we’d all end up poor and unable to continue developing much further because we’re too occupied with mere survival. In How much economic growth is necessary to reduce global poverty substantially?, he writes:
in the section Is it possible to achieve both, a reduction of humanity’s negative impact on the environment and a reduction of global poverty?, he adds:
So for discussions on how to approach individual “consumption” or policymaking around it, we could start a list of specific products to avoid. Would somebody be up for compiling this? It would be a resource I’d link to quite regularly. You can apparently just extract them from the 13 links Max Roser put just above the paragraph cited above. It would make for a great, short and crisp EA Forum post, too.
Making community-building grants more attractive
An organiser from Stanford EA asked me today how community building grants could be made more attractive. I have two reactions:
Specialised career pathways. To the extent that this can be done without compromising effectiveness, community-builders should be allowed to build field-specialisations, rather than just geographic ones. Currently, community-builders might hope to work at general outreach orgs like CEA and 80k. But general orgs will only offer so many jobs. Casting the net a bit wider, many activities of Forethought Foundation, SERI, LPP, and FLI are field-specific outreach. If community-builders take on some semi-specialised kinds of work in AI, or policy, or econ, (in connection with these orgs or independently) then this would aid their prospects of working for such orgs or returning to a more mainstream pathway.
“Owning it”. To the extent that community building does not offer a specialised career pathway, the fact that it’s a bold move should be incorporated into the branding. The Thiel Fellowship offers $100k to ~2 dozen students per year, to drop out of their programs to work on a startup that might change the world. Not everyone will like it, but it’s bold, it’s a round, and reasonably-sized number, with a name attached, and a dedicated website. Imagine a “Macaskill fellowship” that offers $100k for a student from a top university to pause their studies and spend one year focusing on promoting prioritisation and long-term thinking—it’d be a more attractive path.
Would it be useful to compile EA-relevant press?
Inspired by me seeing this Vice article on wet-bulb conditions (a seemingly unlikely route for climate change to become an existential risk): Scientists Studying Temperature at Which Humans Spontaneously Die With Increasing Urgency
If so, what/how? I don’t think full-time monitoring makes sense (first rule of comms: do everything with a full comms strategy in mind!) but I wonder if a list or Airtable would still be useful for organizations to pull from or something...
My hope is that people who see EA-relevant press will post it here (even in Shortform!).
I also track a lot of blogs for the EA Newsletter and scan Twitter for any mention of effective altruism, which means I catch a lot of the most directly relevant media. But EA’s domain is the entire world, so no one person will catch everything important. That’s what the Forum is for :-)
I’m not sure whether you’re picturing a project specific to stories about EA or one that covers many other topics. In the case of the former, me and others at CEA know about nearly everything (though we don’t have it in a database; no one ever asks). In the case of the latter, the “database” in question would probably just be… Google? I’m having trouble picturing the scenario where an org needs to pull from a list of articles they wouldn’t find otherwise. (But I’m open to being convinced!)
I think David Nash does something similar with his EA Updates (here is the most recent one). While most of the links are focused on EA Forum and posts by EA/EA-adj orgs, he features occasional links from other venues.
Good flag, thanks!
I enjoyed learning about the Henry Spira award. It is given by the Johns Hopkins School of Public Health to “honor animal activists in the animal welfare, protection, or rights movements who work to achieve progress through dialogue and collaboration.”
The criteria for the award are based on Peter Singer’s summary of the methods Spira used in his own advocacy. Many of them seem like strong guiding principles for EA work in general:
See the bottom of this page for a list of winners (most recently in 2011; I don’t know whether the award is still “active”).
At work so have no mental space to read this carefully right now, but wonder if anyone has thoughts—specifically about whether there’s any EA-relevant content: MIT Predicted in 1972 That Society Will Collapse This Century. New Research Shows We’re on Schedule. (vice.com)
These models predicted growth followed by collapse. The first part has been proven correct, but there is little evidence for the second. Acting like past observations of growth are evidence of future collapse seems like an unusual example of Goodman’s New Riddle of Induction in the wild.
Thank you, so helpful!
To clarify—“little evidence” implies that you think observations of current conditions aligning with model predictions, e.g. “Previous studies that attempted to do this found that the model’s worst-case scenarios accurately reflected real-world developments,” are weak?
Maybe someone should make ~1 Anki card each for lots of EA Wiki entries, then share that Anki deck on the Forum so others can use it?
Specifically, I suggest that someone:
Read/skim many/most/all of the EA Wiki entries in the “Cause Areas” and “Other Concepts” sections
Anki cards based on entries in the other sections (e.g., Organisations) would probably be less useful
Make 1 or more Anki card for many/most of those entries
In many cases, these cards might take forms like “The long reflection refers to… [answer]”
In many other cases, the cards could cover other insights, concepts, questions, etc. raised in the body of the entry
Making such cards seems less worthwhile for cases in which either:
the entry mainly exists as a tag (without itself having much content)
the entry is about a quite well-known thing and doesn’t really say much that’s not well-known in its body (e.g., the International relations tag)
Export the file for the resulting deck and share it on the Forum and maybe elsewhere
Other people can then either use the whole deck or pick and choose which parts of the deck to use (e.g., deleting cards when they come up, if the person feels those cards aren’t relevant to their interests and plans)
I think this could also be done gradually and/or by multiple people, rather than in one big batch by one person. It could also be done for the LessWrong Wiki.
If someone does make this deck, I would very likely use some/many of the cards myself and also promote the deck to a bunch of other people.
(I also currently feel that this would be a sufficiently useful action to have taken that I’d be inclined to reward the person with some token amount of my own money to signal my appreciation / to compensate them for their time / because me saying that now might incentivise them. I’d only do this if the cards the person makes actually seems good. Feel free to contact me if you want to discuss that.)
Turning the EA Wiki into a (huge) Anki deck is on my list of “Someday/Maybe” tasks. I think it might be worth waiting a bit until the Wiki is in a more settled state, but otherwise I’m very much in favor of this idea.
There is an Anki deck for the old LW wiki. It’s poorly formatted and too coarse-grained (one note per article), and some of the content is outdated, but I still find it useful, which suggests to me that a better deck of the EA Wiki would provide considerable value.
Why this might be worthwhile:
The EA community has collected and developed a very large set of ideas that aren’t widely known outside of EA, such that “getting up to speed” can take a similar amount of effort to a decent fraction of a bachelor’s degree
But the community is relatively small and new (compared to e.g. most academic fields), so we have relatively little in the way of textbooks, courses, summaries, etc.
This means it can take a lot of effort and time to get up to speed, lots of EAs have substantial “gaps” in their “EA knowledge”, lots of concepts are misinterpreted or conflated or misapplied, etc.
The EA Wiki is a good step towards having good resources to help people get up to speed
A bunch of research indicates retrieval practice, especially when spaced and interleaved, can improve long-term retention and can also help with things like application of concepts (not just memory)
And Anki provides such spaced, interleaved retrieval practice
I’m being lazy in not explaining the jargon or citing my sources, but you can find some explanation and sources here: Augmenting Long-term Memory
If one person makes an Anki deck based on the EA Wiki entries, it can then be used and/or built on by other people, can be shared with participants in EA Fellowships, etc.
Possible reasons not to do this:
“There’s a lot of stuff it’d be useful for people to know that isn’t on EA Wiki entries. Why not make Anki cards on those things instead? Isn’t this a bit insular?”
I think we can and should do both, rather than one or the other
Same goes for having Anki cards based on EA sources vs Anki cards based on non-EA sources
Personally, I’d guess ~25% of my Anki cards are based on EA sources, ~70% are based on non-EA sources but are about topics I see as important for EA reasons, and 5% are random personal life stuff
See also Suggestion: Make Anki cards, share them as posts, and share key updates
“This seems pretty time-consuming”
I think there are a lot of people in the EA community for whom engaging with the EA Wiki entries to the extent required to make this deck would be worthwhile just for themselves
I also think there are even more people in the EA community for whom using all or a subset of these cards will be worthwhile
(Though there are also of course people for whom these things aren’t true)
“Many of the entries probably won’t actually be that well-suited to Anki cards, or aren’t on very important things”
Agreed
But many will be
The card-maker(s) can skip entries, and the card-users can delete some cards from their own copy of the deck
“This seems like rote learning / indoctrination / stifling creativity / rah rah”
I quite strongly feel that these sorts of concerns about these like Anki cards are often misguided, including this case
I can expand on that if anyone actually does feel worried about this idea for this reason