I graduated from Georgetown University in December, 2021 with degrees in economics, mathematics and a philosophy minor. There, I founded and helped to lead Georgetown Effective Altruism. Over the last few years recent years, I’ve interned at the Department of the Interior, the Federal Deposit Insurance Corporation, and Nonlinear.
2. Beyond freeing pigs and chooks from cages, animal welfare concern leads to absurd conclusions when you start thinking about wild animal suffering or try to quantify insect suffering.
Eh, I agree the conclusions might be counterintuitive and even weird, but disagree pretty strongly that they’re absurd.
Even granting that only freeing mammals from cages is good and worthy, I’m (subjectively, not super rigorously) quite confident that indeed getting chickens and/or pigs out of cages is both more robustly good and ethically more important than any of the GiveWell charities.
4. Global development will probably have positive effects on animal welfare anyway and may even be necessary in a lot of cases (richer countries are generally the ones that adopt better animal welfare rules)
Not impossible, but seems very unlikely and would be suspicious if helping humans happened to also be the best way to help animals. I don’t think it comes very close, in fact, though I’m unsure what the sign is
5. Global development is more broadly appealing than than fish rights and AI safety. Focus on longtermism and fringe animal welfare issues is part of what makes the EA label and community alienating to people.
I agree with the first sentence, which is why I suspect that most of the ethical value from global dev runs through community building/attracting newcomers and optics, and this effect is plausibly pretty big in magnitude. But I think we should have a very high bar for not doing something morally important because some people might think it’s weird or silly, even if some amount of activity optimized for broad appeal is warranted
This seems like a major failure, and FWIW I think you should currently be getting paid to do some sort of longtermism-relevant research (or other knowledge work), and it’s a failure that you’re not. 🙃 indeed lol
Though I should register that I know Harrison IRL so infer whatever biases you should
Maybe you’re already aware of ikejime and have concluded that it can’t be cheaply scaled, but in case you haven’t, check it out.
Yeah, I consider that the best case slaughter method and regret that it seems so labor intensive, but seems like there might be other less bad methods than the current status quo
Is this different from GiveWell because GiveWell doesn’t try to estimate, like, the nth-order effects of AMF? I think I’m convinced by the cluelessness explanation that those would cancel out in expectation so we should be fine with first and maybe second-order effects.
Sure, I think 4th+ order effects are likely impossible to model, but 2nd and maybe even 3rd not so much. I’d bet (though far from certain) you could get a well-identified study for the causal effect of e.g. malaria nets on total life years lived/population/pop growth in a certain geographic region, at least for some period of time
(As I responded on Twitter and hope to turn into a forum post) I think aligning intra-EA status with impact is basically the whole point of EA community-building, so this is very important. I would guess that organizational operations is still too low-status and neglected: we need more people who are willing to set up payroll. (Low confidence, willing to be talked out of this, but it seems like the case to me.)
Strong +1 on this, would be a super interesting and productive post IMO!
Thanks for the correction, that is definitely good news for the fish (albeit slightly bad news for my research judgement lol)!
Although another consideration pointing in the other direction is that larger fish probably have larger brains with more neurons, which may render them more morally relevant
I tend to think you’re right, but don’t think it’s wildly overdetermined—mostly because animal suffering reduction seems more robustly good than does preventing extinction (which I realize is not the sole or explicit goal of longtermism, but is sometimes an intermediate goal)
One concern I have about raising the salience of emotions in EA is that might more frequently turn into prospective incentives (generally bad) instead of retrospective motivation (generally good).
What I mean is that people might (naturally and likely unconsciously) gravitate towards projects and cause areas that inspire positive emotions (say, awe about the future) and away from those that induce negative ones (say, distress about suffering).
Not sure how I missed this, but great post and this seems super important and relatively neglected. In case others think it would be worth coining a term for this specifically, I proposed “p-risk” after “p-zombies,” in a Tweet (might find later) a few months back.
More substantively, though, I think the greater potential concern is false-positives on consciousness, not false negatives. The latter (attributing conscious experience to zombies) would be tragic, but not nearly as tragic as causing astronomical suffering in digital agents that we don’t regard as moral patients because they don’t act or behave like humans or other animals.
In direct violation of the instruction to put ideas in distinct comments, here’s a list of ideas most of which are so underbaked they’re basically raw:
Meta/infrastructure
Buy a hotel/condo/appt building (maybe the Travelodge?) in Berkeley and turn it into an EA Hotel
Offer to buy EAs no-doctor-required blood tests like this one that test for common productivity-hampering issues (e.g. b12 deficiency, anemia, hypothyroidism)
Figure out how to put to good use some greater proportion of the approximately 1 Billion recent college grads who want to work at an “EA org”
This might look like a collective of independent-ish researchers?
(uncertain) I think a super high impact thing that a single high school senior could decide to do is to attend a good state honors college or non-Ivy university without a thriving EA club/scene and make it happen there.
I tentatively think it would be worth, say, turning down Harvard to go to the U of Maryland Honors college and start an EA group there
To incentivize this, make it cool and normal to credibly say that this is what one did
To a first approximation, 0% of students outside of the Ivies and maybe 10 other campuses have heard of EA. The fruit is underground at this point
Animal welfare
(Ok this one is like 60% baked) A moderately-intensive BOTEC says that wild fish slaughter accounts for 250 million years of extreme fish suffering per year from asphyxiation or being gutted alive
The linked BOTEC also includes my conservative, lower bound estimate that this is morally equivalent to ~5B human deaths per year
Idea: idk, like do something about this. Like figure out how to make it cheap and easy to kill fish en masse in a quicker and/or more humanely
This one might turn into an actual forum post
Related: can we just raise (farm) a ton of fish ourselves, but using humane practices, with donations subsidizing the cost difference relative to standard aquaculture
This also might turn into a blog post
Hot takes
There should be way more all-things-considered, direct comparisons between cause areas.
In particular, I don’t think a complete case has been made (even from a total utilitarian, longtermist perspective) that at the current funding margin, it makes sense to spend marginal dollars on longtermist-motivated projects instead of animal welfare projects.
As just one demonstration, the Fish Welfare Initiative says they have a nearly $200k funding gap for 2022
Note (as of 6pm June 24) I may update this comment and/or break parts into their own comments as I recall other ideas I’ve had
From a Twitter thread a few days ago (lightly edited/formatted), with plenty of criticism in the replies there
Probably batshit crazy but also maybe not-terrible megaproject idea: build a nuclear reactor solely/mainly to supply safety-friendly ML orgs with unlimited-ish free electricity to train models
Looks like GPT-3 took something like $5M to train, and this recent 80k episode really drives home that energy cost is a big limiting factor for labs and a reason why only OpenAI/Deepmind are on the cutting edge
In 2017, the smallest active U.S. nuclear reactor generated ~5M MWh. It looks like that amount of energy would sell for $600M
But it only costs about $150M to generate that amount via a nuclear plant. And larger power plants would give a delta of a couple billion per year between cost and price
The theory of change here is that it would allow Redwood/Anthropic/other future orgs to have a decisive advantage over less safety-conscious ones. This is especially relevant if making an aligned AGI requires a lot more compute (and hence electricity) than making a Clippy
Also, I don’t see why it couldn’t be built in some country with more nuclear-friendly policies...and wouldn’t even have to be connected to the grid I’m thinking of a plant connected to a Boeing plant-sized warehouse full of GPUs and nothing else (except maybe an EA hotel lol)
I legit want to know where on the spectrum from “Hi! 👋 nuclear physicist here. This is incredibly stupid, please delete your account” to “huh this warrants a longer+more rigorous EA Forum post” this seems to fall. Thanks!
Central idea: can we train AI to generalize out of distribution?
I’m thinking, for example, of an algorithm like the following:
Train a GPT-like ML system to predict the next word given a string of text only using, say, grade school-level writing (this being one instance of the object level)
Assign the system a meta-level award based on how well it performs (without any additional training) at generalizing; in this case, that is, predicting the next word from more advanced, complex writing (perhaps using many independent tests of this task without updating/learning between each test, and allowing parameters to update only after the meta-level aggregate score is provided)
Note: the easy→hard generalization is not a necessary feature. Generalization could be from fiction→nonfiction writing or internet→native print text, for instance.
After all these independent samples are taken, provide the AI its aggregate or average score as feedback
(Maybe?) repeat all of step I on a whole new set of training and testing texts (e.g., using text from a different natural language like Mandarin)
Repeat this step an arbitrary number of times
For example, using French text, then Korean, then Arabic, etc.
Each time a “how well did you generalize” score is provided (which is given once per natural language in this example), the system should improve at the general task of generalizing from simple human writing to more complex human writing, (hopefully) to the point of being able to perform well at generalizing from simple Hindi (or whatever) text to advanced Hindi prediction even if it had never seen advanced Hindi text before.
^Steps 1-3 constitute the second meta-level of training an AI to generalize, but we can easily treat this process as a single training instance (e.g., rating how well the AI generalizes to Hindi advanced text after having been trained on doing this in 30 other languages) and iterate over and over again. I think this would look like:
Running the analogs of steps 1-4 on generalizing from
(a) simple text to advanced text in many languages
(b) easy opponents to hard ones across many games,
(c) photo generation of common or general objects (“car”) to rare/complex/specific ones (“interior of a 2006 Honda Accord VP”), across many classes of object
And (hopefully) the system would eventually be able to generalize from simple Python code training data to advanced coding tasks even though it had never seen any coding at all before this.
And, of course, we can keep on adding piling layers on.
A few notes
I think the following is one way of phrasing what I hope might happen with method: we are using RL to teach an ML system how to do ML in such a way that it sacrifices some in-distribution predictive power for the ability to use its “knowledge” more generally without doing anything that seems dumb to us.
Of course, there are intrinsic limits to any system’s ability to generalize. The system in question can only generalize using knowledge X if X exists as information in the object-level training provided to it.
This limits what we should expect of the system.
For example, I am almost certain that even an arbitrarily smart system will not be able to generate coherent Mandarin text from English training data, because the meaning of Mandarin characters doesn’t exist as “latent knowledge” in even a perfect understanding of English.
Anyone here know Python?
My hands-on experience with ML extends to linear regression in R and not an inch more, so I’m probably not the best person to test this theory out. I’ve heard some LWers know a bit of Python, though.
If that’s you, I’d be fascinated and thankful to see if you can implement this idea using whatever data and structure you think would work best, and would be happy to collaborate in whatever capacity I can.
Appendix: a few brief comments (from someone with much more domain knowledge than me) and responses (from me):
Comment
Is this just the same as training it on this more complex task (but only doing one big update at the end, rather than doing lots of small updates)?
Response (which may help to clarify why I believe the idea might work)
I don’t think so, because the parameters don’t change/update/improve between each of those independent tests. Like GPT-3 in some sense has a “memory” of reading Romeo and Juliet, but that’s only because its parameters updated as a result of seeing the text.
But also I think my conception depends on the system having “layers” of parameters corresponding to each layer of training.
So train on simple English-->only “Simple English word generation” parameters are allowed to change...but then you tell it how well it did at generalizing out of distribution, and now only its “meta level 1 generalization” parameters are allowed to change.
Then you do the whole thing again but with German text, and its “Meta level 1 generalization” parameters are allowed to change again using SGD or whatever. If this works, it will be the reason why it can do well at advanced Hindi text without ever having read advanced Hindi.
Treat this whole process as the object level, and then it updates/improves “meta level 2 generalization” parameters.
Comment:
This looks vaguely like curriculum learning, which apparently doesn’t really work in LLMs https://arxiv.org/abs/2108.02170, I think a similar experiment would be like train on simple+advanced text for English, French, Mandarin etc, but only simple Hindi, and then see if it can do complex Hindi.
Response
I think that’s a pretty different thing because there are no meta level parameters. Seems like fundamentally just a flavor of normal RL
Or do pretraining with English, French, Mandarin, and Hindi, but only do fine tuning with English, French, Mandarin, and see if it can then do the tasks it was fine tuned for in Hindi.
My prediction: it learns to generalize a bit (the scores on the novel Hindi tasks are higher than if there was no fine tuning with the other languages) but worse than the other languages generalize. As the models are scaled up, this ‘generalization gap’ gets smaller.
Seems like this might depend on the relative scaling of different meta level parameters (which I described above)?
Like for example whenever you scale the # of object level params by a factor of 2, you have to scale the number of nth meta level parameters by 2^(n+1).
And just to clarify, there’s a huge black box in my mind between “inflammation decreases” and “depressive symptoms decrease.” I have no idea what the mechanisms are there!
Thanks, updating slightly in the direction of “not effective” but not a ton, mostly because I have a pretty high prior on anything that causally reduces systemic inflammation to be effective for depression. It would be quite surprising to me if at least one of the following is true:
Reducing abnormally high systemic inflammation in depressed people doesn’t improve symptoms
EPA supplementation among depressed people doesn’t on average decrease levels of systemic inflammation
EPA supplementation causes depressive symptoms in enough people to offset the effect described in (1) and (2)
Thanks for pointing this out. I should—and plan to—look into this more by checking out the individual studies used in the Cochrane review. Worth noting that the review was of antioxidant supplementation (vits A, E, C, selenium) in particular rather than a multivitamin per se.
I wouldn’t be shocked if any physiological harm can be traced to Vit A and E supplementation in excess of, say, 300% of RDA. It is a little concerning that the one I recommended contains 170% and 130% respectively.
Also, speaking for myself alone, I think I’d be willing to trade a small increase in mortality for a decreased likelihood of some inadequacy dampening my subjective wellbeing or productivity. As caloric restriction studies suggest, mortality and subjective wellbeing can move in the same direction.
Thanks for representing the global dev camp!
Eh, I agree the conclusions might be counterintuitive and even weird, but disagree pretty strongly that they’re absurd.
Even granting that only freeing mammals from cages is good and worthy, I’m (subjectively, not super rigorously) quite confident that indeed getting chickens and/or pigs out of cages is both more robustly good and ethically more important than any of the GiveWell charities.
Not impossible, but seems very unlikely and would be suspicious if helping humans happened to also be the best way to help animals. I don’t think it comes very close, in fact, though I’m unsure what the sign is
I agree with the first sentence, which is why I suspect that most of the ethical value from global dev runs through community building/attracting newcomers and optics, and this effect is plausibly pretty big in magnitude. But I think we should have a very high bar for not doing something morally important because some people might think it’s weird or silly, even if some amount of activity optimized for broad appeal is warranted