Good idea! I just read through the study preprint last night and was puzzling over how you could get a good grasp on how much of an impact the wikipedia edits are making.
Another option would be to see if there was a model that took in ~half of the PAW edits in it’s training corpus, and test it on all the claims. Then you could see roughly how much of a difference having the info on wikipedia specifically made. Might still need to control for prominence outside wikipedia though.
VeryJerry
Karma: 141
https://homosabiens.substack.com/p/optimization-is-often-immoral I’m not sure I’d go quite as far as Duncan in saying it’s fully immoral, but I do think he has a great point. Also note that even LLM’s usually don’t use a greedy selection of the next token. In practice, a greedy algorithm doesn’t give as good of results as adding a little temperature. Set a bar, then pick randomly from everyone above it. Or something besides letting millions of dollars sit in the bank.
Based on everything you’ve looked into, what odds do you put on fish sentience?
I would add, I’m not convinced that all “unconscious” states in humans are equal. I’ve heard reports of people who “wake up” during surgery, but were still unable to move. Note that I haven’t looked into this in any depth and it may very well be an urban legend. It could be that some anesthetics knock out ability to move and long term memory, such that it appears as though the person is not feeling pain and they can’t report pain they don’t remember, although they may have felt it at the time.
This is similar to my experience trying to get into AI x Animals specifically. I work full time at a standard software engineering job, and I decided to start getting involved in the space.
I started out trying some writing, but I really struggled with it and felt like I didn’t have much to say that other people weren’t saying better already. I still feel that way.
Then I reached out to Compassioned Aligned Machine Learning (compassionml.com) to see if I could volunteer for them. They got me involved, and they could still use help (and funding!) but I really struggle to contribute much while working full time. I’m really proud of what I’ve done so far, it’s helped me grow a lot and I’m excited to keep helping! But I wish I could properly focus on it.
I’ve tried to get into various funded tracks in this field, but they all tend to get filled up by better candidates. I am glad they are getting the best people, but it’s discouraging that there isn’t much room. I want to do more, but I’m unfortunately not really willing to do the huge amount of work needed to get a paid position in the field or start my own organization for it. I would happily take a big pay cut to be able to focus on it full time though. Huge props to the people who do like the folks running CAML!!
Maybe we’re just in the weird phase waiting for Anthropic to ipo and flood the field with funding?
VeryJerry’s Quick takes
An AI that is to us as we are to other species does not go well for us. It needs to have better values!
Yes, definitely in favor of more pro-animal politics! I think there are a lot of high impact things the movement can be doing. It’s worth focusing on the ones we know, and researching new avenues and theories of change too.
Thanks for bringing this up and going in-depth on the evidence. I’ve always felt uneasy with the cage free campaigns and shrimp stunning.
Part of that is definitely my more “abolitionist” viewpoint. I hear about those and think “wait how much money are we putting towards exploiting animals in a bit less bad way?” Of course an improvement is an improvement so if they are actually better then that is better, and there is benefit to getting momentum. It’s easier to take a mile once you’ve gotten an inch, so to speak. But if we don’t even actually know if the increment we’re pushing is better, that’s a problem that deserves to have alarm bells sounding.
That said, I don’t really know what would be better tactically. Lately I’m pretty AI-pilled, thinking that if we can make AGI aligned with all sentient beings then that will be a huge benefit. But today’s labs aren’t gonna release a model that only ever suggests vegan recipes, and the future of AI is highly speculative.
Sometimes I wonder if we should just start doing door-to-door vegan advocacy, like the jehovahs witnesses and normy politics and deep canvassing (and street epistemology and “smart politics” https://www.lesswrong.com/posts/D2GrrrrfipHWPJSHh/book-review-how-minds-change ).
Assert, don’t describe. Linguistic Features that shift LLM reasoning about animal welfare
Great article, very well said!
I would especially emphasize the human/non-human dynamics. In a sense, we are the TAI of the animal kingdom, and the results have been disastrous for every species we happened to decide to factory farm. Do unto others as you would have others do unto you comes to mind.
Also, for a more artistic and entertaining take on the veil of ignorance, check out https://www.astralcodexten.com/p/being-john-rawls
To expand, posts like these give me https://knowyourmeme.com/memes/someone-who-is-good-at-the-economy-please-help-me vibes.
“Someone who is good at finding grantmakers please help, life on my planet might get wiped out if we don’t fund saving it”
“Hire people who are maybe not quite as elite-of-the-elite-tier talent as you hoped, since they can still do a ton of good”
“no”
when a strong candidate turns down a CG offer, the result is often not “a slightly-less-good grantmaker,” it’s just one fewer grantmaker. We routinely close rounds with fewer hires than we’d planned for.
Have you considered leaving rounds open longer and/or hiring slightly-less-good grantmakers instead of closing the round?
Very well said, it’s so sad what happens to so many animals and like you said the list goes on and on. And the funny thing is, we are animals. If AI goes well for animals, that implies it goes well for us.
But we really don’t want speciesist AI. If we have AI that has a moral circle based on species membership, or based on certain capabilities like intelligence, what happens when it gets to the point where we are no longer intelligent enough to qualify, or the only species that’s morally relevant is the AI species?
For what it’s worth, I’m surprised that people who think abortion is murder aren’t doing more illegal stuff to destroy clinics.
Also for what it’s worth, I think factory farming is so bad that it’s by far the greatest injustice caused by humans in history. Justifiable wars have been fought for orders of magnitude less important things.
If factory farming ends, animal products will be prohibitively expensive for most people, so you’ll end up mostly vegan anyways. Why not start now?
What are gene drives? Is that like genetically modifying insects to not feel pain and then introducing them into the environment so the pain-free genes eventually replace all the standard genes? (Or whatever other genetic feature, not necessarily pain related)
Hmm, that opens up a lot of interesting conversation threads. I actually think some goals will be easier to align ai towards than others, for example we’ve aligned some ai to winning at chess and now they’re better than any human. Obviously that kind of goal is much simpler than any values framework that would be worth aligning agi too, but I think sentientist values would be easier to instill than “human values” (although not in the case of LLMs, I think they’re already basically “aligned” with human values and we now need to shift them towards caring more about all sentient beings). And on top of that, I think sentientist values will care enough about us and our values that a sentientist agi would “go well” for us.
But I’m not even close to an expert, so that’s all very tentative speculation.
Have you seen the moral ambition folks?
I think you could know how to encode preferences into ASI without knowing whether it’s aligned. At that point it’s like a genie, you may well be able encode preferences into it but you might encode bad preferences that you didn’t realize would be bad.