I’m a doctor working towards the dream that every human will have access to high quality healthcare. I’m a medic and director of OneDay Health, which has launched 53 simple but comprehensive nurse-led health centers in remote rural Ugandan Villages. A huge thanks to the EA Cambridge student community in 2018 for helping me realise that I could do more good by focusing on providing healthcare in remote places.
NickLaing
Karma: 14,483
I think this piece makes good arguments abuot AI welfare in general, but I just can’t see the politics of AI moral patienthood being that messy. The current trend is likely to continue—politicians on both sides will deny AI moral patienthood and continue to enact laws that don’t allow it.
I don’t actually think that is unreasonable given the both near-term and long-term risks to humanity from AI. I think people will only consider the welfare of AI AFTER their own welfare and future feels safe and secure from AI. This is a very strong human instinct. Regular people want to feel safe and protected, with only the most altruistic really caring about people (or animals) outside their own communities. I don’t think its fair or reasonable to expect 90% of people to care about AI welfare until AI is safe and aligned.
I think its great that a bunch of people are researching AI welfare, and I agree with a decent chunk of money going into AI welfare research, but I doubt it will have much utility in the real world for a long time. I think we’re look at a pretty non-messy bipartisan political path which shuts down any consideration of AI welfare in the coming years.
Thanks I think your overall point is fair.
My problem is that AI editing still brings about a regression to the mean, and still means we lose some of the human’s voice. I agree it could make writing technically “better” in one sense, but that’s not my problem with AI writing. Whether AI makes writing better or not isn’t important to me when it comes to this question of pre-labelling writing. My problem is that voice-authenticity and writing diversity is lost when AI is involved.
IMO AI makes writing sound more sam-ey and for me at least less interesting. Others might be more excited to read AI edited/written work. That’s great!
I’m OK with AI detectors overcalling it somewhat, if someone actually is using a lot of AI in the editing process. This doesn’t mean they have done anything “wrong”, its just tells me in advance that AI is heavily involved.
If someone is using AI in the writing process, that’s fine it’s no sin. I just think everyone should be able to know and then we can choose in advance how or whether we should read it. Pangram has been a game-changer and I appreciate it a lot!
I think some donors will appreciate this approach and we have some evidence of that as RP have already raised a bunch of money for their allocated fund. But yes Moskovitz is crazy rational about giving, far more than I would be.
Fantastic article great to hear some more meta takes on this.
Within global health, even before this money comes in, solid calculated cost effectiveness calculations are becoming less and less relevant. Unfortunately I don’t think the “next rung down” on the givewell cost effectiveness bar will get most of the cash. Instead coefficient giving and others will likely give a lot of money to speculative “hits based” approaches. For example they just started a 170 million dollar fund for government technical support, which even the fund head themselves admits has very little evidence that it does much, and is very difficult to even make a cost effectiveness analysis for (although givewell did to their credit).
I suspect much of the new cash will go to those, both because and tech people like Risk and it’s often easier to throw bigger amounts of money at speculative projects. I’ve asked a few times for coefficient giving to do a review of all their hits based giving you date—it has been long enough now that we would have seen much of what worked and what didn’t.
I think ability of organizations to actually use money well might be more of a barrier than than. There’s a world (which I hope for) where the best animal welfare orgs are at least saturated with money. I caution against new orgs trying to start by spending too much, to and was really impressed with the reasonableness of the recent wild animal welfare ask.
This makes a huge amount of sense to me. Especially after the Open AI hack of Australia today! Middle powers are probably the most motivated to make sure AI goes well—and they have less to slose. You can get so much more leverage by using a rudder than trying to row a ship against the current.
The idea that the only 2 countries that matter with AI are China and USA seems absurd to me. Of course they are the most important, but they are also dependent on the economy of the rest of the world. #
Nice initiative!
Nice one I didn’t realise it could be so good! Love this analysis.
The big issues with Aspirin might not be the evidence, but the reluctance of us clinicians here and fears of how patients might respond if something goes wrong when they are on. This stuff might just not be a big deal in many contexts though.
1. Safety of NSAIDS in general durign pregnancy and Aspirin specifically still hasn’t been cleared after all these years. “Do no harm” as a principle can stand stronger than a benefit/harm analysis espeically in something like pregnancy.
2. Bleeding. We’re all terrified of this. And if someone gets a miscarriage on aspirin (even unrelated) they are going to ask if its related. Same with PPH.
I think getting this past clinicians might be harder than the program. I actually think this program wouldn’t be that hard to do and could be implemented for WAY less than $32 per woman. I’m low-key excited about this actually
I think if this was lodged squarely into national guidelines that would make it easier. Or maybe it is already and us doctors are still ignoring it :(.
On your original quicktake I don’t completely agree. On this one I do. Now is the time for PR Comms people to be workign overtime, and ideally share some of their strategy here (at least on a high level). There should be at least some co-ordination on how to respond here from EA leadership and I agree that is sorely lacking—even if the strategy is not to engage.
Now is the time for EA PR and marketing departments to shine, with AI safety overton window opening up and attacks coming in.
Absolutely love the specialist/generalist 2x2. I’ve thought about that a bit and tried to express it at times, and your framing certainly makes that easier!
Sample size one here, when I joined the Cambridge EA group I remember hugely respecting that these people who thought chickens were so important they only ate vegan at their gatherings. It attracted me and didn’t put me off at all.
But maybe that kind of signaling was more important to me, a religious person who often values personal integrity above impact.
I agree with your great communication strategy about why they serve that food, super helpful. Maybe EAGs should have a line in their application form about that as well?
Also maybe the vibe has changed since 7 years ago...
Personally I love long oats the same. I think 9⁄10 times I can sniff LLM written stuff a mile off then I don’t read it. I also appreciate that the forum flags it for me to
I don’t think an LLM can yet sound not like an LLM
I think METRs checks are meaningful and maybe the best we have at the moment. They are also like you say compromised and the conflict of interests are immense with huge personal overlap between the labs and safety orgs, and funding streams too.
Geyges seems largely correct, but if we can’t convince governments to regulate properly its better METR is in there doing it. After all METR exposed more about the hugging face hack than Open AI did on its own.
I don’t think a framing of “these connections are completely unacceptable” is helpful given these problems. I think “compromised and far from ideal” is a better framing. Its better to do something than do nothing. I agree its best if government installed internal auditors like they do for banks, but that ain’t happening any time soon.
Companies like Anthropic and Open AI are selfish animals by nature. They may have moments where good humans inside might do the right thing, but fundamentally they thirst for profit and growth. After IPO this will only get worse. We should never expect a company to regulate itself or its industry. Self regulation for harmful companies is a terrible idea and never works.
How can ANthropic “create” a genuinely independent auditor? this seems impossible, almost and Oxymoron.
I love this “Are there other justice movements that ignored the fact that they were justice movements?” and would perhaps even take it further to...
”Are there other justice movements that won without being justice movements.”
The moral underbelly, and even the passion in that underbelly which keeps me going personally is indeed a thirst for justice and liberation (in my case for humans in poverty not animals suffering but same diff). EA helps me enact this chance through finding cost-effective ways to fix the problem. Even among more “intellectual” EAs, I think a decent number of us are still (emotionally at least) motivated towards those ends rather than a utilitarian suffering minimisation mindset.
I 100% agree with every sentiment here, I just haven’t seen this as a huge problem. I feel like the mods have done a pretty good job recently of removing AI slop. I only consider posts when they get past about 10 Karma too which might be saving me tho...
I don’t think “The EA name and brand” is that tricky to reason about really. Also what we “wish” the public could do is unfortunately not reality. I think there are a number of reasonable takes and good arguments for and against focusing on brand, which have been discussed much on the forum. These 2 might be the most obvious
1) “Don’t worry about PR it hurts epistemics and decision making. We should openly discuss any topic even if the public find it revolting or outside the overton window. This will help us truly do what has the highest EV, and achieve the most good—public perception be darned”
2) ” EA being in good standing towards most people helps EA’s do more good. PR is important for attracting people to the movement and makes community building easier. It helps It also helps donors outside EA feel more warm towards EA ideas and spend their money more cost-effectively. If we are not pariahs, or even better seen as good by most people, that will help us redirect more thought, talent and resouces towards the movement.
And everything in between, and probably some orthoganal takes too. There are arguments for and against these. But I don’t think its tricky to reason about as you state here.
From my perspective when we call ourselves EAs, or EA adjacents or even just vaguely part of the community, then we do have responsibility to others and the movement for the consequences of actions like hiring someone like Caroline Ellison. You’ve taken some of that responsibility which is great by posting here and having a discussion.
That said my take (FWIW) I understand your reasons for hiring here, but its probably not wise. I do have a STRONG bent towards second chances though, but I feel it might have been wiser for PR reasons at least to put her in a position which had nothing at all to do with finance. Also you are opening yourself up to a lot of risk if anything bad happens with Manifund finances, even if she had nothing to do it.
[deleted user] I think he’s arguing against my idea not yours. I agree 100% I think there’s no reason at all not to do the retrospective design (AI could do most of it anyway), I’m just saying its still a pretty low level of research really. I agree a big improvenet though.
Yep that’s the one!
I don’t think it would really help vs other interventions much (unless they are also doing rigorous research, which I doubt).
Because you’ll get very specifid information about the differences, It might help you understand in which ways the program helps and doesn’t as well so you could adjust programming as well. Like you might find it helps people get more jobs but not better quality jobs than those who don’t do the program. Or you might find less people enter capabilities rather than safety etc.
I think you misunderstood me here. I wasn’t suggesting you research your selection criteria, I was suggesting a basic Randomised controlled trial design possibliity for one intake to assess the impact of the program. An RCT woudln’t help you much (if at all) with selection criteria). The candidates who were selected and not selected would then be followed over time to see if the MATS program made a difference to their trajectories.
”every MATS cohort counts” might be true, but if the difference in fellows isn’t that big why not randomise one round at least? Not sure how big your cohorts are though. If they are too small this wouldn’t work at all.
Gotcha. Have no idea how good cG are at impact analysis—I’m not sure I’ve seen one publicised by them, I think they have decided against publicising things in general. I don’t think that AI safety people are necessarily that good at impact analysis though—very few would have trained at all in that field. Being good at impact analysis and understanding what is best for AI safety are completely different and underated things.
I get that people have different ideas about what is a “good” outcome, but CoGi at least must have a criteria if they are doing an assessment at all. The criteria could be very complex and nuanced, that’s fine—as long the criteria is clear before you start the study.
I would argue if you don’t know what is a good outcome, there’s probably no point in doing any work at all.
“I don’t think that an independent analysis is necessarily better than the default.” What do you mean by this exactly?
I agree many human practises do this, and we need it to be functional.
I just hate the extreme of it we get with AI assisted writing. I don’t have a perfect answer to… “this argument that AI editing regresses to a mean more a reflection of a regression toward a shared communication mechanism or a regression of creative elements”
It just makes everything sound more the-same and less human to me. This is so clear to me and so many others I don’t think it needs any kind of specific test, although tests have been done.
I agree we can leave it to the author to decide how to write, but the reader/consumer needs agency to decide what and how to read as well. I want to know in advance before I read something how much AI is involved so I can decide whether to read it slowly and enjoy, skim for content or just not read at all. I think labelling systems like for food safety or “made in XXXX country” usually bring more benefit than harm and AI assisted writing is no exception.