Ebenezer Dukakis

Karma: 1,170

Ebenezer Dukakis Feb 3, 2025, 4:53 AM
15 points
6 ∶ 1
in reply to: Jobst Heitzig (vodle.it)’s comment on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing
As an American, I think this is actually a really good point. The only reason to even consider racing with China to build AI is if we believe the outcome is better if America “wins”. Seems worth keeping an eye on the political situation in the US, e.g. for stuff like PEPFAR cancellation, to check if the US “winning” is actually better from a humanitarian perspective. Hopefully lab leaders are smart enough to realize this.

Ebenezer Dukakis Jan 30, 2025, 10:48 AM
11 points
7 ∶ 1
in reply to: huw’s comment on: Capitalism and the Very Long Term

This arrests EAs/longtermists from drawing on centuries of knowledge & movement-building

The track record of anticapitalist advocacy seems quite poor. See this free book: Socialism: The failed idea that never dies

If you’re doing anticapitalist advocacy for EA reasons, I think you need a really clear understanding of why such advocacy has caused so much misery in the past, and how your advocacy will avoid those traps.

I’d say what’s needed is not anticapitalist advocacy, so much as small-scale prototyping of alternative economic systems that have strong theoretical arguments for how they will align incentives better, and scale way past Dunbar’s number.

You don’t need a full replacement for capitalism to test ideas and see results. For example, central planning often fails due to corruption. A well-designed alternative system will probably need a solution for corruption. And such a solution could be usefully applied to an ordinary capitalist democracy.

I concede that AI companies are behaving in a harmful way, but I doubt that anticapitalist advocacy is a particularly tractable way to address that, at least in the short term.

Ebenezer Dukakis Jan 30, 2025, 10:30 AM
3 points
0 ∶ 0
in reply to: Holly Elmore ⏸️ 🔸’s comment on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing

In that case your strategy is just feeding the labs talent and poisoning the ability of their circles to oppose them.

It seems like your model only has such influence going one way. The lab worker will influence their friends, but not the other way around. I think two-way influence is a more accurate model.

Another option is to ask your friends to monitor you so you don’t get ideologically captured, and hold an intervention if it seems appropriate.

Ebenezer Dukakis Jan 30, 2025, 8:58 AM
22 points
3 ∶ 1
in reply to: Holly Elmore ⏸️ 🔸’s comment on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing

I see no meaningful path to impact on safety working as an AI lab researcher

This is a very strong statement. I’m not following technical alignment research that closely, but my general sense is that exciting work is being done. I just wrote this comment advertising a line of research which strikes me as particularly promising.

I noticed the other day that the people who are particularly grim about AI alignment also don’t seem to be engaging much with contemporary technical alignment research. That missing intersection seems suspicious. I’m interested in any counterexamples that come to mind.

My subjective sense is there’s a good chance we lose because all the necessary insights to build aligned AI were lying around, they just didn’t get sufficiently developed or implemented. This seems especially true for techniques like gradient routing which would need to be baked in to a big, expensive training run.

(I’m also interested in arguments for why unlearning won’t work. I’ve thought about this a fair amount, and it seems to me that sufficiently good unlearning kind of just oneshots AI safety, as elaborated in the comment I linked.)

Ebenezer Dukakis Jan 30, 2025, 8:33 AM
2 points
0 ∶ 0
on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing
With regard to Deepseek, it seems to me that the success of mixture-of-experts could be considered an update towards methods like gradient routing. If you could localize specific kinds of knowledge to specific experts in a reliable way, you could dynamically toggle off / ablate experts with unnecessary dangerous knowledge. E.g. toggle off experts knowledgeable in human psychology so the AI doesn’t manipulate you.

I like this approach because if you get it working well, it’s a general tool that could help address a lot of different catastrophe stories in a way that seems pretty robust. E.g. to mitigate a malicious AI from gaining root access to its datacenter, ablate knowledge of the OS it’s running on. To mitigate sophisticated cooperation between AIs that are supposed to be monitoring one another, ablate knowledge of game theory. Etc. (The broader point is that unlearning seems very generally useful. But the “Expand-Route-Ablate” style approach from the gradient routing paper strikes me as a particularly promising, and could harmonize well with MoE.)

I think a good research goal would be to try to eventually replicate Deepseek’s work, except with highly interpretable experts. The idea is to produce a “high-assurance” model which can be ablated so undesired behaviors, like deception, are virtually impossible to jailbreak out of it (since the weights that perform the behavior are inaccessible). I think the gradient routing paper is a good start. To achieve sufficient safety we’ll need new methods that are more robust and easier to deploy, which should probably be prototyped on toy problems first.
What links here?
- Ebenezer Dukakis's comment on The Game Board has been Flipped: Now is a good time to rethink what you’re doing by LintzA (Jan 30, 2025, 8:58 AM; 22 points)

Ebenezer Dukakis Jan 30, 2025, 8:08 AM
9 points
1 ∶ 1
on: The Game Board has been Flipped: Now is a good time to rethink what you’re doing
it’s possible Trump is better placed to negotiate a binding treaty with China (similar to the idea that ‘only Nixon could go to China’), even if it’s not clear he’ll want to do so.

Some related points:
- Trump seems like someone who likes winning, but hates losing, and also likes making deals. If Deepseek increases the probability that the US will lose, that makes it more attractive to negotiate an end to the race. This seems true from both a Trump-psychology perspective, and a rational-game-theorist perspective.
- Elon Musk seems to have a good relationship with Chinese leadership.
- Releasing open-source AI seems like more of a way to prevent someone else from winning than a way to win yourself.
- Chinese leadership may be concerned about x-risks.
I suppose one possible approach would be to try to get some sort of back-channel dialogue going, to start drafting a treaty which can be invoked if political momentum appears.

Ebenezer Dukakis Jan 28, 2025, 10:09 PM
3 points
1 ∶ 0
in reply to: Kelsey Piper’s comment on: The Upcoming PEPFAR Cut Will Kill Millions, Many of Them Children
That’s a good point about patient privacy. On X, you suggested that PEPFAR has had a sizable macro impact on AIDS in Africa. Maybe Africans who are old enough to remember could talk about what AIDS was like in Africa before and after PEPFAR, as a way to illustrate that macro impact without violating the privacy of any individual patients.

Of course, insofar as individual patients are willing to speak about their experiences, possibly with some light anonymization such as looking away from the camera, that seems really good to me too.

A couple other thoughts:
- If access to ARV drugs is interrupted, I imagine that could lead to drug-resistant HIV strains, same way your doctor tells you to take your antibiotics consistently.
- Having a big population of immunocompromised people in Africa seems bad in terms of mutation speed for future pandemics. Here’s a paper I just found on Google, about COVID variants which are thought to have arisen from immunocompromised patients.

Ebenezer Dukakis Jan 28, 2025, 8:46 AM
4 points
1 ∶ 2
in reply to: nemeryxu’s comment on: nemeryxu’s Quick takes
I wonder if the response will be seen as more credible if it’s driven by Africans?

Insofar as there’s a motivation behind cutting foreign aid, I suspect that skepticism of the NGO sector is playing a role. I can imagine Trump supporters thinking: This program supposedly helps millions of poor Africans, yet the primary voices advocating for it online are rich-world progressives. Seems fishy.

Ebenezer Dukakis Jan 28, 2025, 8:27 AM
21 points
4 ∶ 0
in reply to: NickLaing’s comment on: The Upcoming PEPFAR Cut Will Kill Millions, Many of Them Children

I’ve received a bunch of Whatsapp messages from Ugandan friends who are very very worried about what this might mean not only for patients (the main issue), but also for jobs and the livelihood of many local NGOS.

I think it could be pretty effective for your Ugandan friends to record videos interviewing people who have been helped by the program, and post the videos online.

A lot of skepticism of foreign aid is driven by the fear that the aid is being captured somewhere along the way, either here in the US or in the target country. Hearing directly from the aid recipients helps address that fear.

As an American, it feels like good things the US does are taken for granted, and the US will be criticized relentlessly by people in other countries no matter what we do. So I wouldn’t suggest scolding the US for withdrawing funds. And in any case, rewards usually work better than punishments for modifying behavior. I think saying thanks, and talking about what it would mean for the program to continue, would be a lot more powerful than scolding. Even for Americans who aren’t particularly altruistic, this would provide tangible evidence that the program is improving America’s reputation.

One mental model I have is that the US is suffering from something akin to EA burnout, and the solution to both “US burnout” and “EA burnout” is a stronger culture of gratitude. In principle, I could imagine that a really good response to this PEPFAR situation actually ends up motivating the US to fund additional effective aid programs.
What links here?
- Ebenezer Dukakis's comment on nemeryxu’s Quick takes by nemeryxu (Jan 28, 2025, 8:46 AM; 4 points)

Ebenezer Dukakis Jan 26, 2025, 3:11 AM
3 points
0 ∶ 0
in reply to: MichaelDickens’s comment on: MichaelDickens’s Shortform
I sold all my NVIDIA stock, since their moat looks weak to me:

https://forum.effectivealtruism.org/posts/rBx9RmJdBJgHkjL4j/will-openai-s-o3-reduce-nvidia-s-moat

I think your reasoning is generally correct. Another argument: If you believe things look sufficiently grim under short timelines, maybe you should invest under the assumption that a recession, or something else, will pop the AI bubble and gives us longer timelines.

Ebenezer Dukakis Jan 24, 2025, 7:54 PM
2 points
0 ∶ 0
in reply to: Milan Weibel🔹’s comment on: titotal’s Quick takes

“AI bad” gets misgeneralized into skepticism about current and future AI capabilities.

This point is interesting. I almost wonder if it’s better to not argue against this. If we argue against it, maybe the left gets attached to this position, and becomes slower to update even as unemployment increases.

Ebenezer Dukakis Jan 24, 2025, 7:49 PM
6 points
2 ∶ 0
in reply to: titotal’s comment on: titotal’s Quick takes
One could also reason that the left can be counted on to be anti-AI going forwards, and the objective for EA should be to foster anti-AI forces on the right. The H1B split shows that tech leaders don’t have ideological control over the right. Sam Altman and Elon Musk don’t get along either. In fact, Sam Altman doesn’t seem to have a strong popular constituency in either party at this point.

Ebenezer Dukakis Jan 24, 2025, 7:28 PM
3 points
3 ∶ 1
in reply to: Cullen 🔸’s comment on: Cullen_OKeefe’s Shortform
Maybe once Sam says it, Dario kinda has to say it to stay competitive for funding?

Ebenezer Dukakis Jan 19, 2025, 11:32 PM
10 points
1 ∶ 1
in reply to: Jeff Kaufman 🔸’s comment on: What are we doing about the EA Forum? (Jan 2025)
I guess my view is that low-effort sharing of new ideas is not rewarded/engaged with, and is undersupplied relative to what’s optimal. When I have a new idea that seems like it could have a big impact, and I quickly post it on the EA Forum, it’s much more out of a sense of duty than a sense of excitement.

I see the “Let’s think about slowing down AI” post as support for my position. We shouldn’t have required a 45-minute read by a senior community member before slowing down AI was taken seriously as a possibility. In a world where Katja is too busy to make that effort-post, I think there’s a chance that EA takes far longer to consider a pivot.

I think high-effort contributions for new ideas aren’t necessarily optimal. I put a fair amount of effort into this post, which looks like a big waste of time in retrospect. In this case, Linch explicitly told me I put too much effort in, and his short comment to that effect got more upvotes than my effort-comment.

The upvote-snowballing mechanism means a small difference in the rate of vote-gaining creates a large difference in attention. It seems like the top 10% of vote-gainers tend to be high-effort, non-controversial stuff, which ideally has some sort of prestige affiliation (“10 pages defending a thesis that is obvious”). So we see a lot of that stuff on the Forum. People copy what’s upvoted, and the Forum ends up rather bland.

The “10 pages defending a thesis that is obvious”-type posts tend to be ones where the author tries to anticipate and respond to every possible criticism or deficiency. An author can’t necessarily predict in advance which axes readers will want more effort on. If you try to predict and address them all, I suspect that contributes to bland writing. It’d be better to address those issues through dialogue than monologue.

Maybe there is a good place for low-effort sharing of new EA ideas elsewhere, and I’m not aware of it. But there does seem to be a suspicious lack of new ideas on the Forum—especially given how fast the world is changing, which should naturally produce new ideas for how to do good. I think lack of new ideas on the Forum is evidence that there’s no good place for low-effort sharing of new EA ideas elsewhere. If EA was good at low-effort sharing of new ideas elsewhere, I would expect some of those ideas to trickle into high-effort “new idea” posts on the Forum, and I’m largely not seeing those posts.

Ebenezer Dukakis Jan 19, 2025, 9:15 PM
15 points
1 ∶ 0
in reply to: Jeff Kaufman 🔸’s comment on: What are we doing about the EA Forum? (Jan 2025)
Compare this comment with this comment. The second comment was posted about a week later. I’m glad the second comment was posted, and I’m glad the suggestion was packaged in a way that made it appealing to Forum users, but I do notice that the packaging seemed to matter a fair amount (“rigor” flavoring). I don’t think overconfidence was a major factor here.

I post a fair number of offbeat ideas like this, and they don’t generally receive much attention, which leaves me feeling demoralized. And then I wrote the grandparent comment, where I got downvoted/disagreevoted for asking if there’s a better place to post offbeat ideas, which is even more demoralizing. Like, what do you guys want from me?

I notice your framing

I would guess that weird EA ideas that were appropriately caveated would do reasonably well here

basically acknowledges that this is a hypothetical, and new ideas mostly don’t get posted here. I feel like I have a pretty good understanding of why! Again, maybe this is an OK or even desirable state of affairs. But I wish we could at least acknowledge it.

Ebenezer Dukakis Jan 15, 2025, 11:37 PM
1 point
0 ∶ 0
in reply to: MichaelDickens’s comment on: Ethics of Whey Protein: Net Negative or Justifiable for Environmental Vegans?
If it’s literally a waste product, I don’t see why that should matter. The costs should just be for processing and transportation, which would also apply to ordinary milk? Yet in my area whey protein powder costs about 30x as much as milk per unit weight.

Ebenezer Dukakis Jan 15, 2025, 11:11 PM
6 points
1 ∶ 2
in reply to: Brad West🔸’s comment on: What are we doing about the EA Forum? (Jan 2025)
It seems that users on this forum want to upvote content which is rigorous and true. So the way to gain karma is to write 10 pages defending a thesis that is obvious, as opposed to writing half a page introducing a thesis that is revolutionary.

That’s not necessarily a problem. I feel like the EA Forum wants to be the end of an idea pipeline, the last step where ideas get final scrutiny, and are stamped for epistemic rigor and community consensus. Yet the beginning and middle of the pipeline sort of don’t seem to exist? At least not on the public internet.

Anyway, let me know if there’s a better place to post my weird EA ideas. My general sense is that weird ideas are not super welcome here.

Ebenezer Dukakis Jan 12, 2025, 7:35 PM
11 points
2 ∶ 0
on: Ethics of Whey Protein: Net Negative or Justifiable for Environmental Vegans?
If what you’re saying is true, why is whey protein so expensive?

Ebenezer Dukakis Jan 9, 2025, 12:49 AM
1 point
0 ∶ 0
in reply to: Nathan Young’s comment on: Nathan Young’s Shortform
Do you know of anything which directly addresses the argument I made here?

My vague impression is that the optimism I see is based on outside-view type forecasts, and people have mostly not taken inside views. I haven’t thought much about bird flu recently though.

Ebenezer Dukakis Jan 3, 2025, 9:46 PM
1 point
0 ∶ 0
in reply to: huw’s comment on: Will OpenAI’s o3 reduce NVidia’s moat?
Thanks.

Regarding decreased demand for GPUs—I agree Deepseek v3 is evidence that demand will decrease. But o3 seems like evidence that demand will increase, since we’ll be doing more test-time compute. Unclear which evidence is more compelling.