Unable to work. Was community director of EA Netherlands, had to quit due to long covid. Everything written since 2021 with considerable brain fog, and bad at maintaining discussions since.
I have a background in philosophy, risk analysis, and moral psychology. I also did some x-risk research. Currently most worried about AI and US democracy. (Regarding the latter, I’m highly ranked on Manifold).
Did you get any responses?
Thanks!
Tbh, I think the Overton window isn’t so important. AI is changing fast, and somebody needs to push the Overton window. Hinton says LLMs are conscious and still gets taken seriously.. I would really like to see policy work on this soon!
Yes I think the correct takeaway is that it’s high variance. I didn’t mean to say that all government jobs would be like that. :)
I like this!
Relevant context for those unaware: supposedly, Good Ventures (and by extension OpenPhil) has recently decided to pull out of funding artificial sentience.
Can you give some examples of topics that qualify and some that don’t qualify as “EA priorities”?
I feel like for the purpose of getting the debate started, the vague question is fine. For the purpose of measuring agreement/disagreement and actually directly debating the statement, it’s potentially problematic. Does EA as a whole have priorities? How much of a priority should it be?
Why do you not recommend policy and/or advocacy for taking the possibility of AI sentience seriously? I’m pretty concerned that even if AI safety gets taken seriously by society, the likely frame is Humanity vs. AI: humans controlling AI and not caring about the possibility of AI sentience. This is a very timely topic!
Also, interesting factoid: Geoffrey Hinton believes current AI systems are already conscious! (Seems very overconfident to me)
Oh, and I was incredibly disappointed to read Good Ventures is apparently not going to fund work on digital sentience?
He seems to believe:
a relatively “low” x-risk from Superintelligence (his 2023 post gives 5%)
that who controls large capital pre-Superintelligence can still use this capital post-ASI (he states so in the Dwarkesh interview)
I think (2) is doing too much work in your argument. Current LLMs can barely pursue goals, and need a lot of prompting. They tend to not have ongoing internal reasoning. So, old arguments would not expect these systems to resist shutdown.
What I meant by “simple input-output system” was that there’s little iterative reasoning, let alone constant reasoning & observation.
I’ve heard an EA say that working for the government was depressing and mostly taught them what not to do. Government can be really dysfunctional.
I don’t think LLMs really tell us much if anything about agents’ incentives & survival instinct, etc. They’re simply input-output systems?
I do agree that “they won’t understand what we mean” seems very unlikely now though.
Ah I see, thanks.
That seems like.. a bad policy?
Did you reach out to Anthropic for comments/answers?
See here for a list of the participants and the standardized list of questions I asked.
Link seems to be missing
Yes, that comment was made by Lukas Gloor here, when I asked what people thought Sam Altman’s beliefs are.
This is the year on year change from 2025 too 2029: −300 −500 −2.400 −2.400 −2.400
Given the instability of any coalition recently, but especially one with the far right, there’s a chance this government won’t make it far enough to implement the big cuts.
In 2024, the Netherlands is expected to spend 3.6 billion on foreign aid. So it’s a 67% cut. This is supposedly the biggest cut in our country’s history.
Common prevalence estimates are often wrong. Example: snakebites and my experience reading Long Covid literature.
Both institutions like the WHO and academic literature appear to be incentivized to exaggerate. I think the Global Burden of Disease might be a more reliable source, but have not looked into it.
I advise everyone using prevalence estimates to treat them with some skepticism and look up the source.
The PVV is currently in formation with a right wing party, a moderate/Christian democrat right wing party, and a farmers party. Voting pro animal welfare would have complicated the formation.
In addition, I doubt that many far right voters are more pro animal welfare than pro-rural/pro-farmers. To me, this seems like it was a legitimate belief held by a few people high in the party they had to give up for strategic reasons.
I don’t think it was a way to screw over Muslims.
I am concerned about the H5N1 situation in dairy cows and have written and overview document to which I occasionally add new learnings (new to me or new to world). I also set up a WhatsApp community that anyone is welcome to join for discussion & sharing news.
In brief:
I believe there are quite a few (~50-250) humans infected recently, but no sustained human-to-human transmission
I estimate the Infection Fatality Rate substantially lower than the ALERT team (theirs is 63% that CFR >= 10%), something like 80%CI = 0.1 − 5.0
The government’s response is astoundingly bad—I find it insane that raw milk is still being sold, with a high likelihood that some of it contains infectious H5N1
There are still quite a few genetic barriers to sustained human-to-human transmission
This might be a good time to push specific pandemic preparedness policies
Here’s Carnegie’s publications on AI: https://carnegieendowment.org/programs/technology/ai/
I’m personally pretty concerned given the many rumors of farm workers falling ill and looking to join a discussion group.
In case it doesn’t exist, I’ve started a WhatsApp community that you can join via this link: https://chat.whatsapp.com/LD6OAM32PgF7WdJ51ABVsl
You make a lot of good points Lucius!
One qualm that I have though, is that you talk about “AIs” and that assumes that personal identity will be clearly circumscribed. (Maybe you assume this merely for simplicity’s sake?)
I think it is much more problematic: AI systems could be large but have information flows integrated, or run many small, unintegrated but identical copies. I would have no idea what would be a fair allocation of rights given the two different situations.