Remmelt

Karma: 1,219

See explainer on why AGI could not be controlled enough to stay safe:
https://www.lesswrong.com/posts/xp6n2MG5vQkPpFEBH/the-control-problem-unsolved-or-unsolvable

I post here about preventing unsafe AI.

Note that I’m no longer part of EA, because of overreaches I saw during my time in the community (core people leading technocratic projects with ruinous downside risks, a philosophy based around influencing consequences over enabling collective choice-making, and a culture that’s bent on proselytising both while not listening deeply enough to integrate other perspectives).

Remmelt Aug 20, 2024, 4:20 AM
2 points
0 ∶ 0
in reply to: huw’s comment on: If AI is in a bubble and the bubble bursts, what would you do?
What are you thinking about in terms of pre-harm enforcement?
I’m thinking about advising premarket approval – a requirement to scope model designs around prespecified uses and having independent auditors vet the safety tests and assessments.

Remmelt Aug 6, 2024, 3:10 AM
2 points
0 ∶ 0
in reply to: Habryka [Deactivated]’s comment on: Lessons from the FDA for AI
The report is focussed on preventing harms of technology to people using or affected by that tech.

It uses FDA’s mandate of premarket approval and other processes as examples of what could be used for AI.

Restrictions to economic productivity and innovation is a fair point of discussion. I have my own views on this – generally I think the negative assymetry around new scalable products being able to do massive harm gets neglected by the market. I’m glad the FDA exists to counteract that.

The FDA’s slow response to ramping up COVID vaccines during the pandemic is questionable though, as one example. Getting a sense there is a lot of problems with bureacracy and also industrial capture with FDA.

The report does not focus on that though.

Remmelt Jul 28, 2024, 5:59 AM
2 points
0 ∶ 0
in reply to: Rebecca’s comment on: 80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)
They mentioned that line at the top of the 80k Job board.

Still do I see.

“Handpicked to help you tackle the world’s most pressing problems with your career.”

https://jobs.80000hours.org/

Remmelt Jul 27, 2024, 2:47 AM
4 points
0 ∶ 0
in reply to: Vasco Grilo🔸’s comment on: I bet Greg Colbourn 10 k€ that AI will not kill us all by the end of 2027
Ah, I wasn’t clear. To bet that AI will not kill us all by the end of 2027.

I don’t think that makes sense, given the world-complexity “AI” would need to learn and evolve and get tinkered to be able to navigate. I’ve had some conversations with Greg about this.

Remmelt Jul 27, 2024, 2:45 AM
2 points
0 ∶ 0
in reply to: Linch’s comment on: Émile P. Torres’s history of dishonesty and harassment
Change your mind in what way? Could you elaborate a bit?

Remmelt Jul 27, 2024, 2:42 AM
2 points
0 ∶ 0
in reply to: Rebecca’s comment on: 80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)
Could you quote which line you mean? Then I can mention where you can find it back

Remmelt Jul 19, 2024, 2:44 PM
9 points
0 ∶ 0
in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)
We used to list roles that seemed more tangentially safety-related, but because of our reduced confidence in OpenAI...

This misses aspects of what used to be 80k’s position:
❝ In fact, we think it can be the best career step for some of our readers to work in labs, even in non-safety roles. That’s the core reason why we list these roles on our job board.
– Benjamin Hilton, February 2024

❝ Top AI labs are high-performing, rapidly growing organisations. In general, one of the best ways to gain career capital is to go and work with any high-performing team — you can just learn a huge amount about getting stuff done. They also have excellent reputations more widely. So you get the credential of saying you’ve worked in a leading lab, and you’ll also gain lots of dynamic, impressive connections.
– Benjamin Hilton, June 2023 - still on website

80k was listing some non-safety related jobs:
– From my email on May 2023:
– From my comment on February 2024:

Remmelt Jul 19, 2024, 2:13 PM
13 points
2 ∶ 2
on: 80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)
I haven’t shared this post with other relevant parties – my experience has been that private discussion of this sort of thing is more paralyzing than helpful.

Fourteen months ago, I emailed 80k staff with concerns about how they were promoting AGI lab positions on their job board.

The exchange:
- I offered specific reasons and action points.
- 80k staff replied by referring to their website articles about why their position on promoting jobs at OpenAI and Anthropic was broadly justified (plus they removed one job listing).
- Then I pointed out what those articles were specifically missing,
- Then staff stopped responding (except to say they were “considering prioritising additional content on trade-offs”).
It was not a meaningful discussion.

Five months ago, I posted my concerns publicly. Again, 80k staff removed one job listing (why did they not double-check before?). Again, staff referred to their website articles as justification to keep promoting OpenAI and Anthropic safety and non-safety roles on their job board. Again, I pointed out what’s missing or off about their justifications in those articles, with no response from staff.

It took the firing of the entire OpenAI superalignment team before 80k staff “tightened up [their] listings”. That is, three years after the first wave of safety researchers left OpenAI.

80k is still listing 33 Anthropic jobs, even as Anthropic has clearly been competing to extend “capabilities” for over a year.

Remmelt Jun 7, 2024, 7:06 AM
2 points
1 ∶ 0
in reply to: Remmelt’s comment on: Émile P. Torres’s history of dishonesty and harassment

Happy to have a call in a few weeks’ time. Let’s wait until after Émile’s essays regarding accusations and identity by “Fuentes” are (hopefully) out.

See Émile’s substack: https://mileptorres.substack.com/

Remmelt Jun 7, 2024, 7:05 AM
−3 points
0 ∶ 0
in reply to: Remmelt’s comment on: Émile P. Torres’s history of dishonesty and harassment

There will (likely) be a post out

See Émile’s substack: https://mileptorres.substack.com/

Remmelt Jun 7, 2024, 7:03 AM
1 point
0 ∶ 0
on: I bet Greg Colbourn 10 k€ that AI will not kill us all by the end of 2027
I’m happy to join that bet.

Remmelt May 23, 2024, 8:20 AM
3 points
0 ∶ 1
in reply to: JWS 🔸’s comment on: Émile P. Torres’s history of dishonesty and harassment
Hi JWS,

Responding to your thoughtful points:
1. I agree you can have two bad actors at once. The fact that one bad actor is writing distorted critiques that misportray another, does not mean the other person is actually behaving well. The thing I find tough to unpack here is that some (not all) of the critiques I’ve read from you come straight from this fake “Fuentes” account. Given that the person behind that account is misportraying themselves and Émile in obvious places (I already gave a few examples), I would be very careful to not take their critiques at face value. Some years ago, Eliezer Yudkowsky wrote about a hypothetical scenario where one agent is trying to make you believe X is true even if it’s false, and provides very convincing arguments that X is true anyway. How much should you “update” on their arguments that X is true anyway?
2. Good nuance. I did not mean to say that you’re a “core” EA person. Please do consider that if you’re going to take this in your own hands (hopefully not Thanos style!) to defend EA from an outsider whose critiques are making EA look bad, then naturally you’ll get many upvotes from members of this community backing you up. Me writing dissonating comments like these on the EA Forum (despite that EAs like you know me) is putting myself into a minority position, where I make myself wide open to being socially discredited from any angle found by any member motivated against. I hesitate to use this analogy, but it’s a bit like returning to a circle of devout Christians to clear the name of an atheist, who turned out to be wrongly accused by an anonymous Christian. This is not a neutral context of conversation, and experiencing that is not pleasant. I think you did not quite consider this in how you worded your initial reply.
3. [splitting this out from 2.] Where I agree with you is that Émile has repeatedly made overgeneralised blanket statements about people tied to the EA community. And I’ve been correcting Émile on this (eg. see point 1. here: https://x.com/RemmeltE/status/1695071975096340480), in a similar way as I did when advising Glen Weyl against publishing an overgeneralised caustic media article against EA (which along with feedback from someone else, made Glen decide not to publish). I don’t want illegitimate critiques of EA to be shared around, because it makes EAs close off to outside critiques and devote less energy to listening for where to change harmful mindsets/activities in the community.
Perhaps we can pick this up again in DMs or a Calendly call :)

Happy to have a call in a few weeks’ time. Let’s wait until after Émile’s essays regarding accusations and identity by “Fuentes” are (hopefully) out.

And just want to clarify that I do admire you and your work even if I don’t agree with your conclusions. I think you’re a much better EA critic (to the extent you identify as one) than Émile is.

Appreciating your considerate words here. Thank you.

Remmelt May 12, 2024, 2:31 AM
3 points
0 ∶ 0
in reply to: Jason’s comment on: Émile P. Torres’s history of dishonesty and harassment
Good point.

The “Mark Fuentes” backstory of being a public defender who only briefly was involved with EA does not add up. Someone on the periphery like that wouldn’t know enough about the community to fish up all of those quotes and talk about the people involved.

It does start to look like misuse of anonymity when even before “Mark Fuentes” other anonymised accounts like “Claressa Meals” were posting the exact same screenshots and variations of the same accusations. These can be traced back to one specific person.

Remmelt May 11, 2024, 2:00 PM
9 points
0 ∶ 0
in reply to: Ben Millwood🔸’s comment on: Émile P. Torres’s history of dishonesty and harassment

Do you know who it is?

Yes, I know who it is. There are paper trails of posts all pointing to this one person.

If so, do you intend for readers to be able to figure it out based on what you’ve said?

I was not trying to communicate who “Fuentes” specifically was, more that he is an EA insider and not the fake person described in the Fuentes bio.

If so, why not just name them? Can you say how you traced them?

There will (likely) be a post out (by Émile) that goes into this in painstaking detail. So there’s really no reason for me to reveal their name now (given the forum’s doxing rules).

I think the moderator (JP) was acting carefully and fairly in redacting the details I shared. It’s against the forum rules to dox, and the details I shared were enough to make a good guess who the person is.

This is a tricky situation, because “Fuentes” is not an openly anonymous account, but a fake name account started with a fake bio.

They are pretending to be someone else, and have an axe to grind based on their communications in earlier years. In that sense, the situation is more like the Holden astroturfing case (who also held a personal interest in convincing people in a particular direction, though not necessarily “an axe to grind”).

Remmelt May 11, 2024, 9:19 AM
12 points
0 ∶ 2
in reply to: JWS 🔸’s comment on: Émile P. Torres’s history of dishonesty and harassment
Let’s check if we can find some common ground. From my side, I raised the following in DMs before, but we did not get to discuss it:
- Can we agree that ”Mark Fuentes” is a fake name (not just a pseudonym, the bio is fake too)?
- And agree that when a person is using a fake name (and a trail of other names and throwaway accounts) to target one specific person with critiques, that our prior should be that the “fake name” person is not constructing impartial critiques?
Adding this:
- And agree that if the “fake name” person is constructing lots of critiques to target one specific person, that it would put the “targeted” person in an unfair position to expect them to address all of the critiques?
I’d note that Émile doesn’t really push back on many of the claims in the Fuentes article

Yes. I think Émile should have just published their list of specific reasons why the “Fuentes” critiques were ungrounded. From what I heard, a friend dissuaded Émile from publishing that long list because it would suck more attention toward the exchange, when (as Owen Cotton-Barratt pointed at) there already are clear reasons why not to take “Fuentes” seriously.

It looks like Émile is going to publish that list now (now another anonymous account has reposted the “Fuentes” post).

For observers, if you want to go down the twitter rabbit hole when this all kicked off, and get the evidence with your own eyes, start here: https://nitter.poast.org/RemmeltE/status/1627153200930508800#m

Thanks, I recommend checking that thread too. You can see me remarking that I should have done background research before on Émile, and then probing for information.

and the stuff around Hillary Greaves and ‘Alex Williams’ seem far enough to rule someone as a bad-faith actor.

Actually, there is a reason for the Hillary Greaves misquote, as I mentioned in our DM discussion. All I can say is that Émile had no intention of publishing that misquote, and that Émile sent in a correction when they later found out about it.

The ‘Alex Williams thing’ is the only claim (of many) I looked into no more than a bit. I need to look more into that. Can get back to you on that, if you are still open to understanding more of what’s going on.

you treat Andreas far too harshly as well. You said of him “I think you are being intentionally deceptional here, and not actively truth-seeking.”

Yes, I did think Andreas was writing in a way that was deceptive.

I shared one example of this in that thread: that Andreas claimed that Émile made a racist joke, which as far as I can tell Émile didn’t. Émile was pointing out the hypocrisy of Peter Boghossian, who had been actively dismissive of the struggles of ethic minorities until he adopted a child of Asian descent. After that, Peter responded sensitively to people making comments around that, now he was actually affected himself too. Emile made a joke pointing out Peter’s conflicting behaviour, and later publicly apologised because this was still a terrible joke to make.

This is one case of many where Émile turned out to be quoted and twisted out of context by “Mark Fuentes” (the fake name that can be traced back to someone in EA – someone I heard years before was a repressive manager, from staff at the first organisation he left).

Andreas just took on the assertion: that Émile must have said something racist (on prior surprising given how anti-racism Émile is).

And then Andreas later silently dropped his earlier claim of racism (saying he had written that Émile had made a “a puerile joke,” rather than what Andreas actually wrote: “a racist puerile joke”). EDIT: I see now that Andreas still called it a racist comment in the tweet above, and also linked to his original tweet. So Andreas did not misrepresent his earlier statement. My bad, my mistake.

Andreas had stated that he had followed up and checked the “Fuentes” claims, and that after this that he had confirmed the claims were substantially true. If Andreas had actually done his background research, he should have noticed at least a few of the many discrepancies.

Can you see how that therefore came across as “intentionally deceptive” and “not actively truth-seeking” to me?

The alternative case is that Andreas was not being thorough at all in his background research and/or motivated toward a certain conclusion. I am open to that possibility now. Though I wouldn’t call my response at the time “harsh” then. Andreas was not taking enough care.

I do not trust your perspective on this saga Remmelt.

Not sure how to respond to this social claim. I have been open in sharing my reasoning with you, and have listened to your concerns.

The fact that, over a year on, you don’t seem to recognise this and (if anything) support Émile more against EA is a bad sign.

Can you see how that sentence can come across? Trying to split it out with its context, here’s what I get:
- A. Remmelt did not come to the same conclusion that JWS came to: that some EA outsider is untrustworthy.
- B. JWS is an EA insider. The conclusion by this EA insider must be correct (maybe because he privately messaged reasons and/or his comments received upvotes on the EA Forum).
- C. Remmelt (who co-founded EA Netherlands but then gradually noticed harmful overreaches by EA) is supporting that outsider in their critiques of EA.
- D. It is a bad sign to support an untrustworthy outsider’s critiques of EA.
- E. Therefore this must be a bad sign, given C. (Remmelt is supporting the outsider’s critiques of EA) and assuming B. (the EA insider is correct that that outsider is untrustworthy).
We even had a Forum DM discussion about this a while ago, and I provided even more public cases of bad behaviour by Émile

Which of the claims you raised in our DMs, besides the ‘Alex Williams’ claim, did I not address?

Feel free to share here, then we can discuss further. Guess we are both busy, but I can make time to dig into specifics. Will check back later.

Remmelt May 11, 2024, 6:02 AM
1 point
0 ∶ 0
in reply to: JP Addison🔸’s comment on: Émile P. Torres’s history of dishonesty and harassment
Thanks for correcting! I intentionally did not name them, but I see how mentioning two organisations they worked for is enough to identify them.

Remmelt May 8, 2024, 7:22 AM
6 points
3 ∶ 8
in reply to: yanni kyriacos’s comment on: Émile P. Torres’s history of dishonesty and harassment
I actually think EAs have been rather quick to dismiss outside folks when those folks have different views on the world (that can’t be easily translated into EA speak).

I would be much more careful about opening up to insiders that are widely praised as leading figures in the community (like SBF) than to outsiders whose views are commonly perceived to be in conflict with the community’s aims.

Remmelt May 8, 2024, 7:12 AM
−9 points
0 ∶ 7
on: Émile P. Torres’s history of dishonesty and harassment
I dug into this post a year ago.

I was taken aback, and started probing Émile for what was going on. When I checked sources though, I found that two fake accounts had been created (not just pseudonyms) to put out articles that systematically distorted what Émile had said.

Some tweets last year, in order of tweeting: https://x.com/RemmeltE/status/1627317076233957383 https://x.com/RemmeltE/status/1627506364712112128

[[Personal information removed by moderators]]

Remmelt 29 Apr 2024 6:40 UTC
3 points
0 ∶ 0
in reply to: Lovkush 🔸’s comment on: Funding case: AI Safety Camp
Thank you for the incisive questions.
What is the current funding status of AISC?
We received $57k through Manifund plus a $5k donation from a private donor.

Which funding bodies have you asked for funding from and do you know why they are not funding this (assuming they chose not to fund this)?
- For LTFF and SFF, Oliver Habryka was our main evaluator. See his comment here.
- For OpenPhil, see my comment here.
- For Nonlinear, that’s a network of donors who I guess mostly don’t have that much funds to spent. But I don’t know which if any donors there tried evaluating AISC and what their reasons were for not funding.
My understanding is you only just managed to get enough funding to run a budget version of AISC 10, so I presume that means you’ll be looking for funding for AISC 11.
Yes, this is correct. Even then, it is stretching it, because we haven’t gotten an income for running the just finished 150-participant edition (AISC 9). Backpay would be reasonable – to maintain our personal runways.

Remmelt 29 Apr 2024 5:16 UTC
1 point
0 ∶ 0
in reply to: Aaron Gertler 🔸’s comment on: Dear EA, please be the reason people like me will actually see a better world. Help me make some small stride on extreme poverty where I live—by the end of 2024.
Good question!

I haven’t written up a separate post on UCF and how it compares to other charity interventions. I’d consider it, but I am already stretching myself with other work.