harfe

Karma: 909

harfe Jun 26, 2024, 1:08 PM
12 points
12 ∶ 1
in reply to: timunderwood’s comment on: Responses to Manifest are not about Manifest
FYI: Emile Torres is using they/them pronouns. I think you should edit your comment to use their preferred pronouns.

harfe Jun 21, 2024, 8:06 PM
1 point
0 ∶ 0
in reply to: Chris Leong’s comment on: casebash’s Shortform

In fact, I’d go further and suggest that it would be great if they were to set up their own forum.

Manifold already has a highly active discord, where they can discuss all the manifold-specific issues. This did not prevent the EA Forum from discussing the topic, and I doubt it would be much different if Manifold had a proper forum instead of a discord.

This is annoying because many of these discussions rate high on controversy but low on importance for EA.

It might seem low on importance for EA to you, but I suspect some people who are upset about Manifest inviting right-wing people do not consider it low-importance.

harfe Jun 18, 2024, 5:23 PM
1 point
0 ∶ 0
on: Is it possibly desirable for sentient ASI to exterminate humans?
I strongly disagree. I think human extinction would be bad.

Not every utility function is equally desirable. For example, an ASI that maximizes the number of paperclips in the universe would be a bad outcome.

Thus, unless one adopts anthropocentric values, the utilitarian philosophy common in this forum (whether you approve of additivity or not) implies that it would be desirable for humans to develop ASI to exterminate humans as quickly and with as high a probability as possible, as opposed to the exact opposite goal that many people pursue.

Most people here do adopt anthropocentric values, in that they think human flourishing would be more desirable than a vast amount of paperclips.

harfe Jun 17, 2024, 10:09 PM
6 points
0 ∶ 0
in reply to: titotal’s comment on: My experience at the controversial Manifest 2024

I am not sure if he actually took part in the event, but there were people involved with him that were present who said he might be dropping by and that he had bought a ticket

Note that at this point we only have indirect word that he bought a ticket. Also note that anyone can buy a ticket, and if his ticket was cancelled by Manifold (which is probably the thing you want), we would not hear about that directly. Of course, information can emerge that he actually did attend.

harfe Jun 17, 2024, 3:28 PM
1 point
1 ∶ 0
on: Looking for a Document to Introduce AI Risks to Newbies
I suggest https://aisafety.info/

harfe Jun 14, 2024, 2:02 PM
7 points
2 ∶ 0
on: Trump talking about AI risks
Thanks for linking it! I recommend watching the 5-minute video.

Your title sounds like trump thinks there is a risk that AI takes over the human race (maybe consider changing the title).

The actual text from Trump in the video is:

you know there are those people that say it takes over the human race

Given the way Trump talks it can be sometimes difficult to assess what he actually believes. In general, Trump expressed a mix of concern and support for advanced AI. My impression is that Trump was more interested in advancing AI, rather than opposing it out of concern for the human race.

harfe Jun 7, 2024, 12:23 PM
1 point
0 ∶ 0
in reply to: Ozzie Gooen’s comment on: Ozzie Gooen’s Shortform

But if we get to GPT-7, I assume we could sort of ask it, “Would taking this next step, have a large chance of failing?“.

How do you know it tells the truth or its best knowledge of the truth without solving the “eliciting latent knowledge” problem?

harfe Jun 7, 2024, 12:15 PM
1 point
0 ∶ 0
in reply to: Ozzie Gooen’s comment on: Ozzie Gooen’s Shortform
This has been discussed before: https://forum.effectivealtruism.org/posts/GNfWT8Xqh89wRaaSg/unions-for-ai-safety

harfe Jun 5, 2024, 7:47 PM
18 points
10 ∶ 0
in reply to: niplav’s comment on: Caruso’s Quick takes

I am far more pessimistic than him about extinction from misaligned AI systems, but I think it’s quite sensible to try to make money from AI even in worlds from high probability of extinction, since the market signal provided counterfactually moves the market far less than the realizable benefit from being richer in such a crucial time.

I am sympathetic to this position when it comes to your own money. Like, if regular AI safety people put a large fraction of their savings into NVIDIA stock, that is understandable to me.

But the situation with Aschenbrenner starting an AGI investment firm is different. He is not directing (just) his own money, but the much larger capital of his investors into AGI companies. So the majority of the wealth gain will not end up in Aschenbrenner’s hands, but belong to the investors. This is different from a small-scale shareholder who gets all the gains (minus some tax) of his stock ownership.

But even if Aschenbrenner’s plan is to invest into the world-destroying in order to become richer later when it matters, it would be nice to say so and also explain how you intend to use the money later. My guess however is that this is not actually what Aschenbrenner actually believes. He might just be in favour of accelerating these technologies.

harfe Jun 5, 2024, 7:30 PM
1 point
0 ∶ 0
in reply to: NickLaing’s comment on: Review of Past Grants: The $100,000 Grant for a Video Game?
I think you are replying to the wrong comment here.

harfe Jun 4, 2024, 5:01 PM
1 point
0 ∶ 0
in reply to: David Mathers🔸’s comment on: JWS’s Shortform
That does not help me understand what is meant there. I fail to see relevant analogies to AI Safety.

harfe Jun 4, 2024, 4:11 PM
1 point
0 ∶ 0
in reply to: JWS 🔸’s comment on: JWS’s Shortform

‘lost of the mandate of heaven’

What do you mean by that? Presumably you do not mean it in any religious sense. Do you want to say that exclusively longtermist EA is much less popular among EAs than it used to be (i.e. the “heaven” is the opinion of the average EA)?

harfe Jun 3, 2024, 9:13 PM
11 points
9 ∶ 0
in reply to: NunoSempere’s comment on: NunoSempere’s Shortform
I value a forum where people are not rude to each other, so I think it is good that moderators give out a warning to people who are becoming increasingly rude in a short time.

harfe Jun 3, 2024, 4:03 PM
19 points
12 ∶ 1
in reply to: Larks’s comment on: Review of Past Grants: The $100,000 Grant for a Video Game?
I think this is a good point when considering the decisions of LTFF.

However, you can make a similar question about funders who decide whether some money goes to LTFF or to some global health charity. And such funders exist (e.g. OpenPhil).

An these funders have to make the uncomfortable decision whether to save children’s lives or indirectly support some research, video games, and other things via LTFF.

harfe May 23, 2024, 10:15 AM
3 points
2 ∶ 0
in reply to: SaraAzubuike’s comment on: SaraAzubuike’s Quick takes
I don’t think lives of rich people should be more valuable than lives of poor people.

So, using [$ / lives saved] is the wrong approach. We should use [$ / (lives saved * VSL)] instead.

I strongly disagree with that.

harfe May 18, 2024, 6:53 PM
8 points
0 ∶ 0
in reply to: yanni kyriacos’s comment on: Yanni Kyriacos’s Shortform
Is this about the safety teams at capabilities labs?

If so, I consider it a non-obvious issue, whether pushing a talented people into an AI safety role at, e.g., DeepMind is a bad thing. If you think that is a bad thing, consider providing a more detailed argument, and writing a top-level post explaining your view.

If, instead, this is about EA institutions pushing people into capabilities roles, consider naming these concrete examples. As an example, 80k has a job advertising a role as a prompt engineer at Scale AI. That does not seem to be a very safety-focused role, and it is not clear how 80k wants to help prevent human extinction with that job ad.

harfe May 18, 2024, 6:25 PM
4 points
1 ∶ 0
on: “If we go extinct due to misaligned AI, at least nature will continue, right? … right?”
I just wanted to say that the new aisafety.info website looks great! I have not looked at everything in detail, just clicking around a bit, but the article seem of good quality to me.

I will probably mainly recommend aisafety as an introductory resource.

harfe May 15, 2024, 8:58 AM
21 points
10 ∶ 1
in reply to: MichaelStJules’s comment on: Ilya Sutskever has officially left OpenAI
These kinds of resignation messages might not be very informative though. There are probably incentives to say nice things about each other.

harfe Apr 23, 2024, 4:06 AM
51 points
7 ∶ 0
on: harfe’s Quick takes
Consider donating all or most of your Mana on Manifold to charity before May 1.

Manifold is making multiple changes to the way Manifold works. You can read their announcement here. The main reason for donating now is that Mana will be devalued from the current 1 USD:100 Mana to 1 USD:1000 Mana on May 1. Thankfully, the 10k USD/month charity cap will not be in place until then.

Also this part might be relevant for people with large positions they want to sell now:

One week may not be enough time for users with larger portfolios to liquidate and donate. We want to work individually with anyone who feels like they are stuck in this situation and honor their expected returns and agree on an amount they can donate at the original 100:1 rate past the one week deadline once the relevant markets have resolved.

harfe Apr 22, 2024, 2:52 PM
16 points
4 ∶ 2
on: Should we break up Google DeepMind?
Since this is tagged “Existential risk”: What does this have to do with existential risk? Or is it not supposed to be about existential risk, not even indirectly? As far as I can tell, the article does not talk about existential risk. I could make my own guesses and association of this topic with existential risk, but I would prefer if this is spelled out.