I am not sure if he actually took part in the event, but there were people involved with him that were present who said he might be dropping by and that he had bought a ticket
Note that at this point we only have indirect word that he bought a ticket. Also note that anyone can buy a ticket, and if his ticket was cancelled by Manifold (which is probably the thing you want), we would not hear about that directly. Of course, information can emerge that he actually did attend.
I suggest https://aisafety.info/
Thanks for linking it! I recommend watching the 5-minute video.
Your title sounds like trump thinks there is a risk that AI takes over the human race (maybe consider changing the title).
The actual text from Trump in the video is:
you know there are those people that say it takes over the human race
Given the way Trump talks it can be sometimes difficult to assess what he actually believes. In general, Trump expressed a mix of concern and support for advanced AI. My impression is that Trump was more interested in advancing AI, rather than opposing it out of concern for the human race.
But if we get to GPT-7, I assume we could sort of ask it, “Would taking this next step, have a large chance of failing?“.
How do you know it tells the truth or its best knowledge of the truth without solving the “eliciting latent knowledge” problem?
This has been discussed before: https://forum.effectivealtruism.org/posts/GNfWT8Xqh89wRaaSg/unions-for-ai-safety
I am far more pessimistic than him about extinction from misaligned AI systems, but I think it’s quite sensible to try to make money from AI even in worlds from high probability of extinction, since the market signal provided counterfactually moves the market far less than the realizable benefit from being richer in such a crucial time.
I am sympathetic to this position when it comes to your own money. Like, if regular AI safety people put a large fraction of their savings into NVIDIA stock, that is understandable to me.
But the situation with Aschenbrenner starting an AGI investment firm is different. He is not directing (just) his own money, but the much larger capital of his investors into AGI companies. So the majority of the wealth gain will not end up in Aschenbrenner’s hands, but belong to the investors. This is different from a small-scale shareholder who gets all the gains (minus some tax) of his stock ownership.
But even if Aschenbrenner’s plan is to invest into the world-destroying in order to become richer later when it matters, it would be nice to say so and also explain how you intend to use the money later. My guess however is that this is not actually what Aschenbrenner actually believes. He might just be in favour of accelerating these technologies.
I think you are replying to the wrong comment here.
That does not help me understand what is meant there. I fail to see relevant analogies to AI Safety.
‘lost of the mandate of heaven’
What do you mean by that? Presumably you do not mean it in any religious sense. Do you want to say that exclusively longtermist EA is much less popular among EAs than it used to be (i.e. the “heaven” is the opinion of the average EA)?
I value a forum where people are not rude to each other, so I think it is good that moderators give out a warning to people who are becoming increasingly rude in a short time.
I think this is a good point when considering the decisions of LTFF.
However, you can make a similar question about funders who decide whether some money goes to LTFF or to some global health charity. And such funders exist (e.g. OpenPhil).
An these funders have to make the uncomfortable decision whether to save children’s lives or indirectly support some research, video games, and other things via LTFF.
I don’t think lives of rich people should be more valuable than lives of poor people.
So, using [$ / lives saved] is the wrong approach. We should use [$ / (lives saved * VSL)] instead.
I strongly disagree with that.
Is this about the safety teams at capabilities labs?
If so, I consider it a non-obvious issue, whether pushing a talented people into an AI safety role at, e.g., DeepMind is a bad thing. If you think that is a bad thing, consider providing a more detailed argument, and writing a top-level post explaining your view.
If, instead, this is about EA institutions pushing people into capabilities roles, consider naming these concrete examples. As an example, 80k has a job advertising a role as a prompt engineer at Scale AI. That does not seem to be a very safety-focused role, and it is not clear how 80k wants to help prevent human extinction with that job ad.
I just wanted to say that the new aisafety.info website looks great! I have not looked at everything in detail, just clicking around a bit, but the article seem of good quality to me.
I will probably mainly recommend aisafety as an introductory resource.
These kinds of resignation messages might not be very informative though. There are probably incentives to say nice things about each other.
Consider donating all or most of your Mana on Manifold to charity before May 1.
Manifold is making multiple changes to the way Manifold works. You can read their announcement here. The main reason for donating now is that Mana will be devalued from the current 1 USD:100 Mana to 1 USD:1000 Mana on May 1. Thankfully, the 10k USD/month charity cap will not be in place until then.
Also this part might be relevant for people with large positions they want to sell now:
One week may not be enough time for users with larger portfolios to liquidate and donate. We want to work individually with anyone who feels like they are stuck in this situation and honor their expected returns and agree on an amount they can donate at the original 100:1 rate past the one week deadline once the relevant markets have resolved.
Since this is tagged “Existential risk”: What does this have to do with existential risk? Or is it not supposed to be about existential risk, not even indirectly? As far as I can tell, the article does not talk about existential risk. I could make my own guesses and association of this topic with existential risk, but I would prefer if this is spelled out.
Here is another argument against Geometric utility: It does not work if negative utilities are involved: is undefined if is negative. And I think some real-world experiences that involve suffering have negative utility.
FHI has shut down yesterday: https://www.futureofhumanityinstitute.org/
I strongly disagree. I think human extinction would be bad.
Not every utility function is equally desirable. For example, an ASI that maximizes the number of paperclips in the universe would be a bad outcome.
Most people here do adopt anthropocentric values, in that they think human flourishing would be more desirable than a vast amount of paperclips.