At least for a moment in time, I was GWWCâs 10,000th active pledger.
zeshenđ¸
Karma: 242
Yeah 1 and 3 seems right to me, thanks.
On 2, I think there are quite a number of techniques that give you quantitative risk estimates, and itâs quite routine in safety engineering and often required (e.g. to demonstrate that you have achieved 1e-4 fatality threshold and any further risk reduction is impractical). I donât fully understand most of the techniques listed in ISO31010, but it seems that a number of them do give quantitative risk estimates as a result the risk evaluation process, e.g. monte carlo, bayesian networks, F/âN diagrams, VaR, toxicological risk assessment, etc.
If you havenât already seen this paper on risk modelling, they use FTA and bayesian networks to estimate risks quantitatively.
Of course. Many of the these techniques are specific to certain parts of the risk assessment process. The document is unfortunately paywalled, but risk assessment can be said to have these three parts:
Risk identification
Risk analysis
Consequence
Likelihood
Level of risk
Risk evaluation
Risk treatment is missing here becacuse itâs sort of a separate process outside of risk assessment (but is tied to risk evaluation), and ISO 31010 specifically addresses the risk assessement phase. âBrainstormingâ as a class of techniques and can take different shapes and form (in my previous work we used to have structured sessions to generate âwhat-if scenariosâ), and they specifically address mainly 1 and some of 2a above.
But as Jan pointed out in his comment, perhaps safety cases are a meta-framework and not the technique itself, so the quality of a safety case depends on the quality of the evidence put forth alongside the arguments, and this quality may be related to the suitability and implementation of the specific techniques used to generate the evidence.
Iâm curious how and why safety cases got so popular in AI safety. There are so many other risk assessment techniques out there, for reference ISO31010 lists 30 of them (see here) and theyâre far from exhaustive. My instinct is that itâs because safety cases are purely text-based, easily understandable, and does not require proper risk management concepts (e.g. hazards, events, consequences) for it to work; so at some point perhaps someone suggested to use it and the rest of the field just went with it.
(also I donât know how common safety cases are in safety engineering, in my decade in the oil and gas industry I have never heard of it before, though it may be more common in other industries)
Yes.
sumber sel berasal daripada haiwan yang haram
If the source of the cell is from an animal that is haram (i.e. non-halal), then it cannot be considered halal.
The defense in depth thesis is that you are best off investing some resources from your limited military budget in many different defenses (e.g. nuclear deterrence; intelligence gathering and early warning systems; an air force, navy and army; command and communication bunkers; diplomacy and allies) rather than specialising heavily in just one.
Iâm not familiar with how this concept is used in the military, but in safety engineering Iâve never heard of it as a tradeoff between âmany layers, many holesâ vs âone layer, few holesâ. The swiss cheese model is often meant to illustrate the fact that your barriers are often not 100% effective, so even if you think you have a great barrier, you should have more than one of it. From this perspective, the concept of having multiple barriers is straightforwardly good and doesnât imply justifying the use of weaker barriers.
AgreedâI shouldâve made it clearer in the title that I was referring specifically to the AI safety people in EA, i.e. this excludes other EAs not in AI safety, and also excludes other non-EAs in AI safety.
I would be interested to hear the counterpoints from those who have Disagree-voted on this post.
Likewise!
Do you think the âvery particular worldviewâ you describe is found equally among those working on technical AI safety and AI governance/âpolicy? My impression is that policy inherently requires thinking through concrete pathways of how AGI would lead to actual harm as well as greater engagement with people outside of AI safety.
I think theyâre quite prevalent regardless. While some peopleâs roles indeed require them to analyze concrete pathways more than others, the foundation of their analysis is often implicitly built upon this worldview in the first place. The result is that their concrete pathways tend to be centred around some kind of misaligned AGI, just in much more detail. Conversely, someone with a very different worldview who does such an analysis might end up with concrete pathways centred around severe discrimination of marginalized groups.
I have also noticed a split between the âsuperintelligence will kill us allâ worldview (which you seem to be describing) and âregardless of whether superintelligence kills us all, AGI/âTAI will be very disruptive and we need to manage those risksâ (which seemed to be more along the lines of the Will MacAskill post you linked toâespecially as he talks about directing people to causes other than technical safety or safety governance).
There are indeed many different âsub-worldviewsâ, and I was kind of lumping them all under one big umbrella. To me, the most defining characteristic of this worldview is AI-centrism, and treating the impending AGI as an extremely big deal â not just like any other big deals we have seen before, but this will be unprecedented. Those within this overarching worldview would differ in terms of the details, e.g. will it kill everyone? or will it just lead to gradual disempowerment? are LLMs getting us to AGI? or is it some yet-to-be-discovered architecture? should we focus on getting to AGI safely? or start thinking more about the post-AGI world? I think many people move between these âsub-worldviewsâ as they see evidences that update their priors, but way fewer people move out of this overarching worldview entirely.
AI Safety Has a Very ParÂticÂuÂlar Worldview
(semi-commitment for accountability)
Iâm considering writing more about how a big part of AI safety seem to be implicitly built upon an underlying worldview and we have rarely challenged that worldview.
I think there might be some missing links:
> In the current stage of LLMs, one may reasonably have short timelines for AGI coming in the next 3-5 years, as given here. Here, here, here, and here.
I think GWWC uses the term âactive pledgerâ to refer to the pledgers that are still pledgers. I am #10294, which means I was the 10294th person to sign the pledge, but out of those 10294 people, 294 of them either cancelled their pledges or there were double-counts etc. So at the time when I pledged, there were 9999 existing others who were âactiveâ i.e. had not cancelled their pledge. That doesnât mean the on-paper âactiveâ pledgers actually adhere to their pledges, and I donât know if GWWC has a different term for that.
<brag>
You donât know me, but I wasnât just one of the first 10,000. I was the 10,000th.
</âbrag>
Thank you so much! More importantly, congratulations to you and the GWWC team for this milestone!
Thank you very much! I shouldâve known that the pledge # would be different from the active pledgers, but at Iâm glad to know it now. I just made the pledge and I hope to be the 10,000th active pledger, even if momentarily :D
(apologies in advance, this is a bit of a pointless question)
The pledge count is 9993 nowâIâm curious if this amount is updated live? Iâd like to be the 9999th pledger, but Iâm unsure how the counting system works. Thanks!
[Question] How ofÂten is the GWWC pledge count upÂdated?
I see similarities with this paper. It seems your work focuses more on whatâs feasible for geopolitical rivals?
David Thorstad wrote a similar post in case itâs of interest to anyone here.
Agree this is great. But I also wonder if there were any EA interventions that counterfactually led to this. If so, these efforts are probably worth replicating.