Background in philosophy, international development, statistics. Doing a technical AI PhD at Bristol.
Financial conflict of interest: technically the British government through the funding council.
I think you’re right, see my reply to Ivan.
I think I generalised too quickly in my comment; I saw “virality” and “any later version” and assumed the worst. But of course we can take into account AGPL backfiring when we design this licence!
One nice side effect of even a toothless AI Safety Licence: it puts a reminder about safety into the top of every repo. Sure, no one reads licences (and people often ignore health and safety rules when it gets in their way, even at their own risk). But maybe it makes things a bit more tangible like LICENSE.md gives law a foothold into the minds of devs.
Seems I did this in exactly 3 posts before getting annoyed.
That’s cool! I wonder if they suffer from the same ambiguity as epistemic adjectives in English though* (which would suggest that we should skip straight to numerical assignments: probabilities or belief functions).
Anecdotally, it’s quite tiring to put credence levels on everything. When I started my blog I began by putting a probability on all major claims (and even wrote a script to hide this behind a popup to minimise aesthetic damage). But I soon stopped.
For important things (like Forum posts?) it’s probably worth the effort, but even a document-level confidence statement is a norm with only spotty adoption on here.
This is a neat idea, and unlike many safety policy ideas it has scaling built in.
However, I think the evidence from the original GPL suggests that this wouldn’t work. Large companies are extremely careful to just not use GPL software, and this includes just making their own closed source implementations.* Things like the Skype case are the exception, which make other companies even more careful not to use GPL things. All of this has caused GPL licencing to fall massively in the last decade.** I can’t find stats, but I predict that GPL projects will have much less usage and dev activity.
It’s difficult to imagine software so good and difficult to replicate that Google would invite our virus into their proprietary repo. Sure, AI might be different from [Yet Another Cool AGPL Parser] - but then who has a bigger data moat and AI engineering talent than big tech, to just implement it for themselves?
Aschenbrenner’s model strikes me as a synthesis of the two intellectual programmes, and it doesn’t get enough attention.
Robin Hanson is the best critic imo. He has many arguments, or one very developed one, but big pieces are:
Innovation in general is not very “lumpy” (discontinuous). So we should assume that AI innovation will also not be. So no one AI lab will pull far ahead of the others at AGI time. So there won’t be a ‘singleton’, a hugely dangerous world-controlling system.
Long timelines [100 years+] + fire alarms
Opportunity cost of spending / shouting now “we are far from human level AGI now, we’ll get more warnings as we get closer, and by saving $ you get 2x as much to spend each 15 years you wait.”″having so many people publicly worrying about AI risk before it is an acute problem will mean it is taken less seriously when it is, because the public will have learned to think of such concerns as erroneous fear mongering.”
The automation of labour isn’t accelerating (therefore current AI is not being deployed to notable effect, therefore current AI progress is not yet world-changing in one sense)
He might not be what you had in mind: Hanson argues that we should wait to work on AGI risk, rather than that safety work is forever unnecessary or ineffective. The latter claim seems extreme to me and I’d be surprised to find a really good argument for it.
You might consider the lack of consensus about basic questions, mechanisms, solutions amongst safety researchers to be a bad sign.
Nostalgebraist (2019) sees AGI alignment as equivalent to solving large parts of philosophy: a noble but quixotic quest.
Melanie Mitchell also argues for long timelines. Her view is closer to the received view in the field (but this isn’t necessarily a compliment).
Spoilers for Unsong:
Jalaketu identifies the worst thing in the world—hell—and sacrifices everything, including his own virtue and impartiality, to destroy it. It is the strongest depiction of the second-order consistency, second-order glory of consequentialism I know. (But also a terrible tradeoff.)
Shouldn’t the title be “Proportional Representation seems overrated”?
PR is often what people mean by voting reform, in the UK, but there are options without these problems, e.g. approval voting.
I see “effective altruist” as a dodgy shorthand for the full term: “aspiring effective altruist”. I’m happy to identify as the latter in writing (though it is too clunky for speech).
I call shotgun on “On Certainty”, one of the most-wanted books. (The author and I have butted heads before. He is much better at headbutting than me.)
I felt much the same writing it. I’ll add that to my content note, thanks.
The opposite post (reasons not to worry) could be good as well. e.g.
No sign of automation accelerating in general
Maybe extreme lack of consensus about basic questions, mechanisms, solutions.
Mid-term safety work is chasing a uselessly fast-moving target—who was thinking about language model alignment even 3 years ago?
In this one, it’s that there is no main body, just a gesture off-screen. Only a small minority of readers will be familiar enough with the funding apparatus to complete your “exercise to the reader...” Maybe you’re writing for that small minority, but it’s fair for the rest to get annoyed.
In past ones (from memory), it’s again this sense of pushing work onto the reader. Sense of “go work it out”.
It might be better to collate and condense your series into one post, once it’s finished (or starting now). These individual posts really aren’t convincing, and probably hurt your case if anything. Part of that is the Forum’s conventions about content being standalone. But the rest is clarity and evidence: your chosen style is too esoteric.
I don’t think it’s our unwillingness to hear you out. Some of the most well-regarded posts on here are equally fundamental critiques of EA trends, but written persuasively / directly:
Worth noting that multivitamins are associated with very slightly increased mortality in the general population. Cochrane put this down to them overdosing A, E, and beta-carotene, which I don’t expect vegans to be deficient in, so the finding might transfer. (Sounds like you’ve done blood tests though, so ignore me if it helps you.)
The cycle of people coming up with ideas about how to organise people into projects, or prevent redundant posts, or make the Forum more accretive, being forgotten a week later. i.e. We fail to coordinate on coordination projects.
Can anyone in clean meat verify this news? The last time I checked, we were still years off market release.
Conditional on it being a real shock, hooray!
Follow-up post to ARCHES with ranking of existing fields, lots more bibliographies.