Head of Video at 80,000 Hours
(Opinions here my own by default though will sometimes speak in a professional capacity).
Personal website: www.chanamessinger.com
ChanaMessinger
Karma: 4,330
Seems like there’s room in the ecosystem for a weekly update on AI that does a lot of contextualization / here’s where we are on ongoing benchmarks. I’m familiar with:
a weekly newsletter on AI media (that has a section on important developments that I like)
Jack Clark’s substack which I haven’t read much of but seems more about going in depth on new developments (though does have a “Why this matters” section. Also I love this post in particular for the way it talks about humility and confusion.
Doing Westminster Better on UK politics and AI / EA, which seems really good but again I think goes in depth on new stuff
I could imagine spending time on aggregation of prediction markets for specific topics, which Metaculus and Manifold are doing better and better over time.
I’m interested in something that says “we’re moving faster / less fast than we thought we would 6 months ago” or “this event is surprising because” and kind of gives a “you are here” pointer on the map. This Planned Obsolescence post called “Language models surprised us” I think is the closest I’ve seen.
Seems hard, also maybe not worth it enough to do, also maybe it’s happening and I’m not familiar with it, would love to hear, but it’s what I’d personally find most useful and I suspect I’m not alone.
Some added context on the 80k podcasts:
At the beginning of the Jan Leike episode, Rob says:
Two quick notes before that:We’ve had a lot of AI episodes in a row lately, so those of you who aren’t that interested in AI or perhaps just aren’t in a position to work on it, might be wondering if this is an all AI show now.
But don’t unsubscribe because we’re working on plenty of non-AI episodes that I think you’ll love — over the next year we plan to do roughly half our episodes on AI and AI-relevant topics, and half on things that have nothing to do with AI.
What happened here is that in March it hit Keiran and Luisa and me that so much very important stuff had happened in the AI space that had simply never been talked about on the show, and we’ve been working down that coverage backlog, which felt pretty urgent to do.
But soon we’ll get back to a better balance between AI and non-AI interviews. I’m looking forward to mixing it up a bit myself.
Retrospective Metrics: Tools for Collaborative Truth Seeking
I liked this!
I appreciated that for the claim I was most skeptical of: “There’s also the basic intuition that more people with new expertise working on a hard problem just seems better”, my skepticism was anticipated and discussed.
For me one of the most important things is:
Patch the gaps that others won’t cover
E.g., if more academics start doing prosaic alignment work, then ‘big-if-true’ theoretical work may become more valuable, or high-quality work on digital sentience.
There’s probably predictable ‘market failures’ in any discipline – work that isn’t sexy but still very useful (e.g., organizing events, fixing coordination problems, distilling the same argument into new language, etc.).
Generally track whether top priority work is getting covered (e.g., information security, standards and monitoring)
This, plus avoiding and calling out safety washing, keeping an eye out for overall wrongheaded activities and motions (for instance, probably a lot of regulation is bad by default and some could actively make things worse), seem like the strongest arguments against making big naive shifts because the field is in a broad sense less neglected.
More generally, I think a lot of the details of what kinds of engagements we have with the broader world will matter (and I think in many cases “guide” will be a less accurate description of what’s on the table than “be one of the players in the room”, and some might be a lot more impactful than others, but I don’t have super fleshed out views on which yet!
I really loved this! I have basically no knowledge of the underlying context, but I think this symmary gave me a feel for how detailed and complicated this is (reality has a lot of detail and a lot of societies for air conditioning engineers!), a bit of the actual science, as well as some of the players involved and their incentives.
It’s helpful and interesting to look at what small scientific communities are like as analogues for EA research groups.
[Question] What are good “translated arguments” or arguments in need of translation?
From Astral Codex Ten
FRI called back a few XPT forecasters in May 2023 to see if any of them wanted to change their minds, but they mostly didn’t.
I really like this concept of epistemic probation—I agree also on the challenges of making it private and exiting such a state. Making exiting criticism-heavy periods easier probably makes it easier to levy in the first place (since you know that it is escapable).
Did you mean for the second paragraph of the quoted section to be in the quote section?
Thanks so much for this, I really enjoyed it! I really like this format and would enjoy seeing more of it.
This isn’t the point, and there’s likely so much behind each vignette that we don’t see, but I so wish for some of these folks that they are able to find e.g. people/mentors who encourage their “dumb questions”, people who want to talk about consciousness, people who can help figure out what to do with doomer-y thoughts, maybe telling aggregators of information about some of the things listed (community health is one for some topics including some cases of bad management, there are others). I wish them luck, encourage finding an information aggregator, and wonder if maybe the comments here might end up with offers to talk about the things people find hard to talk about. I just have a sense that (exempting all the complexity I don’t see) there are people who want to talk about these things and feel open to weird and heterodox views here!
But I know that we’re more talking about vibes and overall incentive gradients and so on. I’m pretty uncertain what systemic solutions would look like here, but I’ll be curious what your poll ends up finding.
I myself have been worried about the social effects of friends working at AI labs and organizations and whether that’s going to make it harder for me or others to criticize that org or have a negative sentiment towards them. Would love to talk more about that some time, especially with people who work at these places!
A small sadness I have (and not sure what there is to do about this, I appreciate the sharing) is that I think I’m pretty likely to remember the unendorsed ones about the same as the endorsed ones, because the vignettes are the memorable bits. Just an unfortunate fact about this kind of thing.
Right, right, I think on some level this is very unintuitive, and I appreciate you helping me wrap my mind around it—even secret information is not a problem as long as people are not lying about their updates (though if all updates are secret there’s obviously much less to update on)
I appreciate the reminder that “these people have done more research” is itself a piece of information that others can update on, and that the mystery of why they haven’t isn’t solved. (Just to ELI5, we’re assuming no secret information, right?)
I suppose this is very similar to “are you growing as a movement because you’re convincing people or via selection effects” and if you know the difference you can update more confidently on how right you are (or at least how persuasive you are).
I tried for a while to find where I think Oliver Habryka talked about this, but didn’t find it. If someone else finds it, let me know!
I want to just appreciate the description you’ve given of interaction responsibility, and pointing out the dual tensions.
On the one hand, wanting to act but feeling worried that by merely getting involved you open yourself up to criticism, thereby imposing a tax on acting even when you think you would counterfactually make the situation better (something I think EA as a concept is correctly really opposed to in theory).
On the other hand, consequences matter, and if in fact your actions cause others who would have done a better job not to act, and that’s predictable, it needs to be taken into account. This is all really tough, and it bites for lots of orgs or people trying to do things that get negative feedback, and it also bites for the orgs giving negative feedback, which feels worth bearing in mind.
The forum naming conversation feels like an example of something that’s been coming up a lot that I don’t have a crisp way of talking about, which is the difference between “this is an EA thing” as a speech act and “this is an EA thing” as a description. I’m supportive of orgs and projects not branding themselves EA because they don’t want to or want to scope out a different part of the world of possible projects or don’t identify as EA. But I’m also worried about being descriptively deceptive (even unintentionally), by saying “oh, this website isn’t really an EA thing, it’s just a forum where a lot of EAs hang out.” That feels like it confuses and potentially deceives in a way I don’t like. Don’t know how to thread this needle, seems hard!
I’m certainly not an expert in institutional design, but for what it’s worth, it feels really non-obvious to me that:
It seems harder for a decentralised movement to centralise than it is for a centralised movement to decentralise. So, trying to be as centralised as possible at the moment preserves option value.
Like, I think projects find it pretty hard to escape the sense that they’re “EA” even when they want to (as you point out), and I think it’s pretty easy to decide you want to be part of EV or want to take your cues from the relevant OP team and do what they’re excited to fund, whereas ignoring consensus around you, taking feedback but doing something else, and so on, seem kind of hard, especially if your inside view is interested in doing something no one wants to fund!
Thanks for this! Very interesting.
I do want to say something stronger here, where “competence” sounds like technical ability or something, but I also mean a broader conception of competence that includes “is especially clear thinking here / has fewer biases here / etc”
Correctly Calibrated Trust
Trust is a two-argument function
I’m sure this must have been said before, but I couldn’t find it on the forum, LW or google
I’d like to talk more about trusting X in domain Y or on Z metric rather than trusting them in general. People/orgs/etc have strengths and weaknesses, virtues and vices, and I think this granularity is more precise and is a helpful reminder to avoid the halo and horn effects, and calibrates us better on trust.
Thanks!