Tom McGrath

Karma: 80

Tom McGrath 27 Jan 2024 1:10 UTC
15 points
5 ∶ 0
on: RAND report finds no effect of current LLMs on viability of bioterrorism attacks
Seeing this is a positive update on how much to trust safety research on LLM dangers. A bit disappointing it’s getting much less traction than other papers with worse/nonexistent baselines though.

Tom McGrath 18 Sep 2023 18:49 UTC
3 points
0 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Policy ideas for mitigating AI risk
Cool, apologies if that came across a bit snarky (on rereading it does to me). I think this was instance N+1 of this phrasing and I’d gotten a bit annoyed by instances 1 through N which you obviously bear no responsibility for! I’m happy to have pushed back on the phrasing but hope I didn’t cause offence.
A more principled version of (1) would be to appeal to moral uncertainty, or to the idea that a regulator should represent all the stakeholders and I worry than an EA-dominated regulator would fail to do so.

Tom McGrath 18 Sep 2023 18:32 UTC
10 points
1 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Policy ideas for mitigating AI risk
A regulator overrepresenting EA seems bad to me (not an EA) because:
1. I don’t agree with a lot of the beliefs of the EA community on this subject and so I’d expect an EA-dominated regulator to take actions I don’t approve of.
2. Dominance by a specific group makes legitimacy much harder.
3. The EA community is pretty strongly intertwined with the big labs so most of the concerns from there carry over.
I don’t expect (1) to be particularly persuasive for you but maybe (2) and (3) are. I find some of the points in Ways I Expect AI Regulation To Increase X-Risk relevant to issues with overrepresentation of big labs. I think the overrepresentation of big labs would lead to a squashing of open-source, for instance, which I think is currently beneficial and would remain beneficial on the margin for a while.
More generally, I don’t particularly like the flattening of specific disagreements on matters of fact (and thus subsequent actions) to “wants people to be safe”/”doesn’t want people to be safe”. I expect that most people who disagree about the right course of action aren’t doing so out of some weird desire to see people harmed/replaced by AI (I’m certainly not) and it seems a pretty unfair dismissal.

Tom McGrath 18 Sep 2023 18:22 UTC
3 points
1 ∶ 0
in reply to: Steven Byrnes’s comment on: AI Pause Will Likely Backfire
I’ve certainly heard of your work but it’s far enough out of my research interests that I’ve never taken a particularly strong interest. Writing this in this context makes me realise I might have made a bit of a one-man echo chamber for myself… Do you mind if we leave this as ‘undecided’ for a while?
Regarding ELK—I think the core of the problem as I understand it is fairly clear once you begin thinking about interpretability. Understanding the relation between AI and human ontologies was part of the motivation behind my work on alphazero (as well as an interest in the natural abstractions hypothesis). Section 4 “Encoding of human conceptual knowledge” and Section 8 “Exploring activations with unsupervised methods” are the places to look. The section on challenges and limitations in concept probing I think echoes a lot of the concerns in ELK.
In terms of subsequent work on ELK, I don’t think much of the work on solving ELK was particularly useful, and often reinvented existing methods (e.g. sparse probing, causal interchange interventions). If I were to try and work on it then I think the best way to do so would be to embed the core challenge in a tractable research program, for instance trying to extract new scientific knowledge from ML models like alphafold.
To move this in a more positive direction, the most fruitful/exciting conceptual work I’ve seen is probably (1) the natural abstractions hypothesis and (2) debate. When I think a bit about why I particularly like these, for (1) it’s because it seems plausibly true, extremely useful if true, and amenable to both formal theoretical work and empirical study. For (2) it’s because it’s a pretty striking new idea that seems very powerful/scalable, but also can be put into practice a bit ahead of really powerful systems.

Tom McGrath 18 Sep 2023 18:02 UTC
4 points
1 ∶ 1
in reply to: Zach Stein-Perlman’s comment on: Policy ideas for mitigating AI risk
Yeah, this sounds right to me. At present I feel like a regulator would end up massively overrepresenting at least one of (a) the EA community and (b) large tech corporations with pretty obviously bad incentives.

Tom McGrath 18 Sep 2023 17:53 UTC
3 points
0 ∶ 0
in reply to: Tom McGrath’s comment on: AI Pause Will Likely Backfire
(I’m spinning this comment out because it’s pretty different in style and seems worth being able to reply to separately. Please let me know if this kind of chain-posting is frowned upon here.)
Another downside to declaring things empirically out of reach and relying on priors for your EV calculations and subsequent actions is that it more-or-less inevitably converts epistemic disagreements into conflict.
If it seems likely to you that this is the way things are (and so we should pause indefinitely) but it seems highly unlikely to me (and so we should not) then we have no choice but to just advocate for different things. There’s not even the prospect of having recourse to better evidence to win over third parties, so the conflict becomes no-holds-barred. I see this right now on Twitter and it makes me very sad. I think we can do better.

Tom McGrath 18 Sep 2023 17:43 UTC
2 points
1 ∶ 0
in reply to: Joe Collman’s comment on: AI Pause Will Likely Backfire
Thanks I mean more in terms of “how can we productively resolve our disagreements about this?”, which the EV calculations are downstream of. To be clear, it doesn’t seem to me that this is necessarily the hand we’ve been dealt but I’m not sure how to reduce the uncertainty.
At the risk of sidestepping the question, the obvious move seems to be “try harder to make the claim empirically testable”! For example, in the case of deception, which I think is a central example we could (not claiming these ideas are novel):
1. Test directly for deception behaviourally and/or mechanistically (I’m aware that people are doing this, think it’s good and wish the results were more broadly shared).
2. Think about what aspects of deception make it particularly hard, and try to study those in isolation and test those. The most important example seems to me to be precursors: finding more testable analogues to the question of “before we get good, undetectable deception do we get kind of crappy detectable deception?”
Obviously these all run some (imo substantially lower) risks but seem well worth doing. Before we declare the question empirically inaccessible we should at least do these and synthesise the results (for instance, what does grokking say about (2)?).

Tom McGrath 18 Sep 2023 17:24 UTC
5 points
1 ∶ 0
in reply to: Davidmanheim’s comment on: What’s in a Pause?
Thanks—this is clarifying. I think my confusion was down to not understanding the remit of the pause you’re proposing. How about we carry on the discussion in the other comment on this?

Tom McGrath 18 Sep 2023 17:23 UTC
1 point
0 ∶ 0
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire
Interesting—what do you have in mind for fast-progressing architectures explicitly aimed at creating AGI?
On your 2nd point on x-risks from non-LLM AI, am I right in thinking that you would also hope to catch dual-use scientific AI (for instance) in a compute governance scheme and/or pause? That’s a considerably broader remit than I’ve seen advocates of a pause/compute restrictions argue for and seems much harder to achieve both politically and technically.

Tom McGrath 17 Sep 2023 13:36 UTC
1 point
0 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Debate series: should we push for a pause on the development of AI?
I’m trying to make “FLOPstacles” happen for things that mean we can’t just take max FLOP per GPU and multiply by number of GPUs, e.g. mem or interconnect bandwidth.

Tom McGrath 16 Sep 2023 21:13 UTC
3 points
1 ∶ 0
in reply to: Otto’s comment on: Debate series: should we push for a pause on the development of AI?
There’s an important difference between pausing and evals: evals gets you loads of additional information. We can look at the results of the evals, discuss them and determine in what ways a model might have misuse potential (and thus try to mitigate it) or if the model is simply undeployable. If we’re still unsure, we can gather more data and additionally refine our ability to perform and interpret evals.
If we (i.e. the ML community) repeatedly do this we build up a better picture of where our current capabilities lie, how evals relate to real-world impact and so on. I think this makes evals much better, and the effect will compound over time. Evals also produce concrete data that can convince skeptics (such as me—I am currently pretty skeptical of much regulation but can easily imagine eval results that would convince me). To stick with your analogy, each time we do evals we thin out the fog a bit, with the intention of clearing it before we reach the edge, as well as improving our ability to stop.

Tom McGrath 16 Sep 2023 20:30 UTC
3 points
1 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Policy ideas for mitigating AI risk
This is a good insight—I definitely feel like lack of trust (due partly to uncertainty) in the proposed regulator is a big blocker for me feeling at all on board with pause/regulation more broadly. Especially relevant given that I think the original CAIP proposals missed the mark by some margin. I acknowledge that Thomas is writing in his personal capacity, but I think that the link is still relevant.

Tom McGrath 16 Sep 2023 20:13 UTC
10 points
4 ∶ 2
in reply to: Steven Byrnes’s comment on: AI Pause Will Likely Backfire
I certainly give relatively little weight to most conceptual AI research. That said, I respect that it’s valuable for you and am open to trying to narrow the gap between our views here—I’m just not sure how!
To be more concrete, I’d value 1 year of current progress over 10 years of pre-2018 research (to pick a date relatively arbitrarily). I don’t intend this as an attack on the earlier alignment community, I just think we’re making empirical progress in a way that was pretty much impossible before we had good models available to study and I place a lot more value on this.

Tom McGrath 16 Sep 2023 19:56 UTC
9 points
1 ∶ 0
in reply to: Rafael Harth’s comment on: AI Pause Will Likely Backfire
On the last part of your comment—if AGI doesn’t come out of LLMs then what would the justification for a pause be?

Tom McGrath 16 Sep 2023 19:43 UTC
16 points
5 ∶ 0
in reply to: Rafael Harth’s comment on: AI Pause Will Likely Backfire
I’m not sure what one is supposed to do with a claim that can’t be empirically tested—do we just believe it/act as if it’s true forever? Wouldn’t this simply mean an unlimited pause in AI development (and why does this only apply to AI)?

Tom McGrath 16 Sep 2023 19:37 UTC
5 points
0 ∶ 1
in reply to: Zach Stein-Perlman’s comment on: What’s in a Pause?
Ah, seems like my original comment was unclear. I was objecting to the conjunction between (a) AI systems already have the potential to cause harm (as evidenced by the Weapons of Math Destruction, Nature, and MegaSyn links) and (b) a pause in frontier AI would reduce harms. The potential harms due to (a) wouldn’t be at all mitigated by (b) so I think it’s either a bit confused or misleading to link them in this article. Does that clarify my objection?
In general I’m quite uncomfortable by the equivocation I see in a lot of places between “current models are actually causing concrete harm X” and “future models could have the potential to cause harm X” (as well as points in between, and interchanging special-purpose and general AI). I think these equivocations particularly harms the debate on open-sourcing, which I value and feels especially under threat right now.

Tom McGrath 16 Sep 2023 10:40 UTC
4 points
1 ∶ 0
on: What’s in a Pause?
I appreciate the way this post adds a lot of clarity and detail to pause proposals. Thanks for writing it and thanks also to the debate organisers.

However, I think you’re equivocating kind of unhelpfully between LLM development—which would be presumably affected by a pause—and special-purpose model development (e.g. the linked MegaSyn example) which would probably not be. This matters because the claim that AI is currently an emergency and harms are currently occurring. For a pause to prevent these harms, they would have to be from cutting-edge LLMs, but I’m not aware of any compelling examples of such.