To clarify, you think that “buying time” might have a negative impact [on timelines/safety]?
Even if you think that, I think I’m pretty uncertain of the impact of technical alignment, if we’re talking about all work that is deemed ‘technical alignment.’ e.g., I’m not sure that on the margin I would prefer an additional alignment researcher (without knowing what they were researching or anything else about them), though I think it’s very unlikely that they would have net-negative impact.
So, I think I disagree that (a) “buying time” (excluding weird pivotal acts like trying to shut down labs) might have net negative impact and that & thus also that (b) “buying time” has more variance than technical alignment.
edit: Thought about it more and I disagree with my original formulation of the disagreement. I think “buying time” is more likely to be net negative than alignment research, but also that alignment research is usually not very helpful.
Rob puts it well in his comment as “social coordination”. If someone tries “buying time” interventions and fails, I think that because of largely social effects, poorly done “buying time” interventions have potential to both fail at buying time and preclude further coordination with mainstream ML. So net negative effect.
On the other hand, technical alignment does not have this risk.
I agree that technical alignment has the risk of accelerating timelines though.
But if someone tries technical alignment and fails to produce results, that has no impact compared to a counterfactual where they just did web dev or something.
My reference point here is the anecdotal disdain (from Twitter, YouTube, can dm if you want) some in the ML community have for anyone who they perceive to be slowing them down.
I see! Yes, I agree that more public “buying time” interventions (e.g. outreach) could be net negative. However, for the average person entering AI safety, I think there are less risky “buying time” interventions that are more useful than technical alignment.
I think probably this post should be edited and “focus on low risk interventions first” put in bold in the first sentence and put right next to the pictures. Because the most careless people (possibly like me...) are the ones that will read that and not read the current caveats
You’d be well able to compute the risk on your own, however, if you seriously considered doing any big outreach efforts. I think people should still have a large prior on action for anything that looks promising to them. : )
If Buying time interventions are conjunctive (ie. one can cancel out the effect of the others); but technical alignment is disjunctive
If the distribution of people performing both kinds of intervention is mostly towards the lower end of thoughtfulness/competence, (which we should imo expect)
Then technical alignment is a better recommendation for most people.
In fact it suggests that the graph in the post should be reversed (but the axis at the bottom should be social competence rather than technical competence)
To clarify, you think that “buying time” might have a negative impact [on timelines/safety]?
Even if you think that, I think I’m pretty uncertain of the impact of technical alignment, if we’re talking about all work that is deemed ‘technical alignment.’ e.g., I’m not sure that on the margin I would prefer an additional alignment researcher (without knowing what they were researching or anything else about them), though I think it’s very unlikely that they would have net-negative impact.
So, I think I disagree that (a) “buying time” (excluding weird pivotal acts like trying to shut down labs) might have net negative impact and that & thus also that (b) “buying time” has more variance than technical alignment.edit: Thought about it more and I disagree with my original formulation of the disagreement. I think “buying time” is more likely to be net negative than alignment research, but also that alignment research is usually not very helpful.
Rob puts it well in his comment as “social coordination”. If someone tries “buying time” interventions and fails, I think that because of largely social effects, poorly done “buying time” interventions have potential to both fail at buying time and preclude further coordination with mainstream ML. So net negative effect.
On the other hand, technical alignment does not have this risk.
I agree that technical alignment has the risk of accelerating timelines though.
But if someone tries technical alignment and fails to produce results, that has no impact compared to a counterfactual where they just did web dev or something.
My reference point here is the anecdotal disdain (from Twitter, YouTube, can dm if you want) some in the ML community have for anyone who they perceive to be slowing them down.
I see! Yes, I agree that more public “buying time” interventions (e.g. outreach) could be net negative. However, for the average person entering AI safety, I think there are less risky “buying time” interventions that are more useful than technical alignment.
I think probably this post should be edited and “focus on low risk interventions first” put in bold in the first sentence and put right next to the pictures. Because the most careless people (possibly like me...) are the ones that will read that and not read the current caveats
You’d be well able to compute the risk on your own, however, if you seriously considered doing any big outreach efforts. I think people should still have a large prior on action for anything that looks promising to them. : )
An addendum is then:
If Buying time interventions are conjunctive (ie. one can cancel out the effect of the others); but technical alignment is disjunctive
If the distribution of people performing both kinds of intervention is mostly towards the lower end of thoughtfulness/competence, (which we should imo expect)
Then technical alignment is a better recommendation for most people.
In fact it suggests that the graph in the post should be reversed (but the axis at the bottom should be social competence rather than technical competence)