research done by people who are trying to do something else will probably end up not being very helpful for some of the core problems.
Yeah, it’d be good to break AGI control down more, to see if there are classes of problem where we should expect indirect work to be much less useful. But this particular model already has enough degrees of freedom to make me nervous.
I think that it might be easier to assign a value to the discount factor by assessing the total contributions of EA safety and non-EA safety.
That would be great! I used headcount because it’s relatively easy, but value weights are clearly better. Do you know any reviews of alignment contributions?
… This doesn’t seem to mesh with your claim about their relative productivity.
Yeah, I don’t claim to be systematic. The nine are just notable things I happened across, rather than an exhaustive list of academic contributions. Besides the weak evidence from the model, my optimism about there being many other academic contributions is based on my own shallow knowledge of AI: “if even I could come up with 9...”
Something like the Median insights collection, but for alignment, would be amazing, but I didn’t have time.
those senior researchers won’t necessarily have useful things to say about how to do safety research
This might be another crux: “how much do general AI research skills transfer to alignment research?” (Tacitly I was assuming medium-high transfer.)
I think the link is to the wrong model?
No, that’s the one; I mean the 2x2 of factors which lead to ‘% work that is alignment relevant’. (Annoyingly, Guesstimate hides the dependencies by default; try View > Visible)
Thanks!
Yeah, it’d be good to break AGI control down more, to see if there are classes of problem where we should expect indirect work to be much less useful. But this particular model already has enough degrees of freedom to make me nervous.
That would be great! I used headcount because it’s relatively easy, but value weights are clearly better. Do you know any reviews of alignment contributions?
Yeah, I don’t claim to be systematic. The nine are just notable things I happened across, rather than an exhaustive list of academic contributions. Besides the weak evidence from the model, my optimism about there being many other academic contributions is based on my own shallow knowledge of AI: “if even I could come up with 9...”
Something like the Median insights collection, but for alignment, would be amazing, but I didn’t have time.
This might be another crux: “how much do general AI research skills transfer to alignment research?” (Tacitly I was assuming medium-high transfer.)
No, that’s the one; I mean the 2x2 of factors which lead to ‘% work that is alignment relevant’. (Annoyingly, Guesstimate hides the dependencies by default; try View > Visible)