I want to register a gripe: when Eliezer says that he, Demis Hassabis, and Dario Amodei have a good “track record” because of their qualitative prediction successes, Jotto objects that the phrase “track record” should be reserved for things like Metaculus forecasts.
But when Ben Garfinkel says that Eliezer has a bad “track record” because he made various qualitative predictions Ben disagrees with, Jotto sets aside his terminological scruples and slams the retweet button.
I already thought this narrowing of the term “track record” was weird. If you’re saying that we shouldn’t count Linus Pauling’s achievements in chemistry, or his bad arguments for Vitamin C megadosing, as part of Pauling’s “track record”, because they aren’t full probability distributions over concrete future events, then I worry a lot that this new word usage will cause confusion and lend itself to misuse.
As long as it’s used even-handedly, though, it’s ultimately just a word. On my model, the main consequence of this is just that “track records” matter a lot less, because they become a much smaller slice of the evidence we have about a lot of people’s epistemics, expertise, etc. (Jotto apparently disagrees, but this is orthogonal to the thing his post focuses on, which is ‘how dare you use the phrase “track record”’.)
But if you’re going to complain about “track record” talk when the track record is alleged to be good but not when it’s alleged to be bad, then I have a genuine gripe with this terminology proposal. It already sounded a heck of a lot like an isolated demand for rigor to me, but if you’re going to redefine “track record” to refer to a narrow slice of the evidence, you at least need to do this consistently, and not crow some variant of ‘Aha! His track record is terrible after all!’ as soon as you find equally qualitative evidence that you like.
This was already a thing I worried would happen if we adopted this terminological convention, and it happened immediately.
Very many thanks for your responses Rob, you’ve helped me update. (And hopefully others who might have had thoughts kinda similar to mine will view your responses and update accordingly:))
I want to register a gripe: when Eliezer says that he, Demis Hassabis, and Dario Amodei have a good “track record” because of their qualitative prediction successes, Jotto objects that the phrase “track record” should be reserved for things like Metaculus forecasts.
But when Ben Garfinkel says that Eliezer has a bad “track record” because he made various qualitative predictions Ben disagrees with, Jotto sets aside his terminological scruples and slams the retweet button.
I already thought this narrowing of the term “track record” was weird. If you’re saying that we shouldn’t count Linus Pauling’s achievements in chemistry, or his bad arguments for Vitamin C megadosing, as part of Pauling’s “track record”, because they aren’t full probability distributions over concrete future events, then I worry a lot that this new word usage will cause confusion and lend itself to misuse.
As long as it’s used even-handedly, though, it’s ultimately just a word. On my model, the main consequence of this is just that “track records” matter a lot less, because they become a much smaller slice of the evidence we have about a lot of people’s epistemics, expertise, etc. (Jotto apparently disagrees, but this is orthogonal to the thing his post focuses on, which is ‘how dare you use the phrase “track record”’.)
But if you’re going to complain about “track record” talk when the track record is alleged to be good but not when it’s alleged to be bad, then I have a genuine gripe with this terminology proposal. It already sounded a heck of a lot like an isolated demand for rigor to me, but if you’re going to redefine “track record” to refer to a narrow slice of the evidence, you at least need to do this consistently, and not crow some variant of ‘Aha! His track record is terrible after all!’ as soon as you find equally qualitative evidence that you like.
This was already a thing I worried would happen if we adopted this terminological convention, and it happened immediately.
</end of gripe>
Very many thanks for your responses Rob, you’ve helped me update. (And hopefully others who might have had thoughts kinda similar to mine will view your responses and update accordingly:))