A few weeks later, the article writer pens a triumphant follow-up about how well the whole process went and offers to do similar work for a high price in the future.
I don’t really want to do more of this kind of work. Our civilization is hurtling toward extinction by building increasingly capable, general, and unalignable ML systems, and I hope to do something about that. Still, I’m open to trades, and my guess is that if you wanted to pay Lightcone around $800k/year, it would be worth it to continue having someone (e.g. me) do this kind of work full-time. I guess if anyone thinks that that’s a good trade, they should email me.
I understand that you’re taking liberties for your allegory in imagining a “triumphant follow-up”, but I think it’s worth being clear that the actual follow-up all but states that this was a miserable experience and not worth the time and effort:
I did not work on this post because it was easy. I worked on it because I thought it would be easy. I kept wanting to just share what I’d learned. I ended up spending about ~320 hours (two months of work), over the span of six calendar months, to get to a place where I was personally confident of the basic dynamics (even though I expect I have some of the details wrong), and that Alice and Chloe felt comfortable with my publishing.
On June 15th I completed the first draft of the post, which I’d roughly say had ~40% overlap in terms of content with the final post. On Wednesday August 30th, after several more edits, I received private written consent from both Alice and Chloe to publish. A week later I published.
I worked on this for far too long. Had I been correctly calibrated about how much work this was at the beginning, I likely wouldn’t have pursued it. But once I got started I couldn’t see a way to share what I knew without finishing, and I didn’t want to let down Alice and Chloe.
The high price communicates how little Ben wants to do this again, not that he thinks he did something that others should value at that price.
Someone can be at once exhausted and triumphant after completing a major, complicated, lengthy process, and that combination of tones is what I took from the post. See eg:
When I saw on Monday that Chloe had decided to write a comment on the post, I felt a sense of “Ah, the job is done.” That’s all I wanted.
Just to provide the full quote, since I think it’s kind of confusing standing on its own:
I worked on this for far too long. Had I been correctly calibrated about how much work this was at the beginning, I likely wouldn’t have pursued it. But once I got started I couldn’t see a way to share what I knew without finishing, and I didn’t want to let down Alice and Chloe.
My goal here was not to punish Nonlinear, per se. My goal was to get to the point where the accusations I’d found credible could be discussed, openly, at all.
When I saw on Monday that Chloe had decided to write a comment on the post, I felt a sense of “Ah, the job is done.” That’s all I wanted. For both sides to be able to share their perspective openly without getting dismissed, and for others to be able to come to their own conclusions.
I have no plans to do more investigations of this sort. I am not investigating Nonlinear further. If someone else wants to pick it up, well, now you know a lot of what I know!
I think this is meant to be the fantasy version of Closing Notes on Nonlinear Investigation, where Ben writes,
I understand that you’re taking liberties for your allegory in imagining a “triumphant follow-up”, but I think it’s worth being clear that the actual follow-up all but states that this was a miserable experience and not worth the time and effort:
The high price communicates how little Ben wants to do this again, not that he thinks he did something that others should value at that price.
Someone can be at once exhausted and triumphant after completing a major, complicated, lengthy process, and that combination of tones is what I took from the post. See eg:
Just to provide the full quote, since I think it’s kind of confusing standing on its own: