It’s also important to address the deeper assumption here: that I am somehow morally or legally obligated to permanently align my actions with the preferences or ideological views of past philanthropic funders who supported an organization that employed me. That notion seems absurd. It has no basis in ordinary social norms, legal standards, or moral expectations. People routinely change roles, perspectives evolve, and institutions have limited scopes and timelines. Holding someone to an indefinite obligation based solely on past philanthropic support would be unreasonable.
I don’t think a lifetime obligation is the steelmanned version of your critics’ narrative, though. A time-limited version will work just as well for them.
In many circumstances, I do think society does recognize a time-limited moral obligation and social norm not to work for the other side from those providing you significant resources,[1] --although I am not convinced it would in the specific circumstances involving you and Epoch. So although I would probably acquit you of the alleged norm violation here, I would not want others drawing larger conclusions about the obligation / norm from that acquittal than warranted.[2]
There is something else here, though. At least in the government sector, time-limited post-employment restrictions are not uncommon. They are intended to avoid the appearance of impropriety as much as actual impropriety itself. In those cases, we don’t trust the departing employee not to use their prior public service for private gain in certain ways. Moreover, we recognize that even the appearance that they are doing so creates social costs. The AIS community generally can’t establish and enforce legally binding post-employment restrictions, but is of course free to criticize people whose post-employment conduct it finds inappropriate under community standards. (“Traitor” is rather poorly calibrated to those circumstances, but most of the on-Forum criticism has been somewhat more measured than that.)
Although I’d defer to people with subject-matter expertise on whether there is an appearance of impropriety here, [3] I would note that is a significant lower standard for your critics to satisfy than proving actual impropriety. If there’s a close enough fit between your prior employment and new enterprise, that could be enough to establish a rebuttable presumption of an appearance.
For instance, I would consider it shady for a new lawyer to accept a competitive job with Treehuggers (made up organization); gain skill, reputation, and career capital for several years through Treehuggers’ investment of money and mentorship resources; and then use said skill and reputation to jump directly to a position at Big Timber with a big financial upside. I would generally consider anyone who did that as something of . . . well, a traitor and a sellout to Treehuggers and the environmental movement.
This should also not be seen as endorsing your specific defense rationale. For instance, I don’t think an explicit “stipulation about slowing down AI” in grant language would be necessary to create an obligation.
My deference extends to deciding what impropriety means here, but “meaningfully making use of benchmarks, datasets, or tools that were developed during [your] previous roles” in a way that was substantially assisted by your previous roles sounds like a plausible first draft of at least one form of impropriety.
At least in the government sector, time-limited post-employment restrictions are not uncommon. They are intended to avoid the appearance of impropriety as much as actual impropriety itself. In those cases, we don’t trust the departing employee not to use their prior public service for private gain in certain ways.
This is also a massive burning of the commons. It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias. It is valuable for folks to be able to share information freely with folks at such forecasting orgs without having to worry about them going off and doing something like this.
However, this only works if those less worried about AI risks who join such a collaboration don’t use the knowledge they gain to cash in on the AI boom in an acceleratory way. Doing so undermines the very point of such a project, namely, to try to make AI go well. Doing so is incredibly damaging to trust within the community.
I don’t think a lifetime obligation is the steelmanned version of your critics’ narrative, though. A time-limited version will work just as well for them.
In many circumstances, I do think society does recognize a time-limited moral obligation and social norm not to work for the other side from those providing you significant resources,[1] --although I am not convinced it would in the specific circumstances involving you and Epoch. So although I would probably acquit you of the alleged norm violation here, I would not want others drawing larger conclusions about the obligation / norm from that acquittal than warranted.[2]
There is something else here, though. At least in the government sector, time-limited post-employment restrictions are not uncommon. They are intended to avoid the appearance of impropriety as much as actual impropriety itself. In those cases, we don’t trust the departing employee not to use their prior public service for private gain in certain ways. Moreover, we recognize that even the appearance that they are doing so creates social costs. The AIS community generally can’t establish and enforce legally binding post-employment restrictions, but is of course free to criticize people whose post-employment conduct it finds inappropriate under community standards. (“Traitor” is rather poorly calibrated to those circumstances, but most of the on-Forum criticism has been somewhat more measured than that.)
Although I’d defer to people with subject-matter expertise on whether there is an appearance of impropriety here, [3] I would note that is a significant lower standard for your critics to satisfy than proving actual impropriety. If there’s a close enough fit between your prior employment and new enterprise, that could be enough to establish a rebuttable presumption of an appearance.
For instance, I would consider it shady for a new lawyer to accept a competitive job with Treehuggers (made up organization); gain skill, reputation, and career capital for several years through Treehuggers’ investment of money and mentorship resources; and then use said skill and reputation to jump directly to a position at Big Timber with a big financial upside. I would generally consider anyone who did that as something of . . . well, a traitor and a sellout to Treehuggers and the environmental movement.
This should also not be seen as endorsing your specific defense rationale. For instance, I don’t think an explicit “stipulation about slowing down AI” in grant language would be necessary to create an obligation.
My deference extends to deciding what impropriety means here, but “meaningfully making use of benchmarks, datasets, or tools that were developed during [your] previous roles” in a way that was substantially assisted by your previous roles sounds like a plausible first draft of at least one form of impropriety.
My argument for this being bad is quite similar to what you’ve written.