It strikes me that making it easier to change contracts ex post could make the long run situation worse. If we develop AGI, one agent or group is likely to become dramatically more powerful in a relatively short period of time. It seems like it would be very useful if we could be confident they would abide by agreements they made beforehand, in terms of resource sharing, not harming others, respecting their values, and so on. The whole field of AI alignment could be thought of as essentially trying to achieve this inside the AI. I was wondering if you had given any thought to this?
I think it’s not quite right to say that anyone is “changing” the contracts. The more accurate way, in my mind, is that parts of the most concrete contents of performance obligations (“what do I have to do to fulfill my obligations?”) is determined ex post via flexible decision procedures that can account for changed circumstances. Thus I think “settling” is more accurate than “changing,” since the later implies that the actual performance was unsatisfactory of the original contract, which is not true.
You’re right that there are interesting parallels to the AI alignment problem. See here.
There are two considerations that need to be balanced in any case of flexibility: the expected (dis)value of inflexible obligations and the expected (dis)value of flexible obligations. A key input to this is the failure mode of flexible obligations would include something like the ability of a powerful obligor to take advantage of that flexibility. In some cases that will be so large that ex post flexibility is not worth it! But in other cases, where inflexibility seems highly risky (e.g., because we can tell it depends on a particularly contingent assumption about the state of the world that is unlikely to hold post-AGI) and sufficiently strong ex post term-settling procedures are available, it seems possibly worthwhile.
Thanks for sharing this here.
It strikes me that making it easier to change contracts ex post could make the long run situation worse. If we develop AGI, one agent or group is likely to become dramatically more powerful in a relatively short period of time. It seems like it would be very useful if we could be confident they would abide by agreements they made beforehand, in terms of resource sharing, not harming others, respecting their values, and so on. The whole field of AI alignment could be thought of as essentially trying to achieve this inside the AI. I was wondering if you had given any thought to this?
Thanks for your thoughts!
I think it’s not quite right to say that anyone is “changing” the contracts. The more accurate way, in my mind, is that parts of the most concrete contents of performance obligations (“what do I have to do to fulfill my obligations?”) is determined ex post via flexible decision procedures that can account for changed circumstances. Thus I think “settling” is more accurate than “changing,” since the later implies that the actual performance was unsatisfactory of the original contract, which is not true.
You’re right that there are interesting parallels to the AI alignment problem. See here.
There are two considerations that need to be balanced in any case of flexibility: the expected (dis)value of inflexible obligations and the expected (dis)value of flexible obligations. A key input to this is the failure mode of flexible obligations would include something like the ability of a powerful obligor to take advantage of that flexibility. In some cases that will be so large that ex post flexibility is not worth it! But in other cases, where inflexibility seems highly risky (e.g., because we can tell it depends on a particularly contingent assumption about the state of the world that is unlikely to hold post-AGI) and sufficiently strong ex post term-settling procedures are available, it seems possibly worthwhile.