This is an interesting point, and no, we’re not measuring that exactly. The main measure we use is how much we get done with the same resources: either more output for the same time (like the referrals example, roughly 3-4x for about the same time spent), or being able to take on additional work at all (like the Talent Directory Automated Matching, we have something exciting coming up at HIP that’s a mix of both).
On maintenance: if this is about changing underlying systems (tracking new data, therefore needing a new integration), that would be an addition to the manual work also. And when those systems change, that already broke the manual version. The drop-out example in the post touches 15 systems, and we did have a written list, which was often slightly out of date because the program kept evolving. What’s different now is that Claude can flag when there’s something else we might want to link, because it’s already made the same change somewhere similar, rather than us trying to remember system no 16.
You are right, measuring AI maintenance burden without accounting for maintenance that had to be done regardless of AI deployment would not get you anywhere.
Interesting you are have Claude identifying opportunities to simplify/​reuse/​streamline, this could actually mean the per system maintenance burden will go down the more systems you connect. Pretty much the opposite of what I was thinking. Thank you.
This is an interesting point, and no, we’re not measuring that exactly. The main measure we use is how much we get done with the same resources: either more output for the same time (like the referrals example, roughly 3-4x for about the same time spent), or being able to take on additional work at all (like the Talent Directory Automated Matching, we have something exciting coming up at HIP that’s a mix of both).
On maintenance: if this is about changing underlying systems (tracking new data, therefore needing a new integration), that would be an addition to the manual work also. And when those systems change, that already broke the manual version. The drop-out example in the post touches 15 systems, and we did have a written list, which was often slightly out of date because the program kept evolving. What’s different now is that Claude can flag when there’s something else we might want to link, because it’s already made the same change somewhere similar, rather than us trying to remember system no 16.
You are right, measuring AI maintenance burden without accounting for maintenance that had to be done regardless of AI deployment would not get you anywhere.
Interesting you are have Claude identifying opportunities to simplify/​reuse/​streamline, this could actually mean the per system maintenance burden will go down the more systems you connect. Pretty much the opposite of what I was thinking. Thank you.