I think it’s also worth observing that although St Petersburg cases are famously paradox-riddled, these cases seem overwhelmingly important on a conventional utilitarian view even before we consider any exotic hypotheses. Indeed, I personally became unhappy with unbounded utilities not because of impossibility results but because I tried to answer questions like “How valuable is it to accelerate technological progress?” or “How bad is it if unaligned AI takes over the world?” and immediately found that EU maximization with anything like “utility linear in population size” seemed to be unworkable in practice. I could find no sort of common-sensical regularization that let me get coherent answers out of these theories, and I’m not sure what it would look like in practice to try to use them to guide our actions.
May be of interest, from Paul Christiano: