I go back and forth between person-affecting (hedonic) consequentialism and total (hedonic) utilitarianism on about a six-monthly basis, so I sure understand what you’re struggling with here.
I think there’s a stronger intuition that can be made to argue for a person-affecting view though, which is that the idea of standing ‘outside of the universe’ and judging between world A, B & C is entirely artificial and impossible. In reality, no moral choices that impact axiological choices can be made outside of a world where agents already exist, so it’s possible to question the fundamental assumption that comparisons between possible worlds is even legitimate, rather than just comparison between ‘the world as it is’ and ‘the world as it would be if these different choices were made’. From that perspective, it’s possible to imagine a world with one singular happy individual and a world with a billion happy individuals just being literally incomparable.
I ultimately don’t buy this, as I do think divergence between axiology and morality is incoherent and that it is possible to compare between possible worlds. I’m just very uncomfortable with its implications of an ‘obligation to procreate’, but I’m less uncomfortable with that than the claim that a world with a billion happy people is incomparable to a world with a singular happy person (or indeed a world full of suffering).