I think what is true is probably something like “neverending process don’t exist, but arbitrarily long ones do”, but I’m not confident. My more general claim is that there can be intermediate positions between ultrafinitism (“there is a biggest number”), and any laissez faire “anything goes” attitude, where infinities appear without care or scrunity. I would furthermore claim (but on less solid ground), that the views of practicing mathematicians and physicists falls somewhere in here.
As to the infinite series examples you give, they are mathematically ill-defined without giving a regularization. There is a large literature in mathematics and physics on the question of regularizing infinite series. Regularization and renormalization are used through physics (particular in QFT), and while poorly written textbooks (particularly older ones) can make this appear like voodoo magic, the correct answers can always be rigorously be obtained by making everything finite.
For the situation you are considering, a natural regularization would be to replace your sum with a regularized sum where you discount each time step by some discounting factor γ. Physically speaking, this is what would happen if we thought the universe had some chance of being destroyed at each timestep; that is, it can be arbitrarily long-lived, yet with probability 1 is finite. You can sum the series and then take γ→0 and thus derive a finite answer.
There may be many other ways to regulate the series, and it often turns out that how you regulate the series doesn’t matter. In this way, it might make sense to talk about this infinite universe without reference to a specific limiting process, but rather potentially with only some weaker limiting process specification. This is what happens, for instance, in QFT; the regularizations don’t matter, all we care about are the things that are independent of regularization, and so we tend to think of the theories as existing without a need for regularization. However, when doing calculations it is often wise to use a specific (if arbitrary) regularization, because it guarantees that you will get the right answer. Without regularizations it is very easy to make mistakes.
This is all a very long-winded way to say that there are at least two intermediate views one could have about these infinite sequence examples, between the “ultrafinitist” and the “anything goes”:
The real world (or your priors) demands some definitive regularization, which determines the right answer. This would be the case if the real world had some probability of being destroyed, even if it is arbitrarily small.
Maybe infinite situations like the one you described are allowed, but require some “equivalence class of regularizations” to be specified in order to be completely specified. Otherwise the answer is as indeterminant as if you’d given me the situation without specifiying the numbers. I think this view is a little weirder, but also the one that seems to be adopted in practice by physicists.