I agree. Although I worry less about giving up (because that can lead to people working on more important causes) and more about working on the same cause with less passion on dedication, half-assing it. It reminds me of a LessWrong post The correct response to uncertainty is *not* half-speed. Ideally, under uncertainty about whether to continue working on something, you should decide what to do, and then do it with the same dedication you were working on it before. Maybe also put into your calendar a monthly reminder to consider if you should continue working on it and try not to think about it at other times.
Of course, we are human, and that can be difficult. If we think that what we are working on is less important, we might end up prioritizing other aspects of life at the expense of work more. But it’s important to remember that even if we are not working on the most important EA cause, it is still very important and can help many many people or animals. There’s no need to compare yourself to EAs that might be having even more impact most of the time, in the same way there is no need to keep comparing yourself with billionaires when trying to earn money. Let’s just all do what we can.