The primary role of the vote buttons is to create the incentive gradient that determines which comments (and commenters) we get to have. This is perhaps the most powerful tool we have for incentivising some types of commentary. So I think we should practically always vote according to what comments we want to exist. On the margin, I think everyone (including you, based on your last bullet!) should vote more on usefulness.
The primary role of the vote buttons is to create the incentive gradient that determines which comments (and commenters) we get to have. This is perhaps the most powerful tool we have for incentivising some types of commentary. So I think we should practically always vote according to what comments we want to exist. On the margin, I think everyone (including you, based on your last bullet!) should vote more on usefulness.