Is it based on the particle connecting the object of the verb?
Is it purely based on the context of the sentence for distinguishing between
to come, and
to visit (source)?
According to the dictionary, the word 伺う has 5 possible meanings, many of them using を.
So you have to look at the context to find the right meaning.
Looking only at the particle may not be enough. Some sentences may not even have a particle.
But if you have:
it will most certainly be "to visit".