Not sure if this question is opinion based, but I think it's worth trying to ask.
Can native/fluent speakers easily distinguish the sound difference between the polite and potential forms of a verb in normal speech e.g. 行きます versus 行けます?
Is it perhaps the change of particle from を to が or the context the conversation that helps to determine the meaning rather than the pronunciation of the verb? Are there ever situations where this is confusing and you have to ask the speaker to repeat or rephrase what they said?