I think it matters a lot and that it's best to break from the habit if you intend to speak Japanese long-term. I think for place names it's somewhat understandable because the Kana is obviously there mimicking the native place name, sort of like how a French person (no offense) might say they are from "Paris".
However, many people will not understand an English-derived Kana word in native pronunciation and there are many Japanese words that could nominally be considered English that get chopped up and placed in other words, like...
- エアコン ("air-con" for air-conditioner)
- リモコン ("remo-con" for remote control)
... or words that just sound too different due to Japanese phonology, like
...or even words that we may have an anglicized pronunciation for, but the Japanese went a different way, like
- キシリトール (xylitol)
- ワクチン (vaccine)
In short, if you pronounce "English" Kana words natively many people will understand you much of the time, but many people will not understand you much of the time.
Just imagine that a native speaker of French or Greek spoke English fluently, but decided it was OK to pronounce all of the French or Greek-derived words in English as he would in his native area. If you're OK with being that guy then I guess it's cool, but if not...