If spelled タイペイ, people usually read literally //taipei// instead of a long vowel today.
Basically, エイ for //eː// is an orthographical rule limited to traditional Sino-Japanese words, or those has readings systematically deriving from historical Chinese pronunciations. While タイペイ is certainly a Chinese word, it is merely an occasional transcription of a modern Mandarin word form, in that respect nothing different than recent loan words from European languages, and people just read it out straightforwardly. Edge cases do exist such as ケツメイシ, a music group named after Chinese herbal name 決明子 "sicklepod seed", which is clearly a Sino-Japanese word, is almost always pronounced //ketsɯmeiɕi//, because it sounds like but another neologism to the majority who don't know such a word.
Whether you reproduce a syllable whose original sound is like [[ej]] using a long vowel or not is completely arbitrary. For example, a PC brand acer registers themselves as エイサー, so we read as it is spelled; a cellphone brand au as エーユー, and we follow it likewise. It makes an interesting mismatch that au Pay, the digital wallet service it provides, is pronounced エーユー・ペイ.
How to read kanji word 台北 itself is up to you; either the standard on'yomi たいほく, タイペイ, or タイペー would work, like we cannot decide the sole correct pronunciation of Appalachia.
What stated above is not applied to the spelling オウ, for which most people would go with //oː// in any context (except in native 大和言葉).