I somehow think you're complicating sounds fruitlessly. It is much simpler to understand what consonants commonly sound like, when learning your kana. Once you've mastered your Kana, you'll know what sounds are possible in Japanese, when spoken normally. Other alternative approximate pronunciations happen more or less on a per-speaker basis, though a few are regional variants, but should be understood to be equivalent to the "basic" sounds you would have learned. The only part where I could understand confusion is in sounds where two kana sound similar to each other, such as だ and ら.
/xo/ and /ho/ for ほ are definitely an allophone. This "difference" in pronunciation is sometimes present due to particular speech and stress patterns that are unique to a speaker, but both are understood more or less the same. /ho/ is probably the more common sound to hear.
As for ん, this sound is neither an /n/ or /m/ consonant sound -- it is a nasal sound made with a closed mouth or upper tongue position. For that reason, depending on which way the mouth is closing after the previous sound, it can come to sound closer to /n/ or closer to /m/. This also varies by speaker.
Bouncing off what I've been reading in some of the comments, as far as the ら line consonant sounds and flapping/tapping your tongue against the roof of the mouth, it's not impossible to do when it's word-initial. I also speak Spanish and while it's true that Spanish doesn't have that sound at the beginning of a word, opting instead for the rolled "rr" sound, it is entirely possible to tap/flap your tongue at the start of the word. You just need to preempt it by opening your jaw a little more, and avoid making a vowel sound before. If you really have a really rough time with getting it right, consider using a very short clipped "u" sound until you become comfortable with the motion. The words ロシア or らいねん are good practice ones for this.