Why does the expression "impossible for me" in Japanese have 私には instead of just 私は?
I think you've answered it yourself: に gives the sense of "for".
Consider "For me, it's impossible" vs. "Me, I'm [figuratively] impossible."
私は無理です may be colloquially passable, but is logically incorrect, because you are saying “I am impossible” or “I am impossibility.” Clearly you are not “impossible” entity. 私には無理です means （そのことを行うのは）私には(私にとっては) 無理（なこと）です‐“It’s impossible (or difficult) for me (to do it), “ and sounds perfect to me.
Japanese language is pretty loose in distinguishing noun and adjective in use as compared with other languages. We say 彼は正直(潔白)です. But you don’t say he is honesty (innocence) in English. With that said, I think 私は無理です is still awkward.
I am actually not sure if "僕は無理です” is technically incorrect grammar. I can imagine a group of people talking about whether that would ever climb a mountain, and one of them says "僕は。。。。やっぱり無理です”. As "〜は” can mean "as for ~", I don't see a major difference in meaning between 僕は and 僕には in this case.
I would be interested to hear from any native speakers to confirm this, however.