Frankly speaking I cannot clearly understand your point or problem, but I would like to try my best within my understanding. At first sight the two sentences in the first pair seemed to me almost equivalent. After some time of staring at these sentences, I have found a difference, as you have.
- 日本へ行くとき(に)、日本語を勉強しました (beforehand and simultaneously)
- 日本へ行くまえに、日本語を勉強しました (beforehand)
But to emphasize 'simultaneousness' you should say '日本に行っているときに、日本語を勉強しました'. And the two in the second pair are also slightly different in the above viewpoint.
- 日本へ行ったとき、日本語を勉強しました。(beforehand and simultaneously)
- 日本へ行ったあと(で)、日本語を勉強しました。(simultaneously and afterward [after arriving in Japan or after having visited Japan])
I have no definite idea of the causes which lead to these slight differences, even though they may look serious to non-natives. I am not sure, but they may arise not because of semantic differences of まえ and あと, but because of the wide coverage of the meanings of 行く and とき, or the tense discrepancy of 行く and 行った. I have been very much confused before my PC, ending up to say that I am so sorry when what I have said as an answer has missed your point.