This is a variation on an idiom, あさっての方を向く, which means you're focusing on the wrong thing or failing to notice what you should. The key to the idiom is あさって. Literally, it comes from the idea that you're focusing on the day after tomorrow when you should be paying attention to tomorrow!
The idiom, though, is now more general than that, and it can be used without a verb like 向く. In fact, the only strictly necessary part of the idiom is あさって itself!
Phrases like あさっての方向 mean something like "the wrong way (without realizing)", and that "way" can be either literal or figurative. If it's literal, it could be used in phrases meaning going the wrong way, looking the wrong way, or aiming something the wrong way. If it's figurative, it could mean missing the point, talking about the wrong subject, or just failing to notice something. It can even mean feigning ignorance if you're doing it on purpose―close to the English phrase "looking the other way".
To figure out the exact meaning, you'll have to pay attention to context. In your example, it looks like the basic figurative use:
この話があさっての方向に向かっています
In this case, I think your explanation was pretty good―they were saying something "off-topic", right? They were focusing on the wrong thing (figuratively on あさって, not on あした, the thing they should have been focusing on). The speaker was pointing that out. Depending on context, a phrase like "That's besides the point" or "You're barking up the wrong tree" could be appropriate as a translation.
For reference, here is the definition from the 明鏡ことわざ成句使い方事典:
あさってをむく【明後日を向く】
〔使い方〕全く見当違いの方を向く。
「あさってを向いていて、前の車に追突してしまった」
「みんながあさってを向いていて、議論がかみ合わない」
「銃口があさってを向いているから的には当たらない」
◆①あしたに目を向けるべきなのに、あさっての方を見ている
意からいう。「あさっての方を向く」とも。
②「そっぽを向く」は、意図的に見るべき方向から目をそらす
意だが、「あさってを向く」には意図が働かない。
〔誤用〕①協調しない態度をとる意で使うのは誤り。「×いくら説得して
も、あさってを向いていて相手にしない」「×厳しく管理し過ぎ
て、部下にあさってを向かれる」
②「あさって」と「みょうごにち(明後日)」は同意だが、「みょ
うごにちを向く」は誤り。