「いや あいつがさ 不可視境界線{ふかしきょうかいせん}だっけ... それを探{さが}すって聞{き}かなくってさ... 付{つ}き合{あ}ってただけだって。」
Grammar first.
Main clause: 「付き合ってただけだって」
Sub-clause: 「いや あいつがさ 不可視境界線だっけ... それを探すって聞かなくってさ... 」
The subject of the sentence, naturally, is either mentioned or unmentioned in the main clause, agreed? The subject here is the unmentioned speaker.
The main clause means "I was only keeping あいつ company." It ends in a somewhat emphatic quotative particle 「って」. What is left unsaid after that would be something like "I'm telling you", "trust me", etc. If you are already reading something like this, I am sure you have encountered many sentences ending with quotative particles before.
Next, vocabulary.
You totally misinterpreted 「聞かなくってさ」 by guessing its meaning as "I didn't hear it".
「聞かない」 here means "(someone is) so stubborn (that he won't take any advice)". It is a very common word (to describe how kids are, especially). 「あいつ」 is the 聞かない guy here, not the speaker.
「探すって聞かない」 thus means "he is stubbornly determined to look for (it)"
「不可視境界線」 seems to be a newly coined word (for manga/anime?) that means "Ethereal Horizon". As a non-otaku, I will just leave it at that.
All in all, the sentence would mean along the lines of:
"You know, he was so stubborn with the idea of searching for the.. whatcha call it, the Ethereal Horizon? - so, I was just keeping him company."
If I were a Japanese-learner (which is a strange thing to say because I will never be one), I would try to look for the main verb of the sentence and work my way from there. Here it is 「付き合ってた」. And then, I will try to make sure I am getting the correct meaning of every word used.