「別に」is being used here with「無関係な」to indicate that something is "not particularly related".
While you are correct that「別に」is often used in this way with the negative form of verbs and adjectives, here the 形容動詞 「無関係(な)」already has a negative meaning. In fact, if we were to further negate 「別に無関係」 to 「別に無関係じゃない」it would change meaning from 「not particularly related」to something like「not necessarily unrelated」.
So
俺としては、姉についてはあまり触れられたくない話題だ。別に、姉の死を思い出すとつらい…といったようなセンチメンタルとは無関係なところで、余計なことを思い出して気が重くなるからだ。
becomes
For me, the subject of my older sister isn't really one I want touched upon. It's not particularly because I get worked up about it or anything, like that it's painful for me or something to remember her death, it's because it brings back memories I don't really need brought back and gets me a little depressed.