Context: A clan leader wanted to keep MC in her family/clan because she thinks MC is a skilled warrior and can use him. But her subordinate disagreed and wanted to kick MC out, since she's worried that MC could cause harm to her leader. Also this world where MC has been isekai'd to, is in chaos where war happens everywhere, so the subordinate suggested that her clan leader should be very careful to an unknow stranger like MC.

Hope someone an explain to me the correct meaning of this sentence 「……やけに他人事だな、貴様は」(i can only guess but not sure: "...You're really like to worries about someone else's business, aren't you ?" )

More context:

Clan leader「結菜。おまえはどうだ？」

Subordinate「……私はまだ反対。この者の真意が見えないわ。取り入った後、隙を見つけて悪事を働くことも考えられる」

Subordinate「母、道三が美濃でやったように、この下克上の時代、不明の者を側に置くには、細心の注意を払った方がいい」

MC「全くもってその通りだね」

Clan leader「……やけに他人事だな、貴様は」

MC「だって俺がそっちのみんなと同じ立場なら、同じ心配をしてると思うし。……なぁ、やっぱり俺、出て行った方がいいんじゃない？」