I am not sure what ごと means in this context and I am also not too sure about what's exactly the topic in this later sentence.
I would have translated it as
A.You are saying, we should abandon him?
B.Well, yes. If you look at the things objectively, if you think about the futur risks, than... Of course, you might also say, a true friend will stay by your side because of the risks.
I am not to confident in the last part of my translation... So ごと would add nothing new? Is it just a roundabout way to say risky "things"?Which kinda get's lost in translation? Another meaning for ごと that I found was "including" but I couldn't find a site that explains how you would use this kind of ごと and I am not sure if thats what the sentence is conveying.