When leaving before a senior colleague does, people say お先に失礼します, and the senior employee responds お疲れ様でした or ご苦労様でした. お疲れ様でした is also usually said when a colleague of the same seniority is leaving.
However, お疲れ様でした seems like something to say to someone of equal or lower seniority while ご苦労様でした is for someone with lower seniority. It feels awkward (only my opinion, but please feel free to correct if this is a wrong notion) to say お疲れ様でした to someone of higher ranking.
If a senior colleague is leaving, what is the best thing to say?