I think 「さすがに」 generally means something like "Generally acknowledging a fact, but still has more to say regarding that fact in this specific situation". But I recently encountered a usage of this phrase, to which I don't quite think the above explanation is applicable.
To provide the context first, there was a male and a girl making a conversation. The male stood on a stone which was much higher than the place where the girl stood. So in the middle of their conversation, the girl said the following to the male:
「いつまで上から見下ろしているつもり?」.
Then the reaction of the male following this remark is:
「たしかに」
下から覗くよりましだろ、という下品で意味のない冗談はさすがに口にはせず、静馬はそそくさと地上へと降りた。
I think I get the rough meaning of these paragraphs, that is the male (whose name is 「静馬」) accepted the girl's protest and climbed down from the stone so that the two were on the same footing. But at the same time the male came up with a retort, which he decided was rather indecent. At last he refrained from spitting it out. But I don't really understand how「さすがに」 could fit into this paragraph…