I think you need to look at more than just the English translations to get the way わりに is used.
くせに can be used as a somewhat ruder replacement of のに, so it makes sense to say that here. (の there is a nominalizer with no particular meaning of its own whereas くせ has meaning).
In contrast, わりに used to be written as 割に which helps make clearer what it does.
Specifically, it it says "As much/little as X, [unwritten: one might infer Y], but surprisingly Z.
I think the problem for the example sentence is 男 is not a proportional fact X of the relevant sort.
All of the examples being offered online are like:
年の割には若く見える. here also here
= he looks young if you take in account his age [which should make him look older]
= as much as she says she has no money, she seems to travel a lot.
the basic pattern is we need some sort of proportional claim and then a second fact to the contrary.
You don't have that in the sentence above, so it can't be わりに
I think when they're saying "unexpected feeling" they mean that the expression includes some opprobrium. As in we're annoyed at this outcome. (cf.
= even though she doesn't have any money she often travels. (perhaps because she collects airline miles)