This comes from a character who wakes up to somebody else intruding in her home and half-asleep she starts rambling stuff at the intruder seemingly out of context. I'm interested in the bolded part specifically. My attempt at a translation:
In any case, whether the actor is possessed or not, the fact that their face frightens me and is something I can't help really troubles me.
Not sure I'm on the right track here with いかんともしがたい and 痛し痒し completely throws me for a loop since as far as I know it's used with two choices in a "damned if I do, damned if I don't" kind of situation of which I can't really see any here.
Any ideas? :)