And your attempt to translate is:
"I want to say, call me if something happens, but I can't do anything if you call me wrong".
My question would be "Where are you getting 'call me wrong' from?"
「悪いが」 in that sentence means "I'm sorry, but", "unfortunately", etc.
「呼ばれても」 is in the passive voice, meaning "if I am called (by you)".
Note that I used the active voice "if you called me" instead for a more natural-sounding translation.
So, the sentence means:
"I'd like to ask/tell you to call me if something happens, but I'm sorry I wouldn't be able to do anything if you called me."