赤髭 refers to a fictional, ideal doctor who lived in a downtown of Edo, though there could be perhaps some real models. He took a good care of poor people living in the slums. He didn’t receive doctor’s fee from the needy.
He didn’t mind to visit and treat sick persons anytime whenever at any place wherever in their need. Not only being a good-hearted, lighthearted and merciful man with sense of justice, he was also a master of martial art. He fought resolutely against power and injustice, and busted on rascals and punished those who bully the weak.
There are various versions of 赤髭譚 – Akahige stories in novels, films, and TV drama series. 赤髭 is an ideal character of a man, and there’s no derogatory sense in the word, 赤髭 sui generis.
I suspect the O.P.'s been confusing 赤髭 with 赤毛, which certainly means “westerner(s)” and has a derogatory tone.
We also have the similar expression, 紅毛人, of which literal translation is a man with red hair. Both words, 赤毛 and 紅毛人 were invented in the late Edo era when Japanese first encountered the so-called 黒船 ‐ the Black ship and their crew led by the Captain Mathew Perry in 1853. The words were current until early 19 century. But both words are completely obsolete today, together with 毛唐、a derogatory description of westerners.