In this context, 惚れた女 clearly means "the woman whom he fell in love with" rather than "the woman who fell in love with someone". The subject of 惚れる is 男. Grammatically, this is an innate ambiguity of Japanese relative clauses, which lack relative pronouns like whom, where, etc. You have to determine the correct meaning from the context.

• 男は女に惚れた。　 The man fell in love with the woman.

• 男が惚れた女　 the woman whom the man fell in love with
• 惚れた女　 the woman whom the man fell in love with (subject is omitted)

• 女は誰かに惚れた。 The woman fell in love with someone.

• 誰かに惚れた女　 the woman who fell in love with someone
• 惚れた女　 the woman who fell in love with someone (target is omitted)

Likewise, 紹介した人 can mean both "the person who introduced someone" and "the person whom someone introduced" depending on the context. あげた人 can mean both the giver and the receiver. Also see: Relative Clause Ambiguous

As an aside, please recheck the meaning of 無粋.

In this context, 惚れた女 clearly means "the woman whom he fell in love with" rather than "the woman who fell in love with someone". The subject of 惚れる is 男. Grammatically, this is an innate ambiguity of Japanese relative clauses, which work by moving a modified noun and removing the accompanying particle such as が/を/に/へ. You have to determine the correct meaning from the context.

• 男は女に惚れた。 
  The man fell in love with the woman.

• 男が惚れた女 
  the woman whom the man fell in love with (note that に is gone)
• 惚れた女 
  the woman whom the man fell in love with (subject is omitted)

• 女は誰かに惚れた。
  The woman fell in love with someone.

• 誰かに惚れた女 
  the woman who fell in love with someone (note that は is gone)
• 惚れた女 
  the woman who fell in love with someone (target is omitted)

Likewise, 紹介した人 can mean both "the person who introduced someone" and "the person whom someone introduced" depending on the context. あげた人 can mean both the giver and the receiver. Also see: Relative Clause Ambiguous

As an aside, please recheck the meaning of 無粋.