This 故【ゆえ】 is a special type of noun called 形式名詞 ("formal noun" in the sense of "technically/apparently a noun"). Other formal nouns include とき, ため, はず, etc.
Since 故 is a (formal) noun, something that modifies it must be an attributive (i.e., "noun-modifying") expression. So what are "noun-modifying" in Japanese?
- Attributive form of an adjective
- Noun + の / No-adjective + の
- Attributive form of a verb, a.k.a. a relative clause
- Lexicalized attributives (その, この, etc)
- Attributive form of a verb + が
The first four bullets should be easy if you understand 故 is technically a noun. They are all valid usages of 故に/故の.
The last one may seem tricky if you don't know what's going on. This is a special construction derived from Old Japanese. This が is actually a linking particle like の, and the verb before it is actually a nominalized verb. Please read: Why is there a が in 深淵に臨むが如し?
Since 故 is a stiff and literary expression, it still tends to be used along with が when it is modified by a verb. But other simpler usages are also perfectly possible. To put this simply, the が between the verb and 故 is optional. In your case, there is another が near 故, so I even feel the sentence sounds slightly better without が before 故.
Finally, it's not that odd to use 故の in casual conversations. Using a bit of idiomatic expressions from archaic Japanese is usually natural. See: Nuance and conversational use of なきにしもあらず