Let's break 予報外れの雨が降った into its two basic parts.
The core sentence is just 雨が降った, "It rained." And so any translation should reflect that this is the underlying sentence.
The second part is
予報外れ. This does mean "the forecast got it wrong". However, it's actually not a sentence; it's a noun phrase. A rendering that would better reflect this would indeed be something like "despite the forecast". Since this fragment is functioning grammatically as a noun, it needs something to connect/bind it to the core sentence. This is the の that joins the two parts. (In other words, this is the connector you asked about in your comment.) This の makes 予報外れ a modifier for 雨. Thus 予報外れの雨 is "rain which was outside the forecast".
外れ thus can be variously translated here: despite, outside, contrary to. They all capture the right meaning. But when we break the sentence apart or put it back together we may discover that some renderings into English sound better to our ear than others.
Thus 予報外れ + の + 雨が降った means
It rained despite the forecast.
In your comment, you asked "So, if I were to put it in a literal sense, would "The rain that took place was a wrong prediction of the forecast" be a decent translation?" You've flipped the structure of the sentence around. So this is not a literal rendering; it's a reinterpreted rendering. The literal meaning is the one given above and in the manga.
The rain that took place was a wrong prediction of the forecast
would correspond to something more like
降った雨は予報外れだった
(which I suspect sounds just as clunky in Japanese as the English it's being translated from.)
Notice how your "literal" rendering has flipped the sentence structure around.
Fundamentally, the meaning may be the same but, if you want to make sure your understanding of the Japanese is correct, you want to be able to structure the translation as closely as possible to how it's structured in the original language.