From an English perspective, it might be better to think of まま as meaning something like "just as [whatever came before], with / in the state of [whatever came before]".
Examples:
- ドア開【あ】けっ放【ぱな】しのまま ("with the door left open", [something else])
- 食【た】べていないまま ("with [someone] not having eaten yet", [something else])
- 思【おも】いのまま ("just as [someone] thinks")
Adding the に as in your sample text comes across like 思【おも】いのまま[に]{●}飛【と】んだ ("[the broom] flew, just as [Harry] was thinking"). If we view the に as analogous to English preposition "in", then we might also translate that (a bit clunkily, but more directly) as "[the broom] flew, in the way that [Harry] was thinking".
As I understand it, とおり refers to following something more explicitly, as an outgrowth of the underlying verb 通【とお】る ("to pass through"), from the idea of "going through or following through [something explicitly expressed]". Meanwhile, まま to me expresses a "squishier" kind of idea, where the "just as" sense applies to something that hasn't necessarily been explicitly expressed. There are some grammatical subtleties as well, and my sense is that we couldn't use 思【おも】いのとおり (for that matter, the MS IME won't even suggest the proper kanji-less spelling of とおり when using this construction). Instead, we would have to use either 思【おも】[っ]{●}[た]{●}とおり – as in, someone thought something (past tense, completed action), and then something else happens to follow through on those thoughts — or 思【おも】うとおり – as in, someone thinks something (present tense, ongoing action), and then something else happens to follow through on that thinking.
(Note that I am not a native speaker, so this is based on my studies and experience at the remove of a second-language learner.)
