It seems as though だろう is expressing something that is outside of what I've normally seen it used for (e.g. as a substitute for でしょう to mean "probably")
Japanese Language Stack Exchange is a question and answer site for students, teachers, and linguists wanting to discuss the finer points of the Japanese language. It only takes a minute to sign up.Sign up to join this community
This だろう does have the function of inference, "I suppose" or "probably will ～". The subject of 侮っていた is 私, but the subject of 行動しないだろう can be either 私 or someone else who has been mentioned in the previous context. と is just a quotative particle.
I think [someone] will not act right away.
I had been under the impression that [they/I] (probably) would not act right away.