Is there a colloquial phrase in Japanese that would be the same as or similar in meaning to the English phrase "caught red handed"?
The idiomatic part "red-handed" can be translated with 現行犯(で).
For "catch" you use an appropriate form of 捕まる・捕える or 逮捕（する） or 取り押さえる.
We caught the suspect red-handed.
There also appears to be a compound noun 現行犯逮捕（する） for "being caught in the act / red-handed".