This is one of those instances where we as English speakers encounter a term and assume that it matches its English equivalent perfectly, but actually the distinction between transitive and intransitive verbs is a little bit fuzzier than you may have been led to believe. As such this answer may not be very intuitive.
The basic point is that taking a direct object doesn't necessarily mean a verb is transitive. In this dictionary entry for 他動詞:
And the entry for 自動詞:
Basically it's saying that the line is fuzzy in Japanese while there are some clear rules in Western languages.
So what about 渡る, specifically? Why does this fall into that fuzzy category?
Check out this discussion. Examining the definitions a little closer we see that for a verb to be transitive it must be on operation on that object. When you cook rice you are acting on the rice. When you kick a ball you are acting on the ball. But when you cross a bridge, are you acting on the bridge? Are you doing something to it? Not really; the bridge just happens to be where you're walking.
Another way to look at it is to say that the を isn't marking an object but a path. You can find the same kind of misunderstanding with 歩く. When you say 道を歩く, does 歩く become a transitive verb? No, and this site can support that. At the end of the day the operating phrase seems to be "自他の対応がない."