Combining Thomas's answer with Marcus's comment on the question, I think I've got it, but neither response is really complete on its own. I'd like to position the answer in the way that I'm thinking about it, so that I can (1) make sure this is correct, and (2) help those who might think like me.
It's all about whether the noun feels like a direct object or a topic. If it's something particular, a specific instance of an object, directly being acted upon by a transitive verb, that's obviously a case for using を. "This song", "those people", "the red car", etc.
If, on the other hand, the noun is more nebulous, not easily enumerated, it might feel more like a topic.
OK, example time.
その歌{うた}を聞{き}いたこと・・・
the experience of listening to that particular song
を is the natural particle here because we're doing something to a particular object (that song). Contrast and compare with the following sentences involving 歌...
その歌{うた}を聞{き}いた。
listened to that song
Using を here is fine and natural. It's the normal phrasing.
その歌{うた}は、聞{き}いた。
regarding that song, (I have) listened to it
Using は here is grammatical, but would be odd to say out of context. Imagine that two people are going through a bunch of songs. "concerning that song... [pause while thinking]... I've heard it." It sounds fine in that context.
歌{うた}は聞{き}いたこと・・・
the experience of listening to songs (in general)
Without a particular song in mind, 歌 can refer to songs in general. In this case, は can work since "songs in general" feels like a topic. Or maybe more importantly, "songs in general" isn't a direct object you can apply a transitive verb to.
その歌{うた}は聞{き}いたこと・・・
I'm not sure how you'd translate this; it sounds odd to me. It sounds to me like you'd separate it along the lines of: "concerning this song" ... "the experience of having listened to it". The は almost forces a pause, for me.
It's kind of a roundabout way of getting there, but I think this is what Marcus meant when he said 「こんなに大きな・・・」 is a setup for は. "a peach as big as this" is more suited as a topic of a sentence. It's not referring to any one peach, it's referring to a class of peaches.
こんなに大{おお}きな桃{もも}をみた
Essentially, this sounds strange because を isn't attached to an object or set of objects.