You may be right that most of the examples of nouns following the V連用形＋物 pattern usually describe the direct object of the verb. But that is far from always the case.
One noteworthy exception that springs to mind is 生き物 (living thing). This would notionally refer to the subject of the verb (e.g. the 私 in 私が生きる), rather than a direct object, which this verb only rarely uses (e.g. 瞬間を生きる meaning 'to live in the moment').
Slightly separately but possibly helpful to your understanding, I think it is worth noting here that there are also many words following this pattern that have very specific interpretations (unlike the examples you gave). For instance, 吸い物 doesn't refer to anything that uses the verb 吸う (e.g. a cigarette); rather, only a dish of a clear broth with ingredients/garnish floating in/on it. EDIT: It seems that I may have had an slightly overly narrow view of the word 吸い物 myself. Please see comments below.
吹き出物, meaning a 'pimple' or a 'rash', is a good example of one noun which sits in the intersection of not being a direct object, and having a specific meaning. A pimple is not a direct object (taking を) of the verb, as the verb 吹き出る (meaning gush out, spout, break out) is intransitive. Similarly, 吹き出物 doesn't correspond to anything that is gushing out e.g. 水が吹き出る.
So, it's not worth, in my opinion, trying to find a strict rule here regarding it being a direct object of the verb, as opposed to some other part of speech. It's rather just that the verb plays some role in the thing involved, and the expression may convey a broad meaning (like 食べ物 or 飲み物) or a more specific one (like 吸い物).