I'm pretty sure the わけでもない here applies to both 娯楽を無駄だと言って and 一切封じられていた.
I don't know the grammatical mechanisms behind it well enough to give you an argument as to why using that sort of information, but if you look for translated example sentences that also have that construction, all of them include the clause with て as being included in わけでもない
For example, from here. I've omitted surrounding information to make it easier to read. Roughly corresponding parts are bolded.
 are not racing young Americans to the bottom.
 I didn’t truly enjoy studying English at first.
Also note that in the second sentence the part with 〜て is necessary for わけではない. 英語を勉強していたわけではありません would be a false statement
That's not to say that 〜て〜わけでもない or わけではない never excludes the 〜て in わけでもない, but it certainly is a possibility that it includes it. There are certain scenarios where it is excluded more than included.
When you look at situations in which 〜て isn't included specifically for 言って, it seems to mostly be when the statement before 言って is opposing the statement after it from the point of view of the speaker in one of two constructions. I used Google search tools to check for sentences containing "と言って*わけでもない." There are several categories to those: those that are と言っているわけでもない (accidental results), those that contain かと言って or からと言って, and those that contain だと言って, [adjective]と言って, or [verb]と言って. I then searched for each individually, with the main considerations here being that Google refuses to show all of the results it display as existing, and therefore the sample size is low. (And also that there's no easy way to search for [adjective]と言って, or [verb]と言って)
かと言って and からと言って are often listed as grammar points in their own right and both generally indicate that the clause before them opposes the clause after in a sort of "even if this is is the case" way. Listed below are some examples with loose translations I've provided that I think can be assumed to be so from context.
Just because (you)'re slow doesn't mean that's a bad thing.
You're not going to die from being incontinent.
Note: neither of these appear on the actual websites but do appear in the previews for whatever reason
It has a good feeling of weight to it, but despite that it's not heavy
It's like slurping soba through thinned shoyu, but that's not to say it's bad
だと言って (only contained one result)
It does appear almost like the speaker is saying "just because someone is pretty doesn't mean they're wanted," but within the context of the post (she is referring to literally being told she is pretty) + the use of だと言って and not かと言って or からと言って, it seems like it's actually "it's not like you call me pretty and you want me." The fact that she was called pretty is known, so the わけでもない has to not apply to キレイだと言って. However, following the logic of the first examples, in a different context it should also be able to apply to both statements. I think it can't be determined whether or not キレイだと言う is something that actually happened without more information.
To summarize this unnecessarily longwinded answer, 漫画やゲームといった娯楽を無駄だと言って封じられていた as a whole (the saying AND the preventing put together) 100% did not happen, but the sentence doesn't have any concrete information on whether or not 漫画やゲームといった娯楽を無駄だと言って happened or not.