This is a question that I have found in a JLPT-N5 mock test paper.
弟はへや___そうじをしました。
And the answer for that is の, but I selected を. Could someone possibly clarify this please? Why の and why not を? What is wrong if one puts を there?
You cannot have two を in a single clause. The structure [Noun]を[Verb] can be realised as:
Where in そうじする is the verb in 1, and する is the verb in 2
Also note that in 2, そうじ is syntactically a noun and へやのそうじ is syntactically a noun, as opposed to そうじする which is a verb in 1.
See also these:
Related Answer: snailplane explaning that you cannot have two を in a single clause
Related Question: Can we optionally include (or exclude) an を particle in between the noun of the する-verb and the する itself?
Related Question: How do I add an object to noun を します verbs?
Related Question: Difference between [Noun]する vs [Noun]をする vs [Noun]だ
It is probably because in this case the word 掃除{そうじ} is a not a verb but a noun. In fact, you can see it is followed by the particle を.
It's a small but very important thing to notice. If that particle wasn't there it would be indeed a (suru-) verb and your answer would be correct.
To elaborate as you asked, the point is the following:
The word 掃除{そうじ} is by itself a noun, but it can become a verb if used in combination with する (so called suru-verbs).
If you use it as a verb you could say: 弟はへやをそうじしました and you could translate this as "My little brother cleaned the room" (the verb is cleaning).
The particle を instead marks the object of the sentence (replying to the question the younger brother (subject) did what? Cleaning (object)). Hence そうじをしました means that the cleaning is the object of the sentence. Object of what? Of the verb "to do" (しました). Referred to what? What is that is being cleaned? The room. So it makes sense to use の to specify that we are talking of the cleaning "of the room".
So in this case you would translate 弟はへやのそうじをしました as My little brother did the cleaning of the room (here the verb is to do and cleaning is a noun).
So finally why is wrong to put two を? Because you would have two objects and it would not be clear to what the verb refers to. It would be like "My brother did..." what? The cleaning or the room (as if he created a room)? In other words, there would be no clear connection between subject object and verb and the grammatical structure of the sentence falls. In short, you can't have two を in a single clause, as you said.