Why is the bold part passive form? How can you be worn by uniform?
It is a rhetorical use of passive voice. 服に着られる basically means the person in the clothes gives a funny impression, e.g. the size is too big, it simply does not go well, etc.
(Added) It is less common than the two below, but e.g. 制服に着られている is typically used for a boy who entered junior high school ( ≒ 13 years old), for which it is first required to wear uniforms (near Tokyo). It means the uniform does not fit the boy, not just physically but also in the sense that the overall impression does not feel right somehow.
There are not many verbs that are used in this way, but two other typical cases are:
酒は飲んでも飲まれるな is a common phrase which literally means even if you drink alcohol, don't be drunken by it. Possibly this makes some sense in English as well. Basically it means it's ok to drink, but not too much.
This is even more literal. The latter means to be used by money, which basically means that someone is worried too much about money and sort of manipulated by money itself.
So the basic connotation of this kind of passive is that the person does not have control over the object (服, 酒, 金) which s/he should have.