I think the issue here is that you are explicitly trying to map the English word "look(ed)" into a Japanese verb, when you don't really need to. Learn to think in Japanese, don't (always) try to translate your thoughts from English.
So instead of trying to find a verb, you can just say that their clothes/style/whatever were cool.
If you really want to use 見える, the correct grammar would be かっこよく見えた. But to me 1) that feels like something you just wouldn't say to someone, and 2) かっこいい and 見える are like different levels of familiarity, so I don't know that they'd be used together, even if talking about the person to someone else.