This is from the manga Blue Period. Meaning of the sentence aside, I also want to understand what role is なんか and なん playing here.
なんか is a word used to make light of the word before it. It's interchangeable with なんて here. なんな right after it is a contraction of なるな. See this answer for the rule.
So the sentence is basically 冷静になるな ("Don't be cool/calm") but with the implication that 冷静 is an undesirable thing to him for now. He noticed that the dispassionate and realistic analysis in the previous pages would only make him lose passion. So he started to encourage himself by saying 冷静になんかなんな.