I see the words white and black but can not recall a reference to Japanese with regards to color.
黄色【おうしょく】人種【じんしゅ】 or 黄色【おうしょく】人【じん】 (lit. "yellow race") is the Japanese term used to refer to Mongoloids, which includes most east Asian people. The idea of roughly categorizing people of the world as one of 白色人種 (Caucasoid), 黒色人種 (Negroid) and 黄色人種 (Mongoloid) has been widely accepted at least here in Japan. According to Wikipedia, however, it's no longer considered as a biologically valid classification.
While 黄色人種 is generally accepted as jargon, that does not mean Japanese people widely believe their own skin is 黄色 (yellow). You should use 肌色 to refer to the color of average Japanese people, which is somewhere between thin pink and yellow. Actually some people may feel offended when they're referred to as 黄色い肌の人 or something like that. I'd rather not go into detail, but see the discussion here.