As the question title says, I can't seem to find what the difference is between 人文科学 and 文系. I've seen both used and JP dictionaries aren't helpful here. Is one more colloquial than the other?
文系 is one of the two major categories which Japanese high schoolers have to choose from normally at the end of their first year of high school, the other being 理系. Once a student commits to one of them, the curriculum they take in the following 2 years will follow from that, as well as the subjects they get tested on in their university entrance exams. Japanese students have to choose their major they would like to study in university when they apply. And the majors are put in these two rough categories. For example, 東大 has 理科一類、二類、三類、文科一類、二類、三類, which correspond to law, engineering, sciences etc. When people talk about 文系/理系 it is always about this separation in high school and by extension the ensuing separation in their higher education. And they are used to talk about students (people).
人文科学 means the humanities referring to the actual disciplines. This term mostly applies in the context of post high school higher education (university, graduate school, etc.), rather than about people. This is a term used in the academia. Mind you, it is not a term without ambiguity. In an American academic context, to me at least, the humanities refer to disciplines such as history, the arts, languages, linguistics, etc. But I think in Japan there is significant overlap between 人文科学 and social sciences and a lot of people use 人文科学 to refer to disciplines that I would consider social sciences such as political science, sociology, etc.