I've read that 日本人の知らない日本語 translates to: "Japanese (language) that Japanese (people) don't know". But I don't understand how or what the の does in that sentence. If I'm not mistaken 知らない日本語 could mean "Japanese language that (x) don't know" or "even unknown Japanese". But I don't get how the 日本人の fits into the translation.
In your example, 日本人の知らない is a relative clause, equivalent in meaning to 日本人が知らない. This clause as a whole modifies 日本語, so it means the Japanese that Japanese people don't know.
In relative clauses, the subject particle が can be replaced with の:
This is true in double-subject constructions as well:
But you can't replace が with の if there's a direct object marked with を:
It's just standard GA-NO conversion.