4

By "typical modern book " I means:

  1. It's modern book and hardly use ancient languages or any topolects.
  2. It's something like a famous novel(not short story) or book with high reputation.
  3. It's a normal book not a dictionary or technique book.

Additionally, I wonder how many distinct characters can cover 99.7% of the book content.

The source of the problem is that I want to know how many unstudied (meaning of) kanjis I will meet when reading.

  • 1
  • I too tried to ask questions about how many this or that. It's hard to get an answer. – Mathieu Bouville Feb 22 at 11:52
  • Are you asking about the Joyo Kanji List? You mean how many kanji in addition to the Joyo kanji list are used on average? – kandyman Feb 22 at 12:09
  • @kandyman Yes, this is one format of this question, because the source of the problem is that we don't know how many new kanji we will meet, And Can Joyo Kanji list cover 95% ( 150 in 3,000 ) of "distince characters of a book"?If yes, then this question maybe unconstructive. Sorry I am a beginner. – Voyager Feb 22 at 12:20
  • It's a good question. Please don't apologize whether you consider yourself a beginner or not. Everyone is welcome! – snailcar Feb 23 at 3:03
6

There are currently 2136 常用漢字. However, using writers will often use 漢字 outside of this list.

So I ran a script to calculate the total number of unique 漢字 in Harry Potter and the Chamber of Secrets (a modern novel mentioned there) and on ノルウェイの森/Norwegian Wood (a "famous" Japanese language novel, though perhaps not "typical") and compared it to the 常用漢字 list. You can see some of the code (it just has the results for the distinct values in the books) here. Here's what I got

Harry Potter and the Chamber of Secrets: 206 non-常用漢字 out of 1768 漢字 (肱 might be a typo for 肘, 躯 for 体, 荊 for 茨, and 綴 for 絞. I also wonder if 姐 and 嬉 are old forms or my source text has typos. I also have no clue about 杵 and 磔, which are likely wrong. I'll try to confirm with a physical copy in the future.)

{'蘇', '捌', '曾', '鈎', '撫', '剥', '捷', '篭', '恰', '鍮', '撒', '涎', '虱', '樽', '炸', '塵', '椀', '吼', '淀', '絢', '歪', '掠', '瞥', '揉', '眩', '董', '錆', '嘘', '楊', '蝶', '靄', '疼', '呆', '叱', '弛', '攣', '棍', '猾', '藁', '軋', '荊', '揃', '纏', '甥', '這', '樫', '苺', '掻', '惚', '杖', '煤', '咳', '煽', '辻', '瓢', '詫', '捺', '脛', '溢', '溜', '蝋', '鎧', '糞', '芻', '腿', '噛', '唸', '痺', '鍔', '霞', '狡', '翠', '昂', '罠', '燭', '蜘', '牢', '躯', '叩', '杵', '雛', '佇', '鳩', '呟', '灼', '爛', '逢', '嘴', '黴', '呑', '兎', '嘩', '濡', '轟', '鞄', '咎', '棘', '笥', '壷', '閃', '猜', '楓', '聡', '麿', '箪', '捧', '馴', '藹', '蛙', '肱', '馳', '覗', '釘', '茜', '埃', '愕', '雀', '皺', '躇', '鋸', '踝', '穣', '奢', '檻', '箒', '姐', '甜', '嵌', '萄', '螺', '嬉', '唖', '秤', '仔', '掴', '窪', '狼', '翔', '綴', '腔', '箔', '芍', '躊', '鮭', '磔', '鱗', '悶', '禿', '躓', '蛛', '惹', '閂', '哮', '瓜', '腑', '曙', '槌', '沫', '狐', '倦', '卿', '剃', '栗', '燻', '頷', '憑', '斧', '蟻', '橙', '竿', '怯', '紐', '穢', '耽', '尖', '咆', '凱', '蒼', '肋', '錘', '踵', '窩', '縞', '竪', '痙', '鞭', '吊', '喧', '聳', '髭', '焚', '廻', '斐', '堵', '頬', '醇', '瘤', '鋏', '葡', '〇', '鉤', '鷲', '漉', '舐', '滲', '堰'}

ノルウェイの森/Norwegian Wood: 167 non-常用漢字 out of 1746 漢字 (Like before 姐 and 嬉 might be old forms/typos. I'll try to confirm with a physical copy in the future)

{'梯', '遙', '鱸', '鎧', '梢', '徨', '撞', '喧', '昂', '雀', '而', '鳩', '庇', '轟', '燈', '葡', '拗', '函', '禿', '悶', '槌', '訊', '紐', '碗', '鞄', '鮭', '蝕', '蛙', '軋', '舐', '檻', '湘', '茗', '萄', '蓮', '鯵', '綺', '裳', '癬', '胡', '吠', '縞', '詫', '洒', '樵', '奢', '掻', '痰', '爛', '呑', '姐', '錆', '疥', '殆', '糊', '〇', '駕', '罠', '夾', '頬', '淋', '蝿', '楊', '鴉', '歪', '篇', '睨', '吊', '疼', '叩', '嚢', '噛', '咳', '睾', '竪', '唖', '菱', '濡', '囁', '仄', '糞', '蹙', '脆', '稜', '彷', '褪', '漕', '蟻', '曝', '栗', '釘', '杖', '柏', '螢', '甥', '旭', '剃', '耽', '惣', '愕', '癌', '喰', '掴', '醤', '嘘', '嘔', '顰', '腿', '疇', '瑞', '蹟', '嬉', '馴', '蝉', '垢', '鯛', '桐', '弘', '挪', '嘩', '釆', '揃', '云', '撫', '玲', '屑', '噌', '怯', '噂', '這', '桶', '鰻', '髭', '迂', '蛾', '蝶', '瓜', '巳', '鱈', '苔', '苑', '綜', '瞞', '墟', '阿', '辿', '茫', '睥', '茸', '樫', '蠅', '鼠', '弛', '坦', '穢', '倦', '鹸', '椀', '凌', '眩', '伊', '鴨', '叱', '煉', '苺', '捧', '劫'}

If we include 人名用漢字, which brings the total 漢字 to 2999, Harry Potter and the Chamber of Secrets has 104 that aren't in either list and ノルウェイの森/Norwegian Wood has 78.

{'姐', '呟', '壷', '嘘', '鍮', '咆', '呆', '爛', '滲', '瘤', '掴', '肱', '咳', '憑', '穢', '髭', '叩', '禿', '塵', '吼', '腿', '炸', '痺', '篭', '皺', '猾', '噛', '蜘', '藹', '舐', '腑', '頬', '甜', '揉', '黴', '芍', '愕', '煽', '罠', '閂', '聳', '檻', '眩', '燻', '鈎', '嵌', '吊', '痙', '唸', '糞', '咎', '捌', '歪', '攣', '剃', '脛', '蟻', '踵', '荊', '虱', '怯', '蛛', '奢', '蝋', '棍', '靄', '掻', '窩', '〇', '佇', '頷', '躯', '哮', '笥', '躊', '漉', '躓', '箒', '叱', '嘴', '芻', '鍔', '剥', '鋏', '躇', '埃', '箪', '涎', '棘', '疼', '呑', '狐', '猜', '鉤', '踝', '狡', '鮭', '磔', '唖', '軋', '悶', '覗', '蛙', '牢'}

{'拗', '囁', '褪', '彷', '唖', '蹙', '眩', '鰻', '睥', '蠅', '嘔', '愕', '徨', '禿', '鮭', '咳', '疥', '鱈', '髭', '疇', '蛙', '釆', '癌', '醤', '洒', '顰', '鼠', '樵', '〇', '蝕', '鹸', '掴', '呑', '爛', '癬', '螢', '吠', '吊', '蝿', '頬', '悶', '睾', '蟻', '糞', '叩', '怯', '掻', '疼', '茫', '夾', '穢', '垢', '叱', '蛾', '鱸', '歪', '鯵', '睨', '軋', '瞞', '舐', '蝉', '罠', '茗', '脆', '奢', '腿', '挪', '嚢', '檻', '鴉', '仄', '嘘', '噛', '痰', '姐', '墟', '剃'}

As for your second question as to how many 漢字 you need to know to know 99.7% of the ones used, I used the results of this Github repository to get the following:

Aozora Bunko (there are many many "old" stories): 3820

News (June 2014 - June 2015, more samples from 2015, from 4 different sources. Only article titles, subtitles, main text body and image captions were scanned. Weather forecasts and area-specific news were not included). : 2007

Wikipedia (a wikipedia dump: 2835

Twitter (steaming data for 1 week): 2312

Also according to this analysis of 朝日新聞朝夕刊's paper through the entirety of 1993, you need to know 1977 to be able to understand 99.7%.

  • 1
    Great answer! I'm surprised to see the character 联 in your list from ノルウェイの森. I thought that was only a simplified Chinese character. (负, 责, 归, 节 as well) – sazarando Feb 22 at 22:43
  • (and 务 in the Harry Potter list) – sazarando Feb 22 at 22:49
  • @sazarando Holy crap. Thanks for pointing that out. That was my mistake... That's what happens when you use shady sites, there's just some random Simplified Chinese notes in there... As for the Harry Potter, it appears to be 劣, but for some reason it's searching up as 务 despite looking like 劣. Time to check that one out too. – Ringil Feb 23 at 0:47
  • I wonder if the guy who made that data set was taking photos of the pages of those books and running it through an OCR that wasn't distinguishing between CJK characters properly. – sazarando Feb 23 at 0:54
  • @sazarando Yea I think that was it. I found some other very odd typos 卦 for 剥, 尐 for 少, 枞 for 枠, 痚 for 痛, 雤 for 雨, 刉 for 刊, 轡 for 鬱, 寓 for 窩, 蹟 for 躓, 徽 for 黴, 忚 for 応, 麿 for 磨, 嚢 for 髭, 泤 for 泥, 敶 for 敷, 痚 for 痛. There could be a few others too.. – Ringil Feb 23 at 6:34
4

This answer does not contain the exact information you are requesting but maybe it'll be useful to you anyway. I am not aware of any online resource that includes data from modern publications (such a resource would probably be very difficult to compile due to copyrights and the like).

The data below is from the same source as the one I used to answer this question - "Which is the most popular non-Jōyō kanji?". The data includes frequency of appearance data for kanji from 4 resources, Aozora Bunko, Wikipedia, Newspaper, and Twitter. Aozora Bunko might be close to what you want but I believe that the bulk of its corpus is older text which might not reflect modern frequencies.

The tables show the number of kanji that cover a given fraction of total kanji from each source.

Wikipedia
0.745 -------- 649
0.795 -------- 792
0.845 -------- 981
0.895 -------- 1228
0.945 -------- 1588
0.975 -------- 2093
0.985 -------- 2492
0.997 -------- 3529

Aozora Bunko
0.75 -------- 570
0.80 -------- 711
0.85 -------- 902
0.90 -------- 1181
0.95 -------- 1657
0.975 ------- 2248
0.99 -------- 2657
0.997 ------- 3714

Newspaper
0.75 -------- 658
0.80 -------- 789
0.85 -------- 958
0.90 -------- 1185
0.95 -------- 1566
0.975 ------- 1997
0.99 -------- 2424
0.997 ------- 3292

Twitter
0.75 -------- 592
0.80 -------- 710
0.85 -------- 896
0.90 -------- 1151
0.95 -------- 1464
0.975 ------- 1970
0.99 -------- 2347
0.997 ------- 3409

The 2136 Jouyou kanji cover the following fraction of the 4 databases (backward order so they line up)

0.9815 ------ Wikipedia
0.9399 ------ Aozora
0.9914 ------ Newspaper
0.9831 ------ Twitter

  • Could you provide more details about your databases? in addition to I compiled these a while ago for my own curiosity and don't readily have references to the source data (other than the excel file I made). – Voyager Feb 22 at 22:56
  • I think it was from here. It was several years ago though so I can't say for sure. – G-Cam Feb 24 at 4:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.