Unlike some languages (English, French, ...), written Japanese sentences don't have spaces between words.

I know that it is the same in Chinese for example, but the fact that the Chinese language only use one alphabet (hanzi) makes it easier to split words, most of the time it is 1 or 2 characters for each word.

I also know that a lot of it comes from knowing vocabulary/grammar to be able to know where each word begins and ends, but I was wondering if there were any tips that can be used to be able to differentiate them?

  • 7
    I think it's easier to delineate words with 3 scripts than 1.
    – Flaw
    Commented Apr 11, 2013 at 9:27
  • 4
    This raises the question: are there any good examples (preferably in modern Japanese) of sentences with ambiguous word boundaries? I.e. sentences that can be read in two different ways, depending on how you separate the words.
    – Pitarou
    Commented Apr 12, 2013 at 10:35
  • 1
    @Pitarou "私はあくまで執事ですから。" Sebastian's catch phrase from Black Butler is a good example. Commented Apr 13, 2013 at 4:18
  • Not an answer, but an advice: there is a great browser plugin, rikaichan (for Firefox) / rikaikun (for Chrome), that shows vocabulary entries as you just hover your mouse along the text. It allows to check possible word boundaries really fast, even if different segmentations are possible.
    – firtree
    Commented May 9, 2013 at 17:46
  • I've heard of this one and used it for a long time, I'm now using Perapera, it does the same things but I find it better : anyway, thanks for your advice :)
    – tama
    Commented May 10, 2013 at 0:56

3 Answers 3


Certainly vocabulary helps, but you can get quite far by considering the structure of a sentence.

  • Nouns are usually written in kanji (or katakana) and are practically always followed by a particle (を, が, は, から, etc.) (unless they are followed by a copula で, だ, です, etc.)
  • The stem of verbs (including adjectival verbs, or "i-adjectives") is usually written in 漢字, with the ending (the conjugated part) written in kana (called 送り仮名).
  • Adverbs are often written in kana
  • Adjectival nouns (AN) ("na-adjectives") are usually written in kanji and usually followed by な.

With that information, you can usually parse a sentence without knowing any vocabulary (but you need to know a list of particles and knowing the possible conjugation for verbs also helps).

For example,



(N) 住宅地域
(P) に
(V) おける
(N) 本機
(P) の
(N) 使用
(P) は
(AN) 有害な
(N) 電波妨害
(P) を
(V) 引き起こす
(N) こと
(P) が
(V) あり、
(D) その
(N) 場合
(N) ユーザー
(P) は
(N) 自己負担
(P) で
(N) 電波妨害
(P) の
(N) 問題
(P) を
(N) 解決
(V) しなければなりません。

where for the bold lines, I needed to know that

  • こと is a nominalizer, i.e. some sort of auxiliary noun, and thus is not written in 漢字
  • その is a determiner and not an adverb
  • 場合 is also an auxiliary noun and doesn't take a particle here.

But as you can see, you get quite far with the handful of rules above.

  • 2
    Great answer, and thanks for the example, it really helps :)
    – tama
    Commented Apr 11, 2013 at 6:46
  • 2
    "Nouns are usually written in kanji (or katakana) and are always followed by a particle (を, が, は, から, etc.)" Except when those particles are dropped in colloquial writing, of course :-). Commented Apr 11, 2013 at 21:48
  • 1
    Nouns can also be followed by a copula, of course.
    – user1478
    Commented Apr 22, 2013 at 11:55
  • 1
    Now is 住宅地域 one word or two?
    – ssb
    Commented Apr 22, 2013 at 23:34
  • 1
    @ssb I guess you can argue about words, but it certainly is a noun phrase, or, even stronger, a compound noun. After following the simple steps above, there is still more work you have to do before you can look up everything you need in a dictionary.
    – Earthliŋ
    Commented Apr 23, 2013 at 4:18

Separating words in a sentence, at least when done by computer, is called segmentation (分かち書き) or tokenization (トークン化). When using an IME to input Japanese, when you hit the space bar to convert the kana to kanji, the IME has to segment whatever it is that you typed, then use a dictionary to replace the kana with its kanji. As you've probably learned by now, the IME is not 100% accurate. As for general tokenization rules, it depends on what you're tokenizing and what you define as a word or token.

For example, Kuromoji, an open source morphological parser, will segment Earthliŋ's sentence above this way:

住宅 | 地域 | における | 本 | 機 | の | 使用 | は | 有害 | な | 電波 | 妨害 | を | 引き起こす | こと | が | あり | 、 | その | 場合 | ユーザー | は | 自己 | 負担 | で | 電波 | 妨害 | の | 問題 | を | 解決 |し | なけれ | ば |なり | ませ | ん | 。

Even though Earthliŋ parsed the last word しなければなりません as a single token, Kuromoji parsed it as 6 tokens. How you parse depends on what information you want to extract. Kuromoji uses a dictionary to aid the parsing, but another parser, TinySegmenter, is pattern based and also does a very good job:

住宅 | 地域 | に | おける | 本機 | の | 使用 | は | 有害 | な | 電波 | 妨害 | を | 引き起こす | こと | が | あり | 、 | その | 場合 | ユーザー | は | 自己 | 負担 | で | 電波 | 妨害 | の | 問題 | を | 解決 | し | なけれ | ば | なり | ませ | ん | 。

As you can see, there are a few differences between Kuromoji and TinySegmenter (both of these you can use right in the browser). Although no person can tokenize the sentence in the manner these small programs do, it should come naturally and unconsciously as you learn Japanese.

If short on time however, a very crude way to tokenize is to just group characters contiguously by script (hiragana, katakana, kanji):

住宅地域 | における | 本機 | の | 使用 | は | 有害 | な | 電波妨害 | を | 引 | き | 起 | こすことがあり | 、 | その | 場合 | ユーザー | は | 自己負担 | で | 電波妨害 | の | 問題 | を | 解決 | しなければなりません | 。 |

The way Earthliŋ parsed the sentence above was in fact still vocabulary intensive. For example, in order to segment 引き起こすことが into 引き起こす | こと | が (and not, say, 引き起 | こすこ | とが) he probably first started with the fact that こと is a nominalizer and so is a token. Next, delimiting the token こと then yields all 3 tokens 引き起こす | こと | が. If he didn't know beforehand that こと is a nominalizer he would not have known that it was a token and so he would not have been able to choose between the parsings 引き起こす | こと | が (correct) and 引き起 | こすこ | とが (incorrect).

Well, I'm just guessing. Obviously I don't actually know what went on inside the head of Earthliŋ, but I wanted to draw attention to the fact that when it comes to segmenting, having a vocabulary (or a dictionary if you're a computer) results in different segmentation strategies.

  • Thanks for the answer :) I happen to do coding too, so I can understand "tokenizer" notion, and that was an interesting reading, with good links too ! EDIT : and yes, of course vocabulary plays a major role in understanding a sentence, nothing new here :)
    – tama
    Commented May 7, 2013 at 3:12

Great question! This is something that I have struggled with (especially early on... when trying to read any large section of text...) but believe me, it does get easier with time and practice.

It may already be something you are using, but there is at least one thing that has helped me, and that is to look for particles in a given sentence. This doesn't always help... but it can provide boundaries for where many words (or phrases) begin and end.

Other than that, with time and practice it will become more evident where words begin and end as your vocabulary is further enriched!

  • 1
    Thanks for the answer, I knew that the particles could help, except when they are sometimes part of the word before, but you also mentioned that ;)
    – tama
    Commented Apr 11, 2013 at 6:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .