Is it possible to algorithmically convert Japanese text to Romaji?

Question

I am a programmer and I recently wrote two browser extensions, one will translate English text to "pronunciation" replacing all words founf on a web page, and another will replace all Chinese characters with their Pinyin-with-tone-marks counterparts. Now I am thinking if something similar is possible to Japanese? If yes, what would be the approach? Anything can be done algoritmically or you can only use dictionary? Or even dictionary is not keeping in mind that the same characters can sound differently based on context? In other words can you do this conversion without actually attempting to do a machine semantic translation?

Answer 1

Basically this is very difficult.

Real Japanese sentences on the net are mixture of kanji, hiragana, katakana and English alphabet. See Japanese writing system on Wikipedia.

Among these, hiragana and katakana are almost "pronunciation symbols" themselves. You can replace them into romaji using this table and you're 80% done. The remaining 20% is a bit tricky but they can be handled algorithmically. Still, there are various romanization systems, so you have to make a wise decision. There are also some "extended katakana" which may not be transliterated straightforwardly.

Kanji is the difficult part. Character-based replacement makes no sense because one kanji can be read differently in different words, and there are many jukujikun's. So you absolutely need a dictionary of some sort, but even with a dictionary, they are difficult for some reasons.

1. Japanese sentences are written without any spaces, so you cannot determine word boundaries with simple regular expressions. You need a dedicated morphological analyzer for this purpose, for example this and this (I have not tested them). Note that analyzers are not perfect.
2. Sometimes the exact same word or phrase can be read differently depending on the context, although English has a similar problem, e.g., "minute", "read", "wind". See: Difference between こんにち and きょう
3. Some uncommon words (especially proper nouns) are not on any dictionaries, but you still have to make a "reasonable guess" on them. Of course English has the same problem in this regard, but the algorithm for doing this in Japanese might be more complicated.

Answer 2

Naruto has addressed why this is difficult in great detail, so I am not going to talk about that. Attempting to build your own solution from scratch is probably not the right way.

However, Yahoo Japan has an API for producing the readings of words, which can be seen here (though the documentation is entirely in Japanese). Once you have the kana for a word, moving to romaji should be trivial, as all you are doing is mapping phonetic characters.

Answer 3

If you insist on creating something on your own, you might try doing a parser by character that considers the "next" few characters in the char array, and attempts to match the given character and its "nextChars" to words in a dictionary.

In light pseudocode fashion:

array kanjiArray = readChars();
if (kanjiArray.hasNext()){
  // [i+1]
  if (kanjiArray.hasNext().hasNext()){
    // [i+2]
    string dictSearch = kanjiArray[i]+kanjiArray[i+1]+kanjiArray[i+2];
  } else {

  }
}

if (dictSearch in dictionary){
  string translit = getTranslit();
} else { 

  // shave off chars from the end here?

  if (dictSearch == ""){
    // this shouldn't happen because you should always be able to return at least 1 romanization per char, even if the char is already romanized, so return error if you get here
  } else if (dictSearch in dictionary){
    string translit = getTranslit();
  } else {
    // recursion? until you have something readable
  }

}

// shave the defined chars from the start and continue along the sentence/text

What this should do is hopefully this, for example:

漢字すら読めない

1. Reads the whole sentence as 漢字すら読めない
2. Attempts to recognize the whole phrase as a dictionary term
3. Fails, shaves off the い
4. Continues to fail and shave from the end until it matches the word 漢字
5. Transliterates 漢字 from dictionary into Kanji
6. Shaves off 漢字 from start and repeats process with すら読めない
7. (optional?) Reads すら as hiragana and transliterates directly
8. Shaves off すら from start and repeats process with 読めない
9. Recognizes 読めない as a word from dictionary and transliterates as yomenai

Provided that this is pseudocode, the logic is not perfect. There's of course more things to consider, but hopefully this is a starting mental template upon which you'd build. ;)

Answer 4

When considering where you might find such functionality (full text to hiragana with disambiguation) it's worthwhile considering where the demand is. Simply rendering hiragana for display is probably not that much in demand, except possibly for automatic addition of hurigana above text. Text to speech functionality however is very much in demand - a quick search turned up this site: http://voicetext.jp which appears to be run by Sharp. You can enter text with kanji and even romaji and get disambiguated voice. It would seems that arbitrary text to hiragana text functionality would be a subset of the total text to voice functionality. But it doesn't seem sufficiently in demand to be listed on that home page as one of the functions they supply.

They do offer an API for their text to voice.

The voicing is really good.

Maybe they be convinced to offer a arbitrary-text to hiragana API, and/or an arbitrary text to original-text-plus-hurigana API. Or at least offer some advice.

Answer 5

naruto's points about difficulty are valid; building a romaji converter is much more complicated than a pinyin converter, but it's certainly possible to build a system that does this. I wrote a library called cutlet that does the conversion. You can try out the online demo.