Kanji aren't necessary to write Japanese
Your rationale is correct; Japanese is a living, spoken language; people are able to understand each other by sound only, therefore a writing system based on sound has to be sufficient.
Some commentators have mentioned that Japanese speakers often allude to kanji when talking. That's true enough (and it probably reflects something deep about how the mental lexicon is organized¹); but explicit references to kanji remain a minor cultural fact, not a necessary feature of the language. There exists blind people in Japan, for example; the congenitally blind will never have seen a single kanji, and they're nonetheless fluent Japanese speakers. As a Brazilian, I've personally met several Japanese-Brazilians who learned the language from birth, but who never learned the writing system; they could interact with native Japanese people just fine (who'd often be surprised to learn they were born in Brazil), and I'll go ahead and assert they're native speakers, too. One does not have to know kanji to be a Japanese speaker.
We also have several types of sound-based, Japanese text works, so we know it can be done. @Kurausukun's example of children-oriented videogames (and children's books, etc.) is a good point. Other examples would include historical kana literature of the Heian period; Ishikawa Takuboku's Rômazi Nikki (written in Latin characters to prevent his wife from reading it – but easily readable by any Japanese speaker nowadays); and publications by associations like the various Rômazi-kai or Kana-moji-kai.
Pitch accent isn't necessary to understand Japanese either
@Kurausukun and others have mentioned the common argument that pitch accent is necessary for sound-based understanding. It's true that standard Tokyo Japanese distinguishes many words by accent alone (like the famous three-way distinction between HÁshì-gà = "the chopsticks"; hàSHÍ-gà = "the bridge"; hàSHÍ-GÁ = "the edge"). And true, it's a damn shame that they aren't usually marked in writing; accented rōmaji is, in my opinion, a severely underrated tool for beginning foreign students, and I wish it was the norm.
One has to keep in mind, however, that the "homogeneity" of Japanese is a myth. The country has a large number of distinct dialects, and pitch accent in particular varies a lot between them (I'm doing research on accent, and I'd say the amount of variation from the same speaker is already significant…). Some areas, including e.g. most of Fukushima or Miyazaki, feature no pitch accent at all (無型アクセント). Given that people from these areas can understand each other, and that in general a Tokyoite can understand an Osaka comedian without much trouble despite their accents being unpredictably different, it has to be the case that pitch accent in writing isn't essential.
This should come as no surprise, since language is notoriously resilient. Tone in Chinese is responsible for a lot more important distinctions than accent in Japanese, and yet Chinese people can read pīnyīn (Chinese romaji) without tone marks, and often don't bother writing them. Vowels are certainly an important part of the English language, but yuu cuuld wrutu wuthuut vuwul dustunctuns und stull bu undurstuud must uf thu tumu.
Written Japanese is different than spoken Japanese
That being said, there are still some reasons why Japanese people feel so strongly that kanji are important.
The first of them is that the vocabulary of written Japanese isn't the same as the spoken language. To understand this, you have to switch your point of view inside out. It's not that Japanese has so many homophones that it needs kanji to be written. This is exactly backwards; what happens is that kanji allows one to write with lots of homophones, and therefore Japanese writing (and only writing) can use more homophones than normal. The same is true of Literary Chinese, the (artificial, written) language for which kanji were designed.² Chinese, too, is a spoken language (actually a large family of spoken languages), and as such is perfectly understandable by sound alone. However, the Literary version was a kind of super-condensed Chinese that relied on the characters as support. Reading Literary Chinese aloud wouldn't always result in something intelligible (unlike natural, spoken Chinese).
A big source of those Japanese homophones are precisely loanwords from that homophone-heavy Literary Chinese; these Chinese-style words are called kango in Japan. Typical spoken Japanese has about 23% of kango; typical written Japanese has some 41%—almost double as much.³ The more formal, intellectual or specialized the language, the more it tends to use kango.
Due to this difference, if you just blindly convert a text made for kanji into kana or rōmaji, the result often feels clunky and hard to read. The proper thing to do would be to replace kango vocabulary, bringing the written language closer to speech; this is called iikae by Japanese researchers.⁴ If the text is written without using kanji from the outset, the writer will instinctively choose adequate vocabulary, and iikae isn't necessary.
Kanji can be cool
Finally, I think it's important to keep in mind that, just because kanji are not necessary, it doesn't mean they don't add something. Reading kanji text feels different, and there are neurolinguistic reasons for that.⁵ And this means that the use of kanji has æsthetic, cultural, and subjective value.
Through their entire history, the tendency of the Japanese as a culture was never to try and make their writing simple and utilitarian, but rather to experiment, play, innovate with it. They could just have used kana or bonji phonetic characters for everything since a thousand years ago – indeed Heian kana literature was an experiment with this – but opted instead to keep using kanji, and not only that, but to assign multiple readings to them, and mix them with kana in many ways. The resulting system has always been explored creatively, to create many effects; not just in fancy works like the Man'yōshū (almost a puzzle-set of script play), but
also in popular things like Edo-era folk literature or modern-day visual culture.
When Natsume Sōseki writes tonikaku as 兎に角, or yakamashii as 八釜しい, for example, he's making a kind of game with the reader, sharing a sort of intimacy with them. When Anzai Fuyue writes a poem about a butterfly like this—
てふてふが一匹韃靼海峡を渡って行った
A butterfly is travessing the Strait of Tartar and—gone.
—he's using the cognitive weight of kanji characters to cause an effect similar to what Western poets usually do with consonants and obstruents. Natsuhiko Kyōgoku, a writer of detective fiction, relies heavily on archaic kanji and unusual furigana for atmosphere; he'd lose a lot if merely converted to kana. All sorts of linguistic and æsthetic effects can be created with the inventive interplay of kanji and kana and furigana…
…and this sort of thing is extremely common. It doesn't occur just in elevated poems and classical literature, but happens right now in the most popular of forms: in manga, in videogames, in advertisements, in song lyrics, in baby naming… So I'd claim that the Japanese have a living, thriving culture of kanji, and I find it perfectly reasonable that they consistently reject the idea of killing it.
References:
- Terry Joyce, Modeling the Japanese Mental Lexicon (& further research).
- Victor Mair, Buddhism and the rise of the written vernacular in East Asia.
- Miyaji and Kai. 「日本語学」特集テーマ別ファイル 語彙３：語種論／和語。
- Sachiko Matsunaga. The linguistic and psycholinguistic nature of kanji.
- Zev Handel. Logography and the classification of writing systems: a response to Unger.
