this is probably a silly question, but since words with multiple kanji are usually read with on’yomi would that effectively make most japanese vocabulary loanwords? like 森林 or 図書館？i understand differences like how 本 does not mean book in chinese but it just makes me wonder about how japanese these multi kanji words are
As far as Japanese is concerned, loanwords (外来語) usually refer to words brought into Japan from countries other than China and written in katakana. But strictly speaking, it depends on how you define loanwords. Many on-yomi Sino-Japanese words had been around even before Japanese people learned how to write their own native words, so IMHO it doesn't make much sense to call them loanwords. I suppose most English speakers do not want to call common words derived from Latin loanwords, either.
Here are some definitions of 外来語.
外来語. Words brought into Japanese from languages other than Chinese.
I think you are answering your own question in part. Most compound words use a reading that derives from Chinese but remember that Japanese was its own language already before it drew from the Chinese writing system, as far as I know. It might have adopted many loanwords but it also used onyomi reading for many existing Japanese words. That is, in part, why I would say that it might be too much of a stretch to call all these native Japanese terms loanwords on that basis alone.
You call your question silly because you recognize that you would probably be stretching the interpretation of what constitutes a loanword a little, but on the other hand I have had similar musings about the onyomi readings and the origin of the Japanese writing system. It probably does make Japanese a very unique language that way, but I think it would be similar to calling a large portion of English words "loanwords" from Latin or Greek because one can trace their etymology to either.
Going by the narrower definition of a loanword, we usually mean terms that are adopted without or with little translation. There certainly are many such loanwords in Japanese but since most of these terms you would be including with your definition had words in Japanese that were distinct from Chinese before characters were adopted for them I would say that it is simply too much of a stretch to call such a substantial portion of the Japanese vocabulary loanwords.