Looking it up has gotten me mixed answers. I've seen people refer to either as the infinitive. A follow-up: is the -ru on ichidan verbs seen as an auxiliary or is it part of the word?
Japanese is not an Indo European language. As such, it does not share the same structures and grammatical concepts. Arguably, Japanese does not have an infinitive.
What do you mean when you say “infinitive“? Define how you intend the term, and then we might be able to elucidate whether Japanese has an “infinitive” for your purposes.
For what is worth, and always taking into account that it's not the same as an infinitive, as Eiríkr Útlendi explains in their answer, in Japanese you have what is sometimes called the "dictionary form". Just like in some languages such as English and Spanish we look up verbs in a dictionary using the infinitive, in Japanese you would look up the verb in a dictionary by its "dictionary form".
For the ichidan verbs, the dictionary form always ends in る, and for the godan verbs it ends in a mora of the -u (う) column (or row) of the hiragana table, such as く, つ, む, etc. Whether this final moras of the dictionary form are regarded as an auxiliary part or as part of the word, I don't know. Does it really matter? If you look up verbs in the dictionary, they come up as whole words, including る, く or whatever the ending mora is.
To put it in terms of the "masu stem", you can obtain the dictionary form of a verb as follows:
- Ichidan verbs: Add る to the "masu stem".
ます→ たべ → たべ + る → たべる
- Godan verbs: replace the last mora of the "masu stem" by the mora with the same consonant but in the -u (う) column of the hiragana table.
ます→ のみ → のむ
This is my personal opinion, but I think that learning verbs and verb conjugations starting from the masu stem is a mistake, because it just hides the difference between ichidan and godan verbs, whereas learning verbs directly in the dictionary form is more natural, and it is very easy to change from the dictionary form to the masu form.