The kanji has changed its sound long before importation into Japanese.
「[斜]{しゃ}」 (Zhengzhang OC: /*lja/, to scoop/ladle) is comprised of semantic 「斗」 (picture of a ladle > unit of measure for liquids) and phonetic 「[余]{よ}」 (/*la/). You can take the modern meaning, slanted, to be a phonetic loan.
*The unit of measure for liquids might also be seen with the kanji 「升」. Indeed, 「升」 was originally 「斗」 with dots added, indicating that the ladle was full of liquid.
Putting aside characters directly containing 「[余]{よ}」 (e.g. [途]{と}, [除]{じょ}), the exact shape 「余」 has undergone quite a few sound shifts and graphical corruptions in other kanji. You might find some more correspondences with 「[斜]{しゃ}」 in those changes; even though 「[余]{よ}」 itself appears to be rather far phonologically from 「[斜]{しゃ}」, the characters containing variants of 「余」 may still be fairly close in sound to 「斜」.
Observe the glyph evolution of 「余」, and then compare:
- 「余」 is the bottom component of 「[茶]{ちゃ}」, which was originally written as 「荼」.
- 「余」 was originally the top component of 「[舍]{しゃ}」 (Shinjitai: 「舎」).
- 「余」 is sometimes corrupted into 「佘」 to produce a phonetic indicator 「[示]{し}」. In rare circumstances, you might come across a character containing 「佘」; their on'yomi would be something like 「[賒]{しゃ}」.