I think the confusion comes from the function of the particles. It's a little bit hard to grasp because it adds a dimension that we don't have in English. Even if they function as exhaustive and non-exhaustive, it's strictly for what is mentioned, not the point you're making. I think it's easier to understand if we translate them:
- 文法と漢字に集中したほうがいいよ - You should focus on grammar and kanji.
- 文法や漢字に集中したほうがいいよ - You should focus on grammar and kanji (and others I didn't mention).
- 特に文法と漢字に集中したほうがいいよ - You should especially focus on grammar and kanji.
- 特に文法や漢字に集中したほうがいいよ - You should especially focus on grammar and kanji (and others I didn't mention).
As you can see, the や here simply add the meaning that you didn't mention something else. Why you didn't mention them is not implied. It doesn't mean you care or don't care about them. Now you might be wondering why else omit it? We can be creative, maybe you forgot something and you know it so you use や to not exclude it. Maybe you guess there's another one that correspond to your statement but you can't quite identify it. Knowing all that, it's natural to see that, just like in English, we add something like 特に to make our point clear because that particle alone isn't enough to imply what you want.
"Since と is used for exhaustive lists, wouldn't this wording imply that I don't really care at all about any other fruits?"
For the situation of the fruits yes, but in general it only means you listed everything you wanted for your point instead of "caring". The 特に will add the meaning "especially", so the whole point you're making is changing because you're now saying "especially just those two, (implied) others also, just not especially".
"Since や is usually used to give examples within a larger list, wouldn't this suggest that I care about the apples and bananas just as much as any other fruits (or is 特に enough to counteract this)?"
No, rather than implying that you care about any other fruits, it implies that you also care as much only for the fruits you didn't mention. Likewise, with 特に, it becomes "especially focusing on grammar and kanji, plus the ones I didn't mention, (implied) and the others too but not especially".
In other words, there is three categories instead of two. The words you listed, the ones you didn't list and finally the ones that aren't related to your statement.