This is not an answer but a collection of comments based on my personal feeling, but I post it as an answer because it is too long for a comment.
First, here are two clear facts:
- のある simply does not have the same meaning as である. ピアニストのある私の姉 is incorrect.
- Replacing AであるB with AのB sometimes causes ambiguity. For example, ピアニストの姉 can mean either “(my) sister, who is a pianist” or “a sister of a pianist” depending on the context.
Now I will move on to a less clear part.
I agree with you that AであるB sounds a little too formal for a casual conversation (although I would not call it “literary”). In a casual conversation, I would probably avoid a relative clause in this case and say something along the following.
My sister is a pianist, and she is always careful not to injure her hands.
The same meaning as AであるB can be also expressed by AのB in some cases, and I agree that AのB can be less formal than AであるB. But in your case, I find using の in place of である less natural:
Unfortunately, I do not know why I feel it less natural than ピアニストである私の姉.