I assume that you already know the difference between いい and ましだ in isolation, and focus on the difference in the structures:
As you wrote, both (A) and (B) mean that doing Y is better than doing X although both are bad, and this is often used as a statement of how bad doing X is. But I think that I feel a few differences in nuance between (A) and (B).
First, simpler parts:
- (B) is less formal than (A) because ましだ is a little colloquial word.
- (B) puts more emphasis on the badness of the two options than (A). This is because ましだ already means “less bad” (better but still bad) even without the ～するくらいなら part.
But I think that there is another difference. I will try to explain it although I still have difficulty getting hold of it.
While both (A) and (B) can be just a statement of a fact or an opinion, (A) can also be a suggestion for the listener to do Y. I will use nomithekid’s examples:
(1A) can be either:
- a mere statement of the speaker’s opinion about how bad the job is, or
- a suggestion for the listener to quit the company, given that the alternative is doing that bad job.
I think that (1B) is interpreted only as a statement and not a suggestion.