I'm not sure about "とも" in this sentence.
Can anyone explain the usage of とも
It can also be written as:
The と is the case particle as a quotative marker, and the も is the binding particle. I think it's like "I also thought that it might be rude to ask such a thing, but...", the "also/も" implying that you were thinking about other things, too.