Is there a grammatical explanation for this pattern, which I have seen a couple times in writing? I'm guessing it comes from classical grammar.
It is from 'classical' grammar, or rather Early Middle Japanese. -し is the 連体形 (the 'attributive' form, used to modify nouns) of the past tense marker -き. It is used to describe events the speaker knows have happened; in contrast to -けり, which is used for events the speaker has only heard about but not experienced himself. (There are a few other past tense or perfect aspect markers - -ぬ, -り, etc - that are older and in varying stages of loss by Middle Japanese.)
選ばれし者 then means 'the chosen one(s), the one(s) who has/have been chosen'.