Conveniently, the 類語例解辞典 groups three of these together in one heading and adds the other in a note at the end. Here's what its 使い分け section has to say on the matter:
または is used for cases where there are two choices and whatever is not selected will be cast aside. In these cases, the person making the selection has freedom to make either decision.
もしくは is used in cases where there are several choices, and the options are limited to those presented.
あるいは is used when choosing between two things and it is possible to choose both of them at the same time. That said, it carries the expectation that there is, in fact, a correct choice to be made or that the chooser doesn't really have freedom to decide between them.
All of the above are primarily literary words, and are usually replaced by か in everyday conversation.
それとも is used in spoken language. Its use is limited to choosing from what is contained in the question, as can be seen in the following example sentence:
「行く、それとも行かない？」 "Are you going, or aren't you?"