Why is なさい used in ごめんなさい when it is usually used as a command form?
免 is a somewhat old word meaning 'forgiveness'. There is a polite command form that goes お・ご～なさい, such as in おいでなさい or ご覧なさい. If you apply this to 免, you get ご免なさい, literally 'please forgive me'. This command form is also seen in おやすみなさい and おかえりなさい.
Nothing at all's answer is fine the basics, but it could be misleading depending on how much Japanese you are planning to learn.
ご/おV-stemなさい is a polite command conjugation of the verb なさる.
なさる is the keigo equivalent of する (meaning it is respectful). Conversely, いたす is the self-humbling form.
So by using なさる you express respect for the person you are talking to and by using ご/おV-stemください you express it politely.
免ずる means to forgive and this is then conjugated to mean "please forgive me" by using a polite and respectful command conjugation.