I'm at a point where I keep encountering new compound verbs and I'm not sure how I should approach them to study. Is there a list or category of "main" compound suffixes (like 続ける, 始める, etc.) that carries the same meaning for every compound? What are the rules for making compound verbs (if there are any)?
There are two types of compound verbs: syntactic and lexical.
According to Compound Verb Lexicon, there are 30 syntactic compound verbs in Japanese. You can find the full list in the link (Click "Syntactic and Lexical Compound Verbs" on the left pane). 続ける and 始める belong to this category, so their meanings are straightforward regardless of what comes as the first verb. The meanings of most syntactic compound verbs are related to starting, ending, continuing, repeating, etc.
On the other hand, there are also thousands of lexical compound verbs, and it takes time to master them. They are called lexical because they are unpredictable and there is no clear rule (although there are weak tendencies which you can gradually notice). Basically you have to remember each combination one by one.