As had been pointed out, as a general rule, that part of the word - in terms of kana syllables - that changes or inflects is written with okurigana.
See also 「送り仮名の付け方『国語を書き表すための送り仮名の付け方のよりどころ」』・単独の語１・活用のある語・通則１」, which states as a general principle that the inflectional ending is added in kana.
When kanji had been introduced to Japan, and when people began to write Japanese with kanji as well, there was the problem that when writing everything in kanji, you had to indicate somehow which inflection of a word was intended. Chinese didn't have these inflections and thus the writing system had to be adapted.
What the Japanese people came up with was (a) the 弖爾乎波【てにをは】 system(s) which consisted of adding small dots and strokes to indicate a certain inflection (e.g. ~たり or ~て) or particle (e.g. に or を); and (b) - like with Egyptian hieroglyphs - a more flexible dual system, some kanji used for their meaning (semantically) and some for their sound (phonetically).
For example, you could write 恐美多利 for 恐【かしこ】みたり, the last three kanji used phonetically. The latter became man'yougana, then hentaigana and evolved into the well-known kana eventually.
Today, there are 五段 ("u", e.g. 泣く) and 一段 ("ru", e.g. 語る) verbs, but historically, there had been slightly different groups. If you take a verb like taberu, its conjugation had been:
That's called the 下二段【しもにだん】 conjugation; 二段 because the vowel changes between e and u (cf. the ablaut in German
singen-sang-gesungen-Song, and English
tooth-teeth), 下 because these vowel come last (=at the bottom in vertical writing) in the Japanese order
二段 verb became 一段, getting rid of the vowel change, and the 終止形【しゅうしけい】 fell out of use, leaving you with only a final る that changes during inflection. But as a remnant of Classical Japanese, the correct spelling with okurigana is 食べる instead of ✘食る, which would be analogous to 見【み】る. The same principle holds for many other verbs.
The same story holds true for 起きる・起こる, 掛かる・掛ける and many other pairs. They are usually derived from some earlier form of one (shorter) verb, e.g. 掛【か】く.
One reason why new verbs such as 掛かる and 掛ける were formed from an existing verb 掛く was to create transitive/intransitive pairs, or potential/causative verbs such as 見【み】る・見える or 覚【さ】める・覚ます, which gives the explanation for this pattern detailed in the answer by @fosskers.
Furthermore, okurigana help to distinguish between the two words, 起る could theoretically be either おきる or おこる.
For 五段 verbs, historically 四段 (=four vowels; a, i, u, e) verbs, only the last syllable changes, thus 食【は】む, 蝕【むしば】む, 読【よ】む 坐【ましま】す, 嗤【わら】う, 貪【むさぼ】る etc. The inflection of 読む is, for reference:
To illustrate this, consider the two verbs 統【す】べる and 滑【すべ】る. 統べる derives from the 二段 verb 統ぶ (inflection: すぶ・すぶる・すべ・すべ・すぶれ・すべよ), thus its non-inflecting stem is す and it is written 統べる. On the other hand, 滑る is a perfectly normal 四段 (now 五段) verb on る (inflection: すべる・すべる・すべり・すべら・すべれ・すべれ), whose non-inflecting stem just happens to be すべ, thus it is spelled 滑る. In classical Japanese, these two verbs were referred to in their dictionary forms すぶ and すべる and were as different as 行く and 言う, which just happen to exhibit the same past form いった.
Verbs derived from other words retain their inflectional endings. 怪【あや】しむ・怪しがる and the adjective 怪【あや】し, 悲【かな】しむ and 悲し, 苦【くる】しむ and 苦し.
The story continues with adjectives. As for i-adjectives, the conjugation is as follows for 赤【あか】い:
連体形 あかき ／ あかい
連用形 あかく ／ あこう
あこう is a dialectical variation (and used before ございます, as in 有【あり】難【がと】うごさいます, from 難【かた】い) and arose from あかく→あかう→あこう via a sound shift. Thus, only the last syllable is inflected and the proper spelling is 赤い, 赤う, 赤く.
There is a variation of this conjugation for adjectives whose last syllable before the inflecting one is し, e.g. 美【うつく】しい:
連体形 うつくしき ／ うつくしい
連用形 うつくしく ／ うつくしゅう
The final inflectional し is not added for the 終止形 (dictionary-form), thus the proper spelling is 美し or otherwise it would not get any okurigana to indicate it is an adjective, thus it is written 美しい and 美しく.
The same applies to the compound conjugation of adjectives. 赤くない, because that is 赤く + the word 無い. 赤かった, because that is 赤し+あり+つ+あり→あかかりたり→あかかった.
Some adjectives that are formed from an inflected form of another word retain that inflection, e.g. 羨【うらや】ましい, derived from the verb 羨【うらや】む, plus し + the adjectival inflectional ending い; or 頼【たの】もしい, from 頼む.
Compounds & misc.
Next are compound words whose elements are buried under etymology and may have become obscure. The suffix is rarely written with kanji and left in kana, which may look like okurigana.
The proper spelling is 速【すみ】やか, 艶【あで】やか, 健【すこ】やか, 雅【みやび】やか, 明【あき】らか, 滑【なめ】らか, 朗【ほが】らか, 安【やす】らか etc. because they are compounds containing the suffices -やか and -らか, e.g. 艶【あで】+やか and 安【やす】+らか.
静【しず】か, 愚【おろ】か, 確【たし】か, 豊【ゆた】か, 僅【わず】か consist of a combination with some suffix -か. The above -やか might be a combination of や+か.
See this question concerning the three suffixes -ら, -や, -か.
嬉しげ, 在【あ】りげ, 大人【おとな】げ, 可愛【かわい】げ etc. contains the suffix -気【げ】, now often written in kana only.
古【ふる】めく, 唐【から】めく, 春【はる】めく ひしめく, ざわめく etc. contain the auxiliary verb めく ("appear/look like").
欲【ほし】しがる, 強【つよ】がる, 度【た】がる etc. contain the auxiliary verb がる "show signs of".
昂【たか】ぶる (高【たか】+ぶる), 偉【えら】ぶる, 揺さぶる etc. contain the auxiliary verb ぶる (振【ぶ】る) "pose as, pretend to".
綺麗【きれい】の, 綺麗【きれい】な because they are compounds containing the particle の and the verb (as a copula) なり shortened to な.
全【ま(っ)た】く, 全【すべ】て, 決【けっ】して, 妄りに because they are derived from the adjective 全【また】し, the verb すべる, the Sino-Japanese (on-reading + する) verb 決【けっ】する, and the verb 乱【みだ】る, respectively.
曰【いわ】く (from 言う), 見【み】らく, 散【ち】らく etc. because they are all formed with some old element く.
Some nouns derived from verbs can be written without okurigana, e.g. 話【はなし】, 取調室【とりしらべしつ】, 折紙【おりがみ】, 振仮名【ふりがな】, 受付嬢【うけつけじょう】, 立入【たちいり】禁止【きんし】. Sometimes, the okurigana other than for the last verb are omitted in verbal compounds, e.g. 問合【といあわ】せ, 振舞【ふるま】う, 受付【うけつ】ける.
Exceptions / Special cases
Some words are written with additional okurigana: 後【うし】ろ, 蔑【ないがし】ろ (from 無【な】きが代【しろ】). Sometimes you can find variant okurigana spellings such as 既【す】でに, 故【ゆ】えに and 此【こ】れは (instead of 既に, 故に, and 是は).
Additionally, おこなう used to be written as 行う (行ふ), but the spelling 行なう has gained in popularity.
Thinking about おこなう, I would suggest that the main motivation behind these okurigana usages is to help the reader and to avoid ambiguity. 行って could be either おこなって or いって (or ゆって), but 行なって must be おこなって. Likewise, 後 could possibly be のち, ご or うしろ with only the context left to decide which, but 後ろ must be うしろ, no argument about the context required.
既 or 蔑 might be hard to read for some people, but giving part of its reading as 既で or 蔑ろ helps you to guess the word from context, even if you didn't know the kanji at all.
There are explanations for these pattern regarding okurigana usage, and they require some basic understanding of the earlier stages of the language. However, that doesn't change the fact that there are patterns, and even without understanding them in full detail, you will soon start to notice them and be able to tell how to spell a word. For example, you don't need to know about the history of the suffix -やか to notice that many words ending on -やか are spelled with やか as the okurigana.