I don't understand the した性格 part, because how can した modify "personality" which can't >do anything?
きちんと on it's own is an adverb, for example きちんと掃除をする/do the cleaning properly.
It's one of those words you can add する to afterwards to make it into an adjective.
For example: 母さんは性格がきちんとしている / (My) mother has a proper/orderly/impeccable personality.
This gets put in the past tense* when it is brought in front to modify the noun (性格): きちんとした性格の母さん / (My) mother who has an proper personality.
I also don't understand the function of 真面目で, especially what the で particle means.
Let's look at another sentence where it might be easier to understand: 真面目で誠実で優しい人/ a serious and sincere and kind person.
You connect na-adjectives together with で the same way you might connect i-adjectives together with ～くて, for example: 安くておいしいケーキ/cheap and tasty cake.
So 真面目できちんとした性格の母さん here means: (My) mother who has a stiff and proper personality.
*I swear there was a question about this and I wanted to link to it but I couldn't find it...