What is the set phrase used to apologize for not corresponding regularly?
means something like "I apologize for not contacting you for such a long time".
It is a set phrase which is polite and suited for written correspondence and thereby satisfies almost all your requirements, except for the fact that it doesn't mention that you regularly fail to keep in touch. But then, I don't see how mentioning this would have any positive effect.
As you haven't given a clear context for your question, I should just like to mention that the above is better suited if the "late" correspondence was initiated by your letter or email and you are not simply replying late to a particular letter or email. (For the latter, something like 「返事が遅くなり、申し訳ございませんでした」 would probably be more suited.)