I have found this sentence in Odin Sphere. I guess it's kind of old fashion for a strong interdiction. I first thought it came from the negative form of である but it doesn't seem to be replaceable by ではない.
Does anyone know the origin of this form?
Yes, ～するでない is an old-fashioned and pompous way of saying "Don't do ～!".
In modern Japanese, this is a kind of 役割語 (stereotyped role words) which is typically used by noble and/or old people in manga and samurai dramas. This seems to have been used a lot more 100 years ago or so, because I can find many similar expressions (eg. 泣くでない, 穢すでない, 淋しがるでない) in 青空文庫.
As for the etymology, unfortunately I could not find a good article which explains this. One answer in this chiebukuro question says:
So it seems to have originated from somewhere in Kansai region in the Edo period.
In this question two people say it's 江戸時代の武士の言葉, but I don't know whether real samurai in the Edo period actually spoke in this way.