It would require serious philological investigation to figure out the exact situation w/r/t to 衡器, スケール and はかり, but there are a few suggestive pieces of data easily available.
日本国語大辞典 has one example sentence for 衡器. It is from 1909, from a weights-and-measures law ([度量]{どりょう}衡法{こうほう}施行令{しこうれい}):
度量衡器の製作の免許を受けたる者は ... 度器{どき}、量器{りょうき}又{また}は衡器の[修覆]{しゅうふく}[及]{および}販売の[業]{ぎょう}を営{いとな}むことを[得]{う}
"An individual who has received a license to create measuring devices (度量衡器) [...] is permitted to run a business making and selling length-measuring devices (度器), volume-measuring devices (量器) and weight-measuring devices (衡器)."
I'm not a legal pro so this may not be a good translation from that standpoint, but what is of interest to us here is that 衡器 is used in opposition to 度器 and 量器, which are of the same form except for the first character. As far as I can tell, a 度器 is basically a ruler and a 量器 is basically a cup (as in "2 cups of flour"), although of course scaling up to industrial sizes and adjusting for cultural norms (the "cup" might be square). And note that all three are joined together at the start in the word (or is it a phrase?) 度量衡器.
Aozora Bunko has two hits for 衡器, both from the first half of the 20th century, and both giving it the furigana はかり.
So here is my theory. Everything beyond this is speculation.
The word はかり is a native Japanese word, obviously related to the verb はかる, to measure. This word suffices for the concept of "scale" which, as you note, have been known in Japan for a long time.
However, when you are writing in kanbun (Chinese, basically, although often "with Japanese characteristics") or heavily kanbun-influenced Japanese -- which you are, if you are writing official documents like the Meiji laws -- you can't use the word はかり because it is Japanese. You could just assign the word はかり to a single kanji like 衡, and this may have been done in some cases. But there is no guarantee that people will know to read it はかり; they may confuse it with one of 衡's many other meanings.
So you use 衡器 instead: "device for 衡ing." This is much less ambiguous. (I don't know whether the word 衡器 was invented in China or Japan, but either way, the principles are the same.) You now have a word 衡器 that means the same thing as はかり, and many people will actually pronounce it はかり. But you can also force a Chinese pronunciation from the on-yomi of the individual kanji, and that would be こうき. This would make it one of those words "artificially created on the spot" as sawa explains.
And so the word こうき is introduced into Japanese, but it never really gets beyond the formal contexts it was created for. (It probably doesn't help that こう is one of the most common on-yomi in the language; there must be dozens of other こうきs out there.) はかり remains the "default" word for the concept of a scale. Even when popular writers use the spelling 衡器, they still indicate the pronunciation はかり. As the formal contexts where こうき is required grow rarer and rarer, the word fades into further obscurity, although it does live on in dictionaries.
(The fact that the word スケール is borrowed from English at some later point is a side issue. It might have come along for the ride with some new technology -- a new type of scale, somehow distinguishable from the old はかり mechanism. Maybe the electronic scale, as Chris suggests. The point is that スケール does become fairly common -- enough for a shop assistant to recognize it when you used it -- although according to sawa it never displaces はかり.)
