青 ao seems to be used very much interchangeably for both blue and green. Why is that so, and how does 緑 midori play into this?
Beside some of the historical examples mentioned by Derek, there is also an inherent nuance that separates 青 from 'blue', as it is commonly understood in Western culture (and similarly, albeit less strongly, for 緑 and 'green'). This is not unique to Japanese-English and probably applicable to any pairs of sufficiently separate cultures: colours are, for a large part, an artificial construct and there is only limited reasons two people should pick the same arbitrary frontier along the green-blue continuum (or red-orange, or yellow-green etc). I recommend digging in Google Scholar for more on that, but it's worth mentioning that there is much debate on what the causes and extent of these differences between cultures are.
Anyway, back to 青/緑: leaving aside the fact that the past 100 years of intermingling with Western culture have no doubt influenced the native Japanese perception of these colours, there is still a real difference between the range of what a Japanese will call 青い and a Westerner call 'blue' (talking about pure colours here, not objects' traditional colours, which might be tied to historical reasons). 'Blue' for 青 and 'green' for 緑 are just approximations (as are probably most other native colour translations).
A diagram might be the easiest way to put it. Assuming that horizontal line represents the true continuum of hues from blue to green, and the vertical bars, the separation between the two colours in Japanese and English respectively, you'd have:
青い | 緑 ーーーーーーーーーーーーー Blue | Green
As a result, when talking about something on the far left (say, the sky) or the far right (say, fresh verdure), both English and Japanese words agree quite well. If you pick things that are in that middle area where the definitions do not match, you get these quizzical looks and people arguing "what do you mean green? it's obviously blue!" etc.
Sorry for the longwinded answer to what is a fairly basic/obvious point... ;-)
This page in the 日本語Q&A over at ALC addresses this question. Apparently the historical definition of 青【あお】, even when defined narrowly, covered an entire range of colors which are today separated as 青【あお】, 緑【みどり】, and 藍【あい】 (indigo). This trend carried into the modern language, and many words which refer to things that are actually 緑【みどり】 still use 青【あお】: 青葉【あおば】 and 青竹【あおだけ】 are two examples given on the linked page. (This in spite of the fact that 緑【みどり】 has existed alongside 青【あお】 all this time.)
Apparently when traffic signals came along, 緑信号【みどりしんごう】 was the official name at first, but it was eventually overtaken by 青信号【あおしんごう】 in common use and this latter name became the official term.
This partially also has to do with the fact that in old Japanese they only used four colours: あか・あお・しろ・くろ. Obviously, with this limitation, あお came to represent a wide range of different cool shades. Then once more "colours started to be used", a lot of things retained their original descriptions as あお.
Edit: Here is that handout I have from my Japanese teacher which I describe below in the comment. It is from a book she has, but I don't have the reference for it.
- 昔の日本では、あか（紅）・あお（緑）・しろ（白）・くろ（黒）の四色 この四色は、「青く[ない]、青かっ[た]、青い[。]、青い[物]、青けれ[ば]...」と活用するように、 「青い・赤い・黒い・白い」といずれも形容詞になります。 それに対して、緑は形容詞として「緑い」とは使えません。 また「青々と」「赤々と」「黒々と」「白々と」のように副詞にできるのに対して、「緑々と」という表現はありません。 このような点から、青・赤・黒・白の四語と緑は、別なのではないかと考えられます。 文献上で「緑」が用いられるのは、平安時代ごろからです。 ＊注）平安時代：794〜1185 それ以前は、青いが黒から白までの間の広い範囲の色を表したようです。 （特に現代の青・緑・藍の三色を表すことが多かったようです。）
- 1930年に日本で初めて信号機が設置されたとき、 法令では緑色信号だったのですが、新聞が「青」と掲載してしまったことや、 色の三原色、赤・青・黄にあてはめると理解されやすかったために「青」信号が定着してしまいました。 1947年には法令でも青信号と呼ぶようになりました。 日本語の青を表す範囲は広く、「青りんご」や「青葉」など緑のものも青と呼んでいます。 ちなみに現在の信号機は緑ではなく青緑色で点灯前は青になっているはずです。
- あお 青・蒼・藍 → 空・顔色・信号・草