I'm aware that the so-called 'f' in ふ isn't pronounced with the English labiodental fricative [f], but rather the bilabial fricative [ɸ]. I'd like to know if Japanese speakers also use the bilabial fricative to pronounce は, ひ, へ and ほ. The reason I ask is because in romaji, I see these written as 'ha', 'hi', 'fu', 'he', 'ho', and wondered if ふ was a special exception in terms of how it is pronounced.
It is most definitely an exception. The actual phonetic realisation of that series goes like this:
In Middle Japanese they all were pronounced with [ɸ], which you can see in European transcriptions of names from the 1500s and 1600s - the Portuguese wrote e.g.
The ɸ>h sound change hasn't quite finished for many speakers, and so /h/ for them remains [ɸ] before /ɯ/. This probably has to do with the particular details of how Japanese /ɯ/ is articulated - it's ultimately neither rounded nor unrounded, but 'compressed': unlike a canonical [ɯ] your lips are pulled together, but they're not pushed out like with [u]. (I'm writing the Japanese version as <ɯ> anyway because there's no good transcription of the 'compressed' version.) [ɸ] is retained because your lips are doing mostly the same thing that they're doing with the 'compressed' [ɯ].
For other speakers, the ɸ>h sound change is complete, and ふ is [hɯ].