Here's the English equivalents for the IPA:
[ɡ] = the 'g' in 'get'
[ŋ] = the 'ng' in 'sing'
The main difference is that [ŋ] is a nasal consonant, whereas [ɡ] is not. If you try plugging your nose and pronouncing [ŋ], you'll realize that it's not possible. That's because air must flow through the nasal passage, but not the oral passage, for [ŋ]. The simplest answer to your question is that [ŋ] can optionally replace [ɡ] only in the middle of the word or compound (An Introduction to Japanese Linguistics. Tsujimura, 2006), so for example:
- ごま (sesame seeds) can be only [ɡoma]
- 下駄 (garden clogs) can be only [ɡɛta]
- 庭下駄 (garden clogs) can be either /niwaɡɛta/ or /niwaŋɛta/
- 縞柄 (stripped pattern) can be either /ɕimaɡaɾa/ or /ɕimaŋaɾa/
Words and phrases cannot start with a [ŋ]. Since [ɡ]→[ŋ] is optional, speakers are divided into 3 obvious categories:
- Speakers who never use [ŋ]
- Speakers who always use [ŋ] (when it is possible)
- Speakers who sometimes use [ŋ]
A study in 1941 showed that each category was equally about 30% out of 70 middle-school students (Nihongo no Hensen. Kindaichi, 1967).
Generally older speakers use [ŋ] more and younger speakers less. In fact, here is a graph of that relationship:
This graph comes from the paper Variationist Sociolinguistics in The Handbook of Japanese Linguistics (download here). Also, here is a map of the distribution:
It's a high-res map so to actually see it, you can download the original here. But basically the pink is [ɡ] and the green is [ŋ]. This map comes from a survey done by the National Institute for Japanese Language and Linguistics and you can find more maps like it here. Generally [ŋ] is more prestigious than [ɡ] and a little bit of history explains why:
- Before World War II, it was an emphasized point of elementary school education that /g/ should be realized as [ŋ] in non-initial position. Today, many trained television and radio broadcasters use non-initial [ŋ].
- In early nineteenth-century literature (Ukiyoburo written by Shikitei Samba), it is specifically mentioned that speakers in Edo had the nasal word-internally [ŋ] while rural speakers had the plosive [g]
- The results of these analyses, along with the earlier findings and observations, show that the change of word-internal [g] to [ŋ] in Tokyo Japanese originated in yamanote in the early twentieth century and has spread to the entire Tokyo speech community.
Also, there is no significant difference in usage by gender. As for the future trend:
An overall increase in the use of word-internal [g] in the vernacular is observed over several decades. The change is near completion, and the word-internal [ŋ] has been replaced by [g] at a very rapid rate within three generations.
So, [ŋ] is moribund (becoming extinct) at least in the areas major of Japan. As a JSL student you'd prolly be better of just forgetting it even exists. Besides all that, there's a bunch of interesting details and exceptions regarding the [ɡ]/[ŋ] alternation, but that's prolly best left to a separate question on it's own.