I'll take a stab at this with my personal experience and some dictionary entries:
付近 - This just means a place nearby. The dictionary examples, 駅の付近をうろつく and 付近の図書館, just show an idea of general "nearbyness." In English you might liken it to "around" or "in the vicinity of" or even just "area"
近辺 - The dictionary's definition is 近いあたり。近所。近傍。付近。「学校の―」「新宿―」 so I think it's safe just to call this one a synonym of the others. If you look at the kanji it's just a simple combination of near and area, so it's a nearby area. It's worth noting that 近辺 only has 43 entries on alc, so it would appear not to be widely in use.
辺り - This is another one that is defined basically in terms of the others, however this one has a much more general meaning of nearby. For example, you can say ビルの辺り to refer to an area near a building, or 目の辺り or 首の辺り to refer to someone's eyes or around one's neck.
近所 - This is the most basic word that you learn for "neighborhood," as in your neighborhood where you live. In its various uses it retains the meaning of being around where you live.
周辺 - This refers to the outer edges of something, like the outskirts or periphery. It has this literal meaning but can also be used in a more metaphorical way, like to describe the people around you.
辺 by itself is one that I hear just used to point out a general location, like そこら辺 or この辺.