Like pretty much all pronoun (hell, all politeness-level related) issues in Japanese, there just isn't an absolute answer: it's all down to context and to the nature of your relationship with the listener.
The short answer is: a lot less rude than you may have been led to believe
I do remember being given very stern warnings (in manuals or language lessons) about never using it in polite company, lest I gravely offend people by my lack of humility etc. etc.
In reality, as Oren pointed out, young people use it all the time, including in semi-formal situations (e.g. when talking to people they do not know very well).
I recommend you read through the many questions that have already been posted on the proper uses of 私 vs. 僕 vs. 俺: they will probably tell you all you need to know about the situations in which it is OK to use one or the other. But for a quick summary:
Some examples of situations in which 俺 is indeed going to sound very rude:
Talking to your boss or a superior at
work.
Talking to an elderly person you do
not know well (or any elderly,
really).
Typical situations where 俺 is
perfectly innocuous:
Talking to your male
friends/schoolmates/coworkers of same
or lower rank (in age and/or
position).
Talking to people you know
intimately (family members,
girlfriend...).
Situations where it can go both ways and you should rely on context and your better judgement:
- Talking to senior coworkers/friends (i.e. 先輩【せんぱい】)
- Talking to girls or love interests you are not particularly intimate with: I recommend you read up on the 僕 vs. 俺 debate for that one. Some people will recommend 僕 for its "cuteness" and harmlessness, while others will point out that 俺, making you sound like a manly man, is a better fit for courtship.
- When talking to perfect strangers (e.g in a store etc.): I hear it done all the time... Probably depends a lot on your tone but generally safe.
Provided the rest of your sentence and your tone convey the appropriate level of politeness, I generally don't think 俺 will offend anyone. A much higher risk, ihmo, is that it will make you sound slightly silly, because out of character (i.e. more boastful than your age and position allow you to). Rule of thumb is that you should never use it with people that are your father's age or above ;-)