There are special words for counting a small number of people, days, and so on, but let’s begin with the regular pattern.
Usually, the word which describes a number is just the combination of a numeral and a counter word. The numeral does not change depending on the counter word, and the counter word does not change depending on the number. This is what user1205935 and dotnetN00b wrote. However, the precise form of the words describing numbers is a little more complicated than this, even in the regular pattern.
If you know that 一匹のリス (one squirrel) is read as いっぴきのリス and 二匹のリス (two squirrels) is にひきのリス, this should look contradictory to what I wrote in the previous paragraph. This is because the last mora of the numeral and the first mora of the counter word are sometimes fused together. Whether this fusion occurs and how they are fused depend on the ending of the numeral and the consonant in the first mora of the counter word; see Wikipedia. In [一匹]{いっぴき}, the numeral [一]{いち} and the counter word [匹]{ひき} are fused together.
Because the numerals for 11 ([十一]{じゅういち}), 21 ([二十一]{にじゅういち}), 31 ([三十一]{さんじゅういち}), and so on end with 一, the pattern for these numerals is the same as the pattern for 一.
Now you might think that the pattern depends only on the last digit of the number, but that is incorrect! Why? It is because the numerals for 10 ([十]{じゅう}), 100 ([百]{ひゃく}), 1000 ([千]{せん}), and 10000 ([一万]{いちまん}) are completely different, and they are fused with counter words in different ways.
As I wrote earlier, there are some exceptions to this explanation. For example, “one person” and “two people” have special words ([一人]{ひとり} and [二人]{ふたり}, respectively), while the case with more than two people follows the regular pattern with the counter word [人]{にん}. Wikipedia lists most notable exceptions.
See also the question “What are the rules for reading numbers before a foreign counter-word?” by silvermaple. (Despite the title, the answers given there are also applicable to the counter words which are not loanwords.)