To answer the last part of your question first, 数 is part of this for it's pronunciation, not meaning. This is called ateji, which is when kanji are used to phonetically represent words. 寿司 (すし / sushi) is a common example of this. Translations that follow are my own.
According to the Japanese Wikipedia page for 数寄者, 「数寄」originally held the same meaning as 「好き」 and remains in its ateji form due to its now distinct meaning. It shows particular fondness for artistic pursuits. However, in modern usage, its usage is approaching that of 「物好き」 -- which, while literally meaning curious or whimsical, generally holds something more of a negative, even strange connotation (as I understand it).
Looking in Goo's dictionary, three definitions are listed, roughly analogous to those in WWWJDIC:
- A curious (物好き) person. A person of fantastic taste.
- A person devoted to elegant tools, especially those related to tea ceremony. A refined person.
- A lecherous or lustful person.
It also notes in #1 and #3 that it is a synonym for すきもの.
I see two primary possibilities for meaning #3 has come to be. The less likely of the two is the actual meanings of the kanji used. 「数」 in its sense of many, and 「奇」 in its sense of strange or eccentric (as seen in 奇人 kijin) may have combined to roughly mean person of many eccentric tastes, which could have shifted to its current meaning. I think this is likely reading into it too much, and is unlikely.
The other possibility is its association with the word すきもの, which is another possible reading for 「数寄者」, as well as the only reading of 「好き者」. WWWJDIC lists "lecher" as the only definition for 「好き者」, while Goo lists "eccentric person" as well. Despite that, it seems likely that either a misreading of or simply sharing similar pronunciations lead to this meaning being associated with 「数寄者」.
