"先月何日会社を休みましたか?" itself is quite vague in this manner, as it can be translated into both "how many days did you take day offs last month?" and "what date did you take a day off last month?" depending on the context.
If you want to specify "how many", question should contain something like "何日間" or "どのくらい".
If "what date", use "いつ" or "何日に" instead of "何日" for question.
For answer, "四日" can be used in both cases, but if you want to specify, "四日間" (sometimes mean "in a row") or "四日分" ("in total") for "how many", and "四日目" for "what date".
The sentences would be like this:
Boss: 先月いつ休みましたか? (When did you take a day off last month?)
Me: 四日です。 (Day 4.)
Boss: 先月どのくらい休みましたか? (How many days did you take day offs last month?)
Me: 四日です。 (4 days.)