He is has been working since the morning. She comes in and says
Hey, being exhausted from this little bit... Don't you think you're just a bit out of shape?
That's not exactly something you'd tell someone, who has been working here since the morning.
働かせる is "to make so. work"; っぱなし means to do something and let it stay that way, although whatever was done should really have been undone. An alternative translation would be
That's not exactly something you'd tell someone, who has just been left working here since the morning.
The reason why the passive can't be used here is that っぱなし means "to do sth. and leave it that way", whereas 働かせられる is passive (i.e. not active). 働かせられっぱなし sounds like he's left himself being put to work, when he should really have stopped. So in this case, 働かせっぱなし doesn't refer to an action by 人間, which would be natural, but to someone else.