Split the first sentence into these two pieces:
The first part is as you mention "(Because) you're walking slow"
The second part's key phrase is an altered form of 「に決まっている」which means something like "of course this is the reason why..."
Putting the two together, this would be "It's because you're walking too slow (of course)!" as if to pin the blame on you for the group being late as a foregone conclusion. The next sentence then goes on to say "there's no time to waste!" as if to tell you to hurry up.