I remember whooping that guys assYou can also go north of the Inn first and pick up Ajantis before that fight, as long as you don't mind fighting your way through hobgoblins to get him. You can even run over to the guards and they'll help defend you. A lot of people still had trouble with that fight, though.
Guess it depends on if you pick up the 2 crazy assholes on the way or not