Yeah it's a combo of being very depowered + the servants are all weak fighters to begin with save Nezha + we're trying to save these people, not murder the shit out of them + if you die in the game you die for real. Makes combat a super unappealing proposition on every front.
Shine on, you crazy diamond.