Battles in Civ4 are less random and even carry light RPG elements. I've never played Civ5 but from what I've heard, the combat in general is improved but the AI is too dumb to adjust to it.
The AI is super dumb in Civ 4 (and 3) as well, it's just that 5 having an interesting combat system makes it more obvious. the curtain is lifted and you can see that the AI doesn't always really understand the game it's playing.