This is true but he's an ass in general. He's supposed the be a source of conflict you can't get rid of.
It's explored a little later but being concerned about what someone's crimes were before the zombie apocalypses is a little silly, when Lee is obviously one of the most useful, level headed people in the group. There are people who have no criminal record but are useless nervous wrecks or madmen after the zombie apocalypses.
I rather liked Larry. Not that I didn't like Lee, but everyone in the game has their own shade of gray. Yes, he was kind of a dick to Lee, but from his perspective, he has a daughter that's clearly taken a shine to Lee and Larry knows about his past. As you said, pasts in a zombie apocalypse may be looked at through an adjusted lens, but Larry is still looking out for his little girl and Lee is a convicted murderer.