Im also almost done reading A Game of Thrones for the first time. Ive been enjoying it a lot - season 1 has always been my favorite (or season 4, depending on the day) and its nice to experience it again. Im constantly visualizing the characters as the actors because they did such a great job.
Im at the part after Ned is captured and Jon first encounters a wight. My next chapter is Bran so Im not too eager to go back.
I don't have the same issue most people do who read after they watched. Because the character descriptions are so different mostly, I don't usually picture the actors as I read. Joffrey is one of the only exceptions, and actually Jaime too for the most part. I just can't picture the Lannisters with curly hair. But I do picture them a bit younger like in the books, especially Cersei. I don't mean to be mean to Lena Headey, because she is definitely pretty, but Cersei is supposed to be more. I picture her super adorable, and even meaner.
And I always picture Tyrion with the extra ugliness, and a higher more imp like voice. And Tywin I picture as way bigger, like a slightly more dad bod with extra hair version of the Armstrong dude from FMA. With the golden hair and mutton chops to make him look like a lion. I love Dinklage and Dance(the superior DnD of the show.) for their acting ability, but their appearance doesn't fit for the book versions at all.
And Robb is supposed to be more buff too, I picture him like a nice version of a football jock. Same with Renly, he is supposed to be a big buff dude who looks like a younger Bobby B. And actually I do kind of picture Mark Addy as king Bob, except taller with broader shoulders.
My sentiments exactly, especially concerning Cersei; as you said, not taking anything away from the lovely Lena Headey, but I always pictured Cersei having almost elfin-like beauty and really big, intense crazy-eyes. I also didn't get a good mental picture of book-Renly until my second readthrough and really paid attention to the descriptives. I really liked how he was basically a non-brutish version of Robert. ---
Simple questions deserve long-winded answers that no one will bother to read.