It's a psychological thing, as you said.
Nobody expects a grown man to be a fan of a pony cartoon. Therefore, by being a fan of it, you're breaking social norms. You're being a rebel. People do make such a big deal out of individuality these days.
Of course, there's a large fanbase, so you can feel fine breaking the social norm with TONS OF OTHER PEOPLE.
tl;dr: A combination of a desire to fit in and a desire to be different.
--
Don't worry, Bill Murray!
Its things like this that make you wonder why the company that makes M&Ms don't make butter