I know the most hardcore fighter guys are partial to Guilty Gear or KoF for 2D series, but I never knew what the "elite" 3D series was... if there even is one.
-- SephirothG, channeling awesomeness from Mershiness. The Resurrection
I'd say Soul Calibur if we ignore the most recent entries.. Hell, the same could be said for Tekken. Virtua Fighter seems to be the only one on top of it's game in the current gen, but it's also the hardest to get into... so not many people even have an opinion on it.
Popular opinion is Tekken, super Hardcore opinion is Virtua Fighter. VF is too much of a time investment tho for how little people play it, so even to know a lot of people like it better than Tekken, Tekken will still be played more because more people play it. Both of them are absolutely retarded in terms of learning curves tho and Tekken's learning curve isn't even possible for 90% of people because it requires you to play often against Every character in the game because it's 100% match up and frame punishes. VF is a lot option selects and is also very heavy on the match up part too but has way less characters. Soul Calibur is the game most wish was good but always disappoints because it ends up being a broken mess that is anti-fun at mid level even. But at least you can get by in SC by knowing your character and general match up tips.