I always had trouble with abstract math in general (anything pre-algebra and up), mainly because in K-12 I had like maybe four worthwhile math teachers, while the rest were utter garbage who only still had jobs because tenure exists (one of the reasons why I'm hugely against the entire idea of tenure for teachers, or at least tenure without some form of periodic review).
Because Math is THE subject where "building on what you've learned previously" is so damned important, it meant that every year I fell behind because of a shit teacher made every successive year harder.
It even reflected in my SAT scores - I scored a 780/800 on my Verbal, but only 530/800 on Math.
I managed to make it to pre-calc levels in college (mainly because of multiple statistics classes for Psych), but I've always struggled a bit with the abstract side of math whenever I deal with it. Whereas I've always been pretty good at being able to handle more concrete math (addition/subtraction, multiplication/division, fractions/percents, etc) in my head on a moment's notice.
What hurts most though is that it's been decades since school for me. And while I still use concrete math fairly regularly (calculating discounts, tips, measurements, etc), abstract math doesn't come up as often in everyday life. So most of what I did manage to pick up has kind of atrophied (in the same way that I took 7 years of French in school, but if you dropped me in Paris today I couldn't even ask my way to the nearest bathroom).
Interesting response. Yeah, teachers can make or break a student. And yeah, abstract math can be harder to grasp and retain for the reasons you listed.
---
PSN: killersalmon / Epic: aliensalmon1986 Nintendo Network Name: JohnJohn