If you can't handle a situation that small yourself without going to a manager, you deserve to get only 4 hours a week until you quit.
Having personally been in the situation, if the coworker in question doesn't actually get that he's ACTUALLY bothering you (a common problem with the nu-male soyboy types these days) it's hard to make it stop without disrupting workplace dynamics
Minorities are generally in the position of "Do I make a big deal and make it stop, but be seen as a troublesome element... or Do I just let it slide and make work easier for myself" when it comes to these situations
That's when you have to go to management. But then you're doing the problematic minority thing, so you don't really feel like you've won... and then the situation in the OP happens
It's really not fun
Work on your people skills. It's not hard to tell someone if something bothers you without going behind their back and tattling to the manager.
And if you tell them about it then you're being combative, a trouble maker, etc. ---
"If you want to move fast, practice slowly...if you want to move like lightning, practice in stillness."