OP, he might be mesmerized by the bright colors of your workout caftan, following you about like a cat chasing a toy. Or he might be a harassing stalker, following you about like Jeffrey Dahmer chasing a Thai 14-year old. It's hard to tell without pictures and better re-enactments.
What do you do? I hate to see people cave in these situations (that is, you quitting the gym). But if he's dangerous and you value your life, then do what you must.
I'd handle it like this: Tell the gym's manager that you are having a problem with a guy following you and making frightening comments under his breath. Tell the manager you'd like to handle this on your own, this one time, but if there are further issues, you'd like their assistance. Then calmly tell the harasser, avoiding most eye contact, that you are sorry that you bug him somehow, but you'd like him to stop following you around. You are being meek here not because you're a wimp, but because you're avoiding escalation. This is not the time for humor, sarcasm, or anger, or for making any negative characterizations of his behavior, weird as it is.
That is all you say. Tell him thanks, and walk away. Whatever his reaction to your statement is, ignore it. Do this in a very public place. If he continues any harassment, talk to management again, asking them what they can do to help you. If they don't respond, I'd at least contact the police and request to make a report, and I'd contact the gym's corporate management if there is any. This behavior is illegal in many places.
Or, you can follow it up the way I did a guy who stalked me at the ASU gym. He made weird grimacing faces and was often too close for coincidence. He wouldn't make eye contact most of the time. I started by nodding acknowledgement. Some days later I said "sup?" He nodded. Not long after that, he was waiting outside the gym. I sat down nearby and gestured him over. We talked. He was a Marine, was in one of my classes - I hadn't noticed him there as it was a large class and he skipped a lot of them I suspect. He needed a tutor. I tutored him.
We watched basketball games, drank beers, smoked up a few times. Eventually he expressed his curiosity about blowjobs - giving, not receiving them. We had sex, I remember I shoved a Bud Light bottle neck up his ass. He told his mother, who telephoned me asking about who was turning her son out and asking me to stop because she wanted him to get married and make her grandchildren, which was perhaps the most surprising part of this story to me.