How do i say i need space without upsetting him?

So as my kind of wind down/ letting off steam time I love to go to the gym as its. away from everyone, don't get me wrong, i love my friends but I like to have some time to myself right? So anyway one of my friends has just joined the gym and i appreciate its a publiv place and all but he's been saying he will come and see me at the gym and without sounding mean i really dont want that and all of my friends understand that gym time= me time! Its different with him though as we are more than just friends but it hasnt been thay way for long and i don't want to ruin it. There is one friend that comes with me but I've known her my whole life amd its abit different. I just feel like he is crossing a line and it freaks me out a little as like i just. said, only my very closest friend comes to the gym with me and it almost upsets me as I feel like he is barging in. I don't kbow how to tell him this without upsetting him though :( xx