I feel like I'm always the one who feels like I'm being withdrawn from
Remember, everyone experiences loneliness at some point in their lives. You're not alone, and things will get better with time and effort.