I have learned the hard way. When it comes to picking between yourself and someone else. Choose You!! It may never be easy but do it. Do it for yourself. It's what you will need in the long run. I have been there and currently going through it.
You may think certain people will never leave your side and will always bother for you and you may be right. But there will always come a time where that is not true at all. The best thing you can do is put it behind you and choose you. Do what you want to do, and make your life how you want it to be.
It's your future not theirs! Do what makes you happy for the long run. You are the one that has to make these decisions for self to pave the path of your life. Make yourself happy!
Someone told me when you get in that place where you feel like you will never get out of think of 3 things a person who means the world to you, a purpose in your life, and your favorite place to be or go. It's honestly the best advice I could have received at that time.
Make your life what you want it to be! Don't be afraid to tell others no!