I would like to mention that yes, I have done it. It wasn't a toxic relationship but I was uncomfortable and I let them go. I am happier for it and trust me, you will be too.
It's not pretty being in a toxic friendship. It doesn't even have to be toxic. The moment you feel uncomfortable around them, the moment you are stressed to be near them, or you feel scared that they won't listen to your opinions, it's time to let go.
I'm not the kind of person that finds it difficult to let go of people. It is a cycle of life and sometimes, people go and sometimes, they stay. Relationships evolve and people evolve; hoping that things stay the same is a pointless endeavor.
But it is definitely hard to understand that you need to stop talking to someone or else it will make you sadder, more different, move you in a direction that you aren't comfortable with and that changes who you want yourself to be.
It is hard to accept that there is nothing to do now, there is no way to fix it, and you must just stop talking to each other.
You can try your best to explain to them what the issue is, try to resolve it, try to keep that friendship working but sometimes even after that, it simply doesn't work and you do not know where to go from that.
It's ok. It doesn't make you mean or cruel or bad. You are just taking care of yourself and you should. We all should take care of our own mental health before others because we have control over our mental health, not other people's.
Not being friends with someone that you were close to is so weird. You go on with your life and realize that you could fill your time with different things and sometimes, it seems preposterous that there was someone you spent so much of your time with because you don't have those big gaps that you expected to have.
College especially isn't the greatest with friendships. You come in thinking that you are going to find the ones that you will be friends with for the rest of your life and sometimes, you just lose touch and don't talk that much anymore but others that you didn't expect you would become closer to turn out to be the ones that you spend a significant time of your days with.
Friendships are hard and it is harder to even understand if a friendship is blossoming or not but with significant experience, you will realize that your time is precious and spending it with those who you actually care about will make you the happiest.