Feminism is one of those topics that people tend to deem "controversial" as if fighting for equality across the globe is truly something that should be considered a dispute. It's obviously controversial with men, who refuse to accept and acknowledge the amount of privilege that has allowed them to dominate the world; they want the credit for it, they just won't admit that it caused women a lot of pain in order for them to get there.
And though in theory, it shouldn't be, feminism is controversial with women, too. These are the same people that think privilege is a dirty word. Or that acknowledging any sort of inequality somehow makes you sensitive or weak, synonymous with a "liberal."
The unwillingness to educate oneself about intersectional feminism and the disdain for those who have is often boasted about as it's some kind of personality trait. You don't have to be an activist or a feminist by any means but hating on women who are true doesn't do a thing besides revealing your internalized misogyny. The worst part is that many of the critiques from women who claim feminism is useless are based on misconceptions of what modern-day feminism really is. You see a photo of a woman protesting with her breasts out and assume that that's all that feminists do. Radical versions and extremes are the majority of what goes viral/what more people see so it's not best to make assumptions based on that.
Another claim against feminism that's rampant is the idea that just because one woman feels she doesn't "need" it, then neither does anybody else. Not only is this entirely egocentric but it is also ridden with privilege. Just because you and I aren't living in fear of things like a drastic difference in your pay or even worse — honor killings or female genital mutilation — does not mean that other people aren't. We have the ability to use our privilege to raise awareness for these issues. You may not want to but it's completely backward to hate on people who do.
An additional criticism is the idea that:
1. there are only two genders and
2. men and women are biologically different so equality is unattainable.
First of all, scientifically, sex and gender are two different things. Sex is biological while gender is socially and culturally constructed. Gender is a spectrum. Secondly, yes, men and women are different (duh) but that does not warrant inequality that constantly sets women back. You don't have to be a feminist but at least know the facts.
So no, nobody really cares that you're not a feminist and it's definitely not something to shout from the rooftops about. Hating on your own misperceptions of feminism doesn't make you different or cool or unique.
It's not going to change anything but feminism will. Because guess what? We're still fighting for your equality over here.