While many people enjoy listening to music there are some that just don't like it. It's not because they haven't found the right genre or band, it's because they may have musical anhedonia. According to Curiosity "about 3-5% of the world's population has an apathy toward music."
To have musical anhedonia means that when listening to music your brain does not behave in the same way as when other people listen to music. When one listens to music they hear it in the auditory cortex of the brain, which then connects to the nucleus accumbens — the reward center — and that is where they experience a response to the music, as explained in the study "Neural Correlates of Specific Musical Anhedonia."
When you have musical anhedonia, the connection between your auditory cortex and your nucleus accumbens is not as strong. This makes you experience lower levels of activity in your nucleus accumbens, meaning that you don't have any rewarding feelings toward the music. Although this doesn't mean that you hate listening to music, you just find no joy in doing so because your brain isn't responding to it.
The same goes for people on the opposite side of the spectrum. About 55-86% of the population experiences frisson, according to Mitchell Colver Ph. D. Frisson is when you experience "aesthetic chills" and happens most often from listening to music, but could also be triggered by movies, art, and physical contact.
Now one reason why you may get chills from music is that you are someone who "intellectually immerse[s] themselves in music" instead of just letting it fly through your ears, as said by Colver. The other reason works in the same way as musical anhedonia. According to Ben Taub, this is because the connection between your auditory cortex and your ventral striatum — who's main component is your nucleus accumbens — is stronger you have a higher sensitivity to music and thus a larger ability to feel emotions from music.
So if you don't enjoy listening to music keep trying to find an activity to spark your brain's reward center, and if music gives you chills keep on listening.