Going to high school, my classmates and I got thrown into a lot of classes required for graduation that covered specific "core topics"- central courses to the curriculum, but that wasn't necessarily of interest to us. While exposure to all of these topics is good for a well-rounded education in the state education department (it was a public school, after all), it was often harder for my peers and I to focus on all of these subjects- but it definitely makes honing in on one topic you love later that much more exciting.
The beauty of higher education, whether college or trade school or anything else, is that the schematic of your studies centers on whatever you choose. In my experience, yes, there is still a core curriculum of courses not directly related to your area of study. However, at many schools, it is considerably less restrictive. (Disclaimer- I can only claim familiarity with liberal arts colleges and music conservatories. However, I am sure that they are closer to the rule than the exception.) Even the years prior to that, you can often make the effort to shift your schedule more towards a certain subject if you want by adding extra-curricular activities. What does this mean, then?
It means your education is up to you. You get to pick what you become a little more of an expert in. So I say, spend time studying what you love.
Do it because it will foster skills in self-discipline. It's way easier to focus on your work when it isn't boring you half to death. There's so much to learn about any given area of study that, once you open the door, you'll have access to a whole world of knowledge. If it's something you love, interest transforms into curiosity- what better approach to an education than that? It drives motivation to learn and achieve. If you take advantage of your position as a student in an institution with educators who have done what you want to do, who have your best interests at heart, and who can, therefore, share golden advice with you, you can only go farther.
Do it because you already have the passion to go through with it. This is related, but not quite the same. If you love something enough to study it, it shows that you are willing to work for it. When it is going smoothly, the excitement to dive in each day is conducive to progress like nothing else. And when it gets hard and feels like the wrong choice, it will always be worth it once you get to the other side because of how much it means to you.
Do it because it can grow a community. Advice that college students hear all too often is to join clubs because they connect you to those with similar interests. Studying what you love means that you've already joined a group of people who like what you like, just by being a fellow student with a given major or just from sharing classes. That's not to say clubs aren't important- but it definitely gives you a head start.
Do it because there's no better feeling than working towards something you care about and knowing you've made progress. All of the points made above add up into not just growth in that specific area, but growth in your personal journey, and that is something really remarkable.
So whether it's the arts or the sciences, mathematics or communications, or anything else, I encourage you to go out and learn about what you love. There are so many possibilities if you allow yourself to grow nearer to them. Why miss out?