Let me preface this by saying that there is a time and a place for everything. Some days you may want to dress up and some days you may have to dress up to look professional. You are in college to hopefully get a job one day after all.
However, if it's just any other day there is no reason why you shouldn't be allowed to wear whatever you want. It is not a requirement to wear makeup every day. It is not a requirement to put pants and uncomfortable shoes on every day.
You should be allowed to walk around feeling comfortable in your own skin without fear of being criticized for what you are wearing.
Sweatpants and t-shirts are undoubtedly the most comfortable things in this earth. And there's a good chance you spent the last 12 years under a strict dress code and you will get a job shortly after college with a similar dress code. College is the only 4 years of your life where you don't have someone telling you what to wear.
So, wear your swear pants. Wear that oversized t-shirt. Stop worrying about the blemishes on your face and the bumps in your hair. Be confident in who you are because you are perfect.