Question: In a world of magic/superpowers, would there be any demand for sports or games that didn't use magic? Magic here can be though of as a science and is mostly accessible to people who want to learn, but does have a bit of a steep learning curve at the beginning. It can be self taught if one wishes to put forth the effort and there's no ritual/incantation attached, so things are a bit more streamlined (Though, many will claim that you get better and more potent results with a good ol' fashion chant). In fact, magic here is so researched, it can be measured, enhanced and manipulated to the point where they have it down to a science.

Obviously, with the existence magic comes the potential for all new forms of sports and games. Flashy spectacles of daring and thrilling competition, teams raring to show of their adroit tactics and skills. People hold local tournaments and line the stadiums to see who will come out on top!

...That said, with such high octane thrillrides of magical sports and games, does anyone even care to watch or play the ones that don't involve spells and powers being flung around? Would things like football and baseball even have a chance against the likes of something such as that? 

Things that you can assume:

* It is measurable. This includes a person's potential. This way leagues can be created to divide according to aptitude. And it can all be regulated with well defined rules. (Usually people can go up in ability level after lots of practice, usually they don't go down unless they haven't used any magic in forever.)
* Not everyone knows how to use magic. Everyone can learn, but somepeople don't bother, because it doesn't pertain to their daily life and it would be a hassle. (Still most people do take the time, and a lot of times it can be taken in school as an elective)
* Magic has been around for a *very* long time. Thousands of years and is pretty ingrained in society for the most part.

So, do people care about the nonmagical variants of sports and games, or would they have died out when magic began to take over?

Answer: 
"Any sufficiently advanced technology is indistinguishable from magic" - Arthur C. Clarke

We have a world of wonder and excitement right now. Roller coasters, video games, skydiving. There are places to find thrills that we never imagined years ago. E-sports are a thing now. We have a computers that are the best in the world at ancient board games.

Even with all this wonder, we still love soccer and hockey and football. We will always have those: they are a part of our culture, and we're not giving them up. There will be magical sports, but there is another joy in restricting your ability with strict rules for the challenge of it.
E-sports are a field that is currently expanding massively, being almost unheard of only a few years ago, and not even existing not long before that. However, with advances in technology, they are becoming better, and are now a huge deal. 

There is, however, a lot of stigma attached to e-sports by people saying they're "not really a sport", which I can imagine would have been a real issue for magical sports events when magic was discovered.

This would have stagnated the development of magic sports initially, and even 1000 years later there may still be some naysayers who don't see it as a pure sport, particularly if they don't have a natural affinity with magic. Or vice versa, with physical sports no longer being seen as proper sports in favor of magical ones.

As well as having purely physical sports and purely magical ones, you would also likely have sports that combine the two, or using magic to enhance the capabilities of physical sports players.

Think about how much technology is used to develop better cars for Formula 1, or how much statistical analysis is used in baseball. American football players watch back replays of their games to see what they did right and wrong in their plays and iron out any weaknesses. It's likely magic could be used to make physical sports better and cause them to evolve, rather than replacing them outright.