Question: What would it mean to say that mathematics was invented and how would this be different from saying mathematics was discovered? 

Is this even a serious philosophical question or just a meaningless/tautological linguistic ambiguity?

Answer: 
I'm going to posit, admittedly without any research whatsoever about those who've preceded these thoughts, that **an "invention" is a *kind* of "discovery,"** and that whether a thing qualifies as an invention is—yup, you saw it coming—*subjective*.

For example, we might say that the wheel was "invented" on grounds of (1) **non-naturality** (**originality**), and (2) **intention**. That is, prior to the wheel, circle-and-axle forms did not exist in nature, and so of course no one could apply it to the facilitation of movement. Furthermore, it's hard(er) to imagine someone carving a circle with a hole, then carving a spoke, then putting the two together, without having the *intention of rolling* the circle on the spoke, in mind. These circumstances give us cause to say that the wheel was "invented."

But, it's not impossible to imagine, either, that someone might have carved a circle with a hole for absolutely no reason to do with the concept of rolling, then happened to stick a stick in the hole (again, for no premeditated or relevant reason), and only *then* (or sometime later) realized its property of rolling. Note how in this case, we're more inclined to call the wheel a "discovery!"

**I think we tend to call novel discoveries with premeditated results, "inventions."**

So, I would say mathematics, as a general notational/deductive system, was mostly invented. But its concepts were discovered. (And even some notations were indeed discovered, while striving for convenience, concision, and pictorialization!)
I consider an answer too simple if it just affirms one of the alternatives and negates the other.

Naming just a few eminent contributions to mathematics: Complex numbers, set theory, theory of schemes. E.g., the concept of a set has been invented by Cantor, it did not exists before. After the basic concepts like set, power set, cardinality etc. had been invented, the Continuum Problem was discovered, hidden deep in these concepts. 

Therefore I compare mathematics to a game like chess: Inventing new mathematical concepts is like creating new rules of the game. Playing a match means to discover the consequences of the rules and to solve the problems posed by the rules. 

My conclusion: The rules of the game of mathematics have been **invented**. Following the rules mathematicians then **discover** some challenging matches.