Question: What would it mean to say that mathematics was invented and how would this be different from saying mathematics was discovered? 

Is this even a serious philosophical question or just a meaningless/tautological linguistic ambiguity?

Answer: 
I'm going to posit, admittedly without any research whatsoever about those who've preceded these thoughts, that **an "invention" is a *kind* of "discovery,"** and that whether a thing qualifies as an invention is—yup, you saw it coming—*subjective*.

For example, we might say that the wheel was "invented" on grounds of (1) **non-naturality** (**originality**), and (2) **intention**. That is, prior to the wheel, circle-and-axle forms did not exist in nature, and so of course no one could apply it to the facilitation of movement. Furthermore, it's hard(er) to imagine someone carving a circle with a hole, then carving a spoke, then putting the two together, without having the *intention of rolling* the circle on the spoke, in mind. These circumstances give us cause to say that the wheel was "invented."

But, it's not impossible to imagine, either, that someone might have carved a circle with a hole for absolutely no reason to do with the concept of rolling, then happened to stick a stick in the hole (again, for no premeditated or relevant reason), and only *then* (or sometime later) realized its property of rolling. Note how in this case, we're more inclined to call the wheel a "discovery!"

**I think we tend to call novel discoveries with premeditated results, "inventions."**

So, I would say mathematics, as a general notational/deductive system, was mostly invented. But its concepts were discovered. (And even some notations were indeed discovered, while striving for convenience, concision, and pictorialization!)
My view on it is that Mathematics is a system invented by humans to represent things we otherwise can or cannot perceive. For example, we can perceive an object through vision and know it's a triangle, however, our vision alone does not tell us the length of the legs of the triangle. We need math to represent that for us.

JUst to further my point, consider Calculus. Two people who were on completely different sides of Europe, Leibniz and Newton, created a system that that both do the same thing. For Newton, f'(x) is the same as Leibniz' df/dx. Both of them yield a function that represents the slope at any given point on the original function, f(x). They invented a system to represent something we otherwise couldn't perceive (which was pre existing - The shape of a mountain should be enough to prove that the slope exists naturally), the only difference was their notation.