Question: I believe I.M. Gelfand said that when beginning to learn a new subject, one should learn it like a physicist.

In this spirit, what are some helpful and surprising physical intuitions accompanying mathematical objects?

I especially am interested in intuitions that suggest nice proofs...and helpful new ways of thinking.

Answer: 
Well, there are of course many examples but perhaps the Dirac $\delta$-function is the most striking one. With quite some violence to the math of usual calculus, Dirac's idea turned out to be exremely useful: the "function" everywhere zero and very high at $0$ so that the integral is $1$. Applications are ranging from Fourier analysis, (linear) PDE, quantum mechanics, and many more. Now of course we know how to make things rigorous in the framework of Schwartz's test function and distribution theory. But the intuition is clearly from physics.

Similar and also in this framework is the notion of oscillatory integrals...
For beginners, Polya's Induction and Plausible Reasoning has some nice examples of physical reasoning. For example he has a chapter on a shortest path problem that is solved mechanically and optically.