Question: So I've had at least two professors mention that backtracking makes an algorithm non-deterministic without giving too much explanation into why that is. I *think* I understand how this happens, but I have trouble putting it into words. Could somebody give me a concise explanation of the reason for this?

Answer: 
It's not so much the case that backtracking makes an algorithm non-deterministic.

Rather, you usually need backtracking to process a non-deterministic algorithm, since (by the definition of non-deterministic) you don't know which path to take at a particular time in your processing, but instead you must try several.
Thought experiment:

1) Hidden from view there is some distribution of electric charges which you feel a force from and you measure the potential field they create. Tell me exactly the positions of all the charges.

2) Take some charges and arrange them. Tell me exactly the potential field they create.

Only the second question has a unique answer. This is the non-uniqueness of [vector fields](http://en.wikipedia.org/wiki/Vector_potential). This situation may be in analogy with some non-deterministic algorithms you are considering. Further consider in [math limits](http://en.wikipedia.org/wiki/Limit_of_a_function) which do not exist because they have different answers depending on which direction you approach a discontinuity from.