Question: Most importantly, what must change in order for the falling object to change its speed? Is it the distance to the centre of the planet? If you pull the earth away from the object as the object falls, will the object slow down or will it keep accelerating?

Answer: 
As long as there's a non-zero net force acting on the object, it will have a non-zero acceleration and therefore it will continuously change its velocity: $$\vec{F} = m\vec{a}.$$

In the case of gravity, the force is inversely proportional to the distance between the objects squared: $$\vec{F}\_G = G\frac{m\_1m\_2}{r^2}\frac{\vec{r}}{r},$$ where $\vec{r}$ is the vector connecting the two objects and $r=|\vec{r}|$ its length. So the closer the objects are, the stronger the force or -equivalently- the acceleration. Notice that the acceleration is only zero if the objects are infinitely far apart. (I'm assuming no drag, let's only consider the gravitational force for simplicity)

If you pull the earth in the same direction the object is falling so that you maintain the same distance $r$ at all times the object will just keep falling with a constant acceleration. If you pull it faster, the distance will increase and the acceleration will therefore decrease, meaning the velocity of the object will increase more slowly than before. But it will never decrease. Pulling the earth more slowly will only decrease how much the acceleration would increase if you hadn't pulled, so again the velocity keeps increasing.

So to summarize, the object's velocity will always increase, unless you can get the distance to infinity, which should only take you about - an infinite amount of time. And even then you can only get to a constant velocity, never a decreasing one. You need a repulsive force for that (or additional attractive forces on the other side of the object).
I think this may be also an possibility for explaining your answer.. This is also an attempt though the real answer has already been explained. Whenever you throw an object up, it always tries to fall down because it is attracted by gravitational force. Now suppose you throw an object to height *h* then, it will gain *P.E* as *mgh* and *K.E* 0.But since it is attracted by gravitational force it will fall down.. Let it reach at height *x* from the ground surface. Now its new \*PE \* is *mgx* and *KE* is somehow more than previous.. ( from the conservation of energy).. Since it has gain some new *KE* that is more than previous one , it will directly mean that it has gain some velocity.. As there is difference in velocity, there will be of course an acceleration produced. So, in any case if any object is falling toward mass.. then object will always accelerate , provided that Gravitational force is the only force acting.