Question: Most importantly, what must change in order for the falling object to change its speed? Is it the distance to the centre of the planet? If you pull the earth away from the object as the object falls, will the object slow down or will it keep accelerating?

Answer: 
I think this may be also an possibility for explaining your answer.. This is also an attempt though the real answer has already been explained. Whenever you throw an object up, it always tries to fall down because it is attracted by gravitational force. Now suppose you throw an object to height *h* then, it will gain *P.E* as *mgh* and *K.E* 0.But since it is attracted by gravitational force it will fall down.. Let it reach at height *x* from the ground surface. Now its new \*PE \* is *mgx* and *KE* is somehow more than previous.. ( from the conservation of energy).. Since it has gain some new *KE* that is more than previous one , it will directly mean that it has gain some velocity.. As there is difference in velocity, there will be of course an acceleration produced. So, in any case if any object is falling toward mass.. then object will always accelerate , provided that Gravitational force is the only force acting.
> 
> Most importantly, what must change in order for the falling object to change its speed?
> 
> 
> 

Obviously, there's nothing necessary for that. Ignoring ***drag***, the velocity of the object is increasing for sure. Only the acceleration remains constant. Taking *air effects* into account, the object attains the terminal velocity (based on fluid), its $v$ remains constant & $a$ becomes zero. If you're concerned about changing its velocity now, then you either apply another force or *remove* the fluid...

> 
> Is it the distance to the center of Earth?
> 
> 
> 

Well, quite... Once the distance to the center increases, the gravitational pull exerted on the object decreases - leading to less acceleration. Hence, velocity changes *indeed*...

> 
> If you pull the earth away...
> 
> 
> 

The acceleration on the object decreases slowly. If another body attracts it with some comparable force, your object stops accelerating towards it. But, never quite reaches stationary state...

---

Whenever a force is exerted on an object, there's a change in its velocity as a function of time. It's old-school. Check out [Newton's laws](http://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Newton.27s_second_law) and finally substitute gravity and you're done...