If you know the gradient doesn’t change, you could compute it once, store it in a variable, and reuse it elsewhere:
grad = energy(pos).backward()
vel -= stepsize * grad

If you wanted to memoize the calls to backward, one thing you could do is write a function that computes the gradient you want, something like:
prev_computations = {}  # maps pos -> gradient
def energy_backwards(pos):
    if pos in prev_computations:
        return prev_computations[pos]
    #  Otherwise, compute the gradient analytically
    ...
    return grad