I see now. I wasn’t able to get the graph working, but I got the code snippet working. Below is the final one with the right derivatives for anyone who finds this later. One thing I don’t get though is why
theta -= 0.01 * grad behaves different than theta = theta - 0.01 * grad I though the first one was just a shorthand version and exactly the same. Why did that need to change?
def maml_simulation() -> None:
    x_i = torch.tensor(3.0, requires_grad=True)
    x_j = torch.tensor(4.0, requires_grad=True)
    theta = torch.tensor(2.0, requires_grad=True)
    loss_func = lambda x: (x - 10) ** 2
    print(f"before training: theta: {theta}")

    prediction = theta * x_i
    loss_one = loss_func(prediction)

    grad = torch.autograd.grad(loss_one, theta, create_graph=False)[0]
    theta_two = theta - 0.01 * grad
    print(
        f"first prediction: {prediction}, loss: {loss_one}, grad: {grad}, theta after update: {theta_two}"
    )

    prediction = theta_two * x_j
    loss = loss_func(prediction)
    grad = torch.autograd.grad(loss, theta)[0]
    print(f"second prediction: {prediction}, loss: {loss}, second grad: {grad}")