albanD:

torch.stack(out, dim=0)


Thank you for your help @albanD. Your solution allows to backpropagate through the tensor values and makes sense.
Here is the modified working code in case is useful for someone else in the future:
import torch
from torch import nn

class Dyn(nn.Module):
    def __init__(self):
        super(Dyn, self).__init__()
        
    def forward(self, z0, params):
        out = [z0]
        for i in range(9):
            out.append(out[i] + 0.5*torch.mv(params.view(2,2), out[i]))
        return torch.stack(out, dim=0)
    
z1 = torch.Tensor([-1., -1.])
z0 = torch.tensor([0.6, 0.3], requires_grad=True)

params = torch.randn(4, requires_grad=True)/2.
dyn = Dyn()

sol = torch.tensor([[-0.3600,  0.5700],
                    [-0.6270,  0.3615],
                    [-0.7764,  0.0299],
                    [-0.7525, -0.3598],
                    [-0.5350, -0.7181],
                    [-0.1493, -0.9497],
                    [ 0.3330, -0.9768],
                    [ 0.8048, -0.7614],
                    [ 1.1453, -0.3210],
                    [ 1.2485,  0.2677]])

pred = dyn(z0, params)

torch.autograd.grad(pred-sol, params, grad_outputs=torch.ones(10,2))