I am doing something very similar but I have a (nested) list of tensors. The simplest example I have is the following:
import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3, 1)
xs = [x, x]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)

but I get the following error:
x = torch.randn(3, 1)
xs = [x, x]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)
Traceback (most recent call last):
  File "/Users/brando/anaconda3/envs/automl-meta-learning/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-30-f846b7bfae21>", line 4, in <module>
    xs = torch.as_tensor(xs)
ValueError: only one element tensors can be converted to Python scalars

any ideas what is going on? Btw, both give the same error.

I guess the following works but I am unsure what is wrong with this solution:
# %%

import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3)
xs = [x.numpy(), x.numpy()]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)

print(xs)
print(xs.size())

# %%

import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3)
xs = [x.numpy(), x.numpy(), x.numpy()]
xs = [xs, xs]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)

print(xs)
print(xs.size())

output:
import torch
# trying to convert a list of tensors to a torch.tensor
x = torch.randn(3)
xs = [x.numpy(), x.numpy(), x.numpy()]
xs = [xs, xs]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)
print(xs)
print(xs.size())
tensor([[[0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863]],
        [[0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863]]])
torch.Size([2, 3, 3])