Thank y’all for the discussion! I ended up doing the unsqueeze + squeeze route; needed to understand a problem that a colleague ran into (as I’m sure I’ll run into it myself at some point ):

  
      github.com
  
  
    EricCousineau-TRI/repro/blob/53bca048b58b2146e4823c829a3b3e196f1815b7/python/notebooks/torch_gradient.ipynb 12
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.6.0'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",


  This file has been truncated. show original

  
  
    
    
  
  


def scalars_to_tensor_good(*a):
    """
    Will preserve gradient!
    """
    # https://discuss.pytorch.org/t/how-to-concatenate-to-a-tensor-with-a-0-dimension/6478
    # https://discuss.pytorch.org/t/what-is-the-best-way-to-append-scalar-to-tensor/54445/3
    a = [ai.unsqueeze(0) for ai in a]
    return torch.cat(a).squeeze(0)