I guess my example code above does not actually work in a more up to date version of pytorch. It throws an error instead…
Which admittedly weakens my argument that its clear there is a gradient between a[0] and output; but does not seem to satisfy me. If I wanted to know the cost/gradient if a was shifted in a single dimension I would want the above code to work.