You can you look at your situation as finding derivative of f(g(h(w))), and the values you get are h’ = 1, g’=1/2 and f’=0 . Since this is composition, f’!=h’+g’
To see why specifically you get zero you can decorate your graph to print backward values
import torch
import torch.nn as nn
import torch.nn.functional as F

feat_dim, out_dim = 1, 2
encoder = nn.Linear(feat_dim, out_dim, bias=False)
nn.init.zeros_(encoder.weight.data)
batch_size = 1
input = torch.eye(feat_dim)[:batch_size]

def decorate(tensor, name):
    def print_backward(grad):
        print(f"({name}): ", grad)
    tensor.register_hook(print_backward)
    
logits = encoder.forward(input)
norm = torch.logsumexp(logits, dim=1, keepdim=True)
decorate(logits, 'logits')
decorate(logits, 'norm')
norm.expand_as(logits).backward(torch.ones_like(logits))

This prints following
(logits):  tensor([[1., 1.]])
(norm):  tensor([[1., 1.]])

Since both logits and norm vectors are 1’s, their difference will cancel out. The following is an illustration of how 1’s come about

Screenshot 2019-09-15 16.12.41.png1416×832 18.9 KB