May 14, 2020 (New York)
      
In this tutorial 4, the following code is used for forward propagation:
# zero the parameter gradients
optimizer.zero_grad()

# forward
# track history if only in train
with torch.set_grad_enabled(phase == 'train'):
    outputs = model(inputs)
    _, preds = torch.max(outputs, 1)
    loss = criterion(outputs, labels)

    # backward + optimize only if in training phase
    if phase == 'train':
        loss.backward()
        optimizer.step()

# statistics
running_loss += loss.item() * inputs.size(0)
running_corrects += torch.sum(preds == labels.data)

However, could they have not done this instead?
# zero the parameter gradients
optimizer.zero_grad()

inputs.requires_grad = True
outputs = model(inputs)
_, preds = torch.max(outputs, 1)
loss = criterion(outputs, labels)

# backward + optimize only if in training phase
if phase == 'train':
    loss.backward()
    optimizer.step()

# statistics
running_loss += loss.item() * inputs.size(0)
running_corrects += torch.sum(preds == labels.data)

Since inputs is a leaf node and if a leaf node has requires_grad=True
then all subsequent tensors will also have requires_grad=True ?