A loss function is just a python function. You can define a loss function as simple as
def my_loss(pred, actual):
    return pred-actual

Forward step
Start sequentially from your input, which would be pytorch tensors (if they are of some other form, convert them to pytorch tensors). Now do all your operations that you want. Compute the loss, which is just a function.
Backward step
To compute the gradients use loss.backward() and you got the gradients for your variables. If you only want to update a specific tensor, set requires_grad=True for that tensor and requires_grad=False for all other tensors.
Now to update your parameter you can use your desired optimization algorithm.
def my_optimization(param):
    # To get the gradient
    grad = param.grad
    
    # SGD
    param = param - alpha * grad