Parameter comes from the nn package and is just a convenient tool to define parameters in the context of neural networks.
From the autograd point of view, they are Variable with requires_grad=True (hence they are leaf Variables).
The optim package works with any Variable that has requires_grad=True, so you can use it without using nn  (and thus without using Parameters), but it is less convenient for neural network usage.
But for example, you can use pytorch to get numerical value of any mathematical function (even though this is not it’s original purpose). The sample below for example show you how to implement a function that computes numerical gradients for any 1D function:
import torch
from torch import autograd
from torch.autograd import Variable

def square(x):
    return x ** 2
def cube(x):
    return x ** 3

# Implement derivative of fn at point y (fn has to be R -> R)
def derivative(fn, y):
    y.requires_grad = True
    out = fn(y)
    return autograd.grad(out, y)[0]


x = Variable(torch.Tensor([3]))

print("Evaluate square at 3")
print(square(x))
print("Evaluate derivative of square at 3 (should be 2*3)")
print(derivative(square, x))
print("Evaluate derivative of cube at 3 (should be 3*(3)**2")
print(derivative(cube, x))