Hello,
I am pretty new to pytorch and I am trying to implement the trainable layer proposed in this paper : https://arxiv.org/pdf/1607.05666.pdf 74
Here is my code:
import torch.nn as nn
import torch
import numpy as np
from torch.autograd import Variable

class PCEN(nn.Module):
    def __init__(self,in_features,bias=False):
        super(PCEN,self).__init__()

    self.alpha = nn.Parameter(torch.Tensor(in_features,))
    self.delta = nn.Parameter(torch.Tensor(in_features,))
    self.r = nn.Parameter(torch.Tensor(in_features,))
    self.eps = torch.Tensor([0.00001])

def forward(self,x,smoother):
    alpha = self.alpha.expand_as(x)
    delta = self.delta.expand_as(x)
    r = self.r.expand_as(x)

    pcen = (x/(self.eps + smoother)**alpha + delta)**r - delta**r
    return pcen

#40 dimensional filterbank energies
pcen = PCEN(40)

#dummy data
feats = np.random.standard_normal(size=(10,40)).astype('float32')
smoother = np.random.standard_normal(size=(10,40)).astype('float32')

feats = Variable(torch.from_numpy(feats))
smoother = Variable(torch.from_numpy(smoother))

pcen_feats = pcen(feats,smoother)

Q.  the eps parameter is to avoid division by zero and I don’t need to use expand_as with it?
Q. The forward pass seems to be working, I was wondering if there are any obvious errors? Do I need to use register buffers for the parameters ?
Q. In the paper they say to insure parameter positivity, they do gradient updates on the log values of the parameters and then take exponentials. How can I go about doing this?
I have a couple more questions, but I will save them for now.
Thanks,
Gautam