I think you could add some code for timing in your forward function. For example,
def forward(self, x):
    start = time.time()
    out = self.aspp(x)
    print(time.time() - start)

I am new to pytorch, so not sure if there is other better way to do it.