Can you upload the code and the data to git?
Meanwhile take a look at my code, I do something similiar with a CNN:

  
      github.com
  
  
    QuantScientist/Deep-Learning-Boot-Camp/blob/master/day02-PyTORCH-and-PyCUDA/PyTorch/55-PyTorch-using-CONV1D-on-one-dimensional-data-CNN.ipynb 20
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deep Learning Bootcamp November 2017, GPU Computing for Data Scientists\n",
    "\n",
    "<img src=\"../images/bcamp.png\" align=\"center\">\n",
    "\n",
    "### 55 PyTorch Convolutional Nerual Network (CNN) classifier for Numer.Ai Binary Classification problem using CONV1D (one dimentional Convolutional Nerual Network). \n",
    "\n",
    "Web: https://www.meetup.com/Tel-Aviv-Deep-Learning-Bootcamp/events/241762893/\n",
    "\n",
    "Notebooks: <a href=\"https://github.com/QuantScientist/Data-Science-PyCUDA-GPU\"> On GitHub</a>\n",
    "\n",


  This file has been truncated. show original

  
  
    
    
  
  


The most important part is in forward, where I use x = x.view(varSize,int(self.n_hidden)) for converting the Tensor before the Linear layer is applied.