Are you sure x and y are a one-to-one match?
Is learning rate the same as you do in keras?
layer3 = F.relu(self.linear1(layer3)) are you sure to use self.linear1 twice?
you can also add print in forward to make sure the number does not overflow
    def forward(self, bow_vec):
        layer1 = self.embed(bow_vec)
        layer2 = layer1.sum(1).squeeze(1)
        print layer2.data #.....
        layer3 = F.relu(self.linear1(layer2))
        layer3 = F.relu(self.linear1(layer3))
        layer4 = self.linear2(layer3)
        return layer4