Thanks for the help. I did the replacement as you stated above in the following ways.
# way No.1
if not node.val is None:
            if self.cuda_flag:
                variable =Variable(torch.LongTensor([node.word_id]).cuda())
            else:
                variable = Variable(torch.LongTensor([node.word_id]))
            result = self.word2hidden(self.embedding(variable))
            node.calculate_result = result.data
            return node.calculate_result

# way No.2
if not node.val is None:
            if self.cuda_flag:
                node.calculate_result = self.word2hidden(self.embedding(
                        Variable(torch.LongTensor([node.word_id]).cuda()))).data
            else:
                node.calculate_result = self.word2hidden(self.embedding( 
                        Variable(torch.LongTensor([node.word_id])))).data
            return node.calculate_result
# way No.3
if not node.val is None:
            if self.cuda_flag:
                result = self.word2hidden(
                    self.embedding(Variable(torch.LongTensor([node.word_id]).cuda())))
            else:
                result = self.word2hidden(
                    self.embedding(Variable(torch.LongTensor([node.word_id]))))
            node.calculate_result = result.data
            return node.calculate_result

However, they all raised same TypeError, the trace back information is:
Traceback (most recent call last):
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 172, in <module>
    t.train()
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 111, in train
    train_loss = self.train_step(self.data.train)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 143, in train_step
    output = self.model(p_tree, h_tree)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 210, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/align_model.py", line 18, in forward
    p_tree.postorder_traverse(self.rnn)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 187, in postorder_traverse
    func(self)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 219, in __call__
    var = var[0]
TypeError: 'float' object has no attribute '__getitem__'

It seems that the torch.LongTensor([node.word_id]) should be replaced with torch.LongTensor([[node.word_id]]). However, after I fix it, a new RuntimeError is raised. The trace back is:
Traceback (most recent call last):
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 172, in <module>
    t.train()
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 111, in train
    train_loss = self.train_step(self.data.train)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/trainer.py", line 143, in train_step
    output = self.model(p_tree, h_tree)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 210, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/align_model.py", line 18, in forward
    p_tree.postorder_traverse(self.rnn)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 186, in postorder_traverse
    c.postorder_traverse(func)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree.py", line 187, in postorder_traverse
    func(self)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 210, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/shawnguo/PythonWS/KnowledgeEnhancedTE/tree_models.py", line 26, in forward
    self.embedding(Variable(torch.LongTensor([[node.word_id]]).cuda())))
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 210, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/linear.py", line 52, in forward
    return self._backend.Linear()(input, self.weight)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/_functions/linear.py", line 10, in forward
    output.addmm_(0, 1, input, weight.t())
RuntimeError: matrix and matrix expected at /home/soumith/local/builder/wheel/pytorch-src/torch/lib/THC/generic/THCTensorMathBlas.cu:235

I don’t know why the forward function raise up these errors and how to fix them up. And, as I need to optimize self.word2hidden = nn.Linear(self.word_dim, self.hidden_dim, False) and self.hidden2hidden = nn.Linear(2 * self.hidden_dim, self.hidden_dim) in class VanillaRecursiveNN, it seems that node.calculate_result needs to be saved for backprop. So, your solution may not address the problem that I mentioned above, usage of GPU memory growing up ceaselessly. Should I manually free the GPU memory? If yes, how?
Anyway, thanks again for your help. Looking forward for your reply.