Hi. I want to implement character-level embedding.
This is usual word embedding.

Word Embedding

    Input: [ [‘who’, ‘is’, ‘this’] ] 
    -> [ [3, 8, 2] ]     # (batch_size, sentence_len)
    -> // Embedding(Input)
     # (batch_size, seq_len, embedding_dim)

This is what i want to do.
2. Character Embedding
Input: [ [ [‘w’, ‘h’, ‘o’, 0], [‘i’, ‘s’, 0, 0], [‘t’, ‘h’, ‘i’, ‘s’] ] ]
-> [ [ [2, 3, 9, 0], [ 11, 4, 0, 0], [21, 10, 8, 9] ] ]      # (batch_size, sentence_len, word_len)
-> // Embedding(Input) # (batch_size, sentence_len, word_len, embedding_dim)
-> // sum each character embeddings  # (batch_size, sentence_len, embedding_dim)
The final output shape is same as Word embedding. Because I want to concat them later.

Although I tried, I am not sure how to implement 3-D embedding. Do you know how to implement such a data?
    def forward(self, x):
        print('x', x.size()) # (N, seq_len, word_len)
        bs = x.size(0)
        seq_len = x.size(1)
        word_len = x.size(2)
        embd_list = []
        for i, elm in enumerate(x):
            tmp = torch.zeros(1, word_len, self.embd_size)
            for chars in elm:
                tmp = torch.add(tmp, 1.0, self.embedding(chars.unsqueeze(0)))


Above code got an error because output of self.embedding is Variable.
TypeError: torch.add received an invalid combination of arguments - got (torch.FloatTensor, float, Variable), but expected one of:
 * (torch.FloatTensor source, float value)
 * (torch.FloatTensor source, torch.FloatTensor other)
 * (torch.FloatTensor source, torch.SparseFloatTensor other)
 * (torch.FloatTensor source, float value, torch.FloatTensor other)
      didn't match because some of the arguments have invalid types: (torch.FloatTensor, float, Variable)
 * (torch.FloatTensor source, float value, torch.SparseFloatTensor other)
      didn't match because some of the arguments have invalid types: (torch.FloatTensor, float, Variable)

Update
I can do this with for. But it may be slow.
    def forward(self, x):
        print('x', x.size()) # (N, seq_len, word_len)
        bs = x.size(0)
        seq_len = x.size(1)
        word_len = x.size(2)
        embd = Variable(torch.zeros(bs, seq_len, self.embd_size))
        for i, elm in enumerate(x): # every sample
            for j, chars in enumerate(elm): # every sentence. [ [‘w’, ‘h’, ‘o’, 0], [‘i’, ‘s’, 0, 0], [‘t’, ‘h’, ‘i’, ‘s’] ]
                chars_embd = self.embedding(chars.unsqueeze(0)) # (N, word_len, embd_size) [‘w’,‘h’,‘o’,0]
                chars_embd = torch.sum(chars_embd, 1) # (N, embd_size). sum each char's embedding
                embd[i,j] = chars_embd[0] # set char_embd as word-like embedding

        x = embd # (N, seq_len, embd_dim)