<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">max_norm in embedding layer modifies the embedding layer weight in-place which makes it a non-leaf variable. This leads to issues when doing gradient backpropogation.<br>
<strong>Error: RuntimeError: leaf variable has been moved into the graph interior</strong></p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Script to reproduce the behavior:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
import torch
import torch.nn as nn
import numpy as np


class sampleModel(nn.Module):
    def __init__(self):
        super(sampleModel, self).__init__()
        input_layer= nn.Embedding(500, 10, max_norm=1.0)
        output_layer = nn.Linear(10, 2)
        self.layers = nn.Sequential(input_layer, output_layer)
        self.loss_criterion = nn.NLLLoss()

    def forward(self, input, labels):
        em_x = self.layers[0](input).sum(1)
        out = self.layers[1](em_x)
        loss = self.loss_criterion(out, labels)
        return loss


if __name__ == &quot;__main__&quot;:
    device = torch.device('cpu')
    
    model = sampleModel().to(device)

    data = []
    for i in range(0,500):
        data.append(np.random.randint(0, 20, size=30))
    
    inputx = torch.LongTensor(data).to(device)
    labels = torch.LongTensor( [0]*250 + [1]*250 ).to(device)
    
    loss = model(inputx, labels)
    print(loss)
    loss.backward()"><pre class="notranslate"><code class="notranslate">
import torch
import torch.nn as nn
import numpy as np


class sampleModel(nn.Module):
    def __init__(self):
        super(sampleModel, self).__init__()
        input_layer= nn.Embedding(500, 10, max_norm=1.0)
        output_layer = nn.Linear(10, 2)
        self.layers = nn.Sequential(input_layer, output_layer)
        self.loss_criterion = nn.NLLLoss()

    def forward(self, input, labels):
        em_x = self.layers[0](input).sum(1)
        out = self.layers[1](em_x)
        loss = self.loss_criterion(out, labels)
        return loss


if __name__ == "__main__":
    device = torch.device('cpu')
    
    model = sampleModel().to(device)

    data = []
    for i in range(0,500):
        data.append(np.random.randint(0, 20, size=30))
    
    inputx = torch.LongTensor(data).to(device)
    labels = torch.LongTensor( [0]*250 + [1]*250 ).to(device)
    
    loss = model(inputx, labels)
    print(loss)
    loss.backward()
</code></pre></div>
<p dir="auto">Stack trace:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="File &quot;bug_report_script.py&quot;, line 34, in &lt;module&gt;
    loss.backward()
File &quot;/data/anaconda/envs/bletchley/lib/python3.6/site-packages/torch/tensor.py&quot;, line 118, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
File &quot;/data/anaconda/envs/bletchley/lib/python3.6/site-packages/torch/autograd/__init__.py&quot;, line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: leaf variable has been moved into the graph interior"><pre class="notranslate"><code class="notranslate">File "bug_report_script.py", line 34, in &lt;module&gt;
    loss.backward()
File "/data/anaconda/envs/bletchley/lib/python3.6/site-packages/torch/tensor.py", line 118, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/data/anaconda/envs/bletchley/lib/python3.6/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: leaf variable has been moved into the graph interior
</code></pre></div>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">calculating max_norm should not make the embedding layer weight non leaf variable.</p>

<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch version: 1.2.0<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.0.130</p>
<p dir="auto">OS: Ubuntu 16.04.6 LTS<br>
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609<br>
CMake version: version 3.5.1</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.0.130<br>
GPU models and configuration:<br>
GPU 0: Tesla M60<br>
GPU 1: Tesla M60</p>
<p dir="auto">Nvidia driver version: 410.79<br>
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.4.2</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip] numpy==1.17.2<br>
[pip] torch==1.2.0<br>
[pip] torchvision==0.4.0<br>
[conda] torch                     1.2.0                    pypi_0    pypi<br>
[conda] torchvision               0.4.0                    pypi_0    pypi</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">Surprisingly, if the device is set to 'cuda', the script runs without error</p>

<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a></p>
<p dir="auto">Hi all,</p>
<p dir="auto">So there're some great things about PyTorch, but one of the not-great things is that it uses a mostly-but-not-fully different API than the one used for numpy, theano, and tensorflow.  I find myself having to consult a <a href="https://github.com/torch/torch7/wiki/Torch-for-Numpy-users">lookup-table</a> when I want to run familiar commands.</p>
<p dir="auto">Are there any plans to make a numpytorch, where the API matches numpy's as closely as possible?</p>
<p dir="auto">An advantage would be that many functions written for numpy could also be used by pytorch.  It would also be nice to be able to call matplotlib functions (plot, imshow, etc) directly with torch variables without having to do any hacking of matplotlib.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mruberry/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mruberry">@mruberry</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/rgommers/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/rgommers">@rgommers</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/heitorschueroff/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/heitorschueroff">@heitorschueroff</a></p>