I am not calling func_that_uses_l1 directly. It is being called by forward method. Specifically, the forward function calls somefunc which calls func_that_uses_l1. If I remove the nesting, the problem vanishes.
I am 95% certain this is a bug.