The way in which TensorFlow implements a certain type of layer may be drastically different from the way in which PyTorch implements the same layer. Permuting the indices till they look like the correct shape, may not be enough.
If you want to do this import on your own, you may have to drill deep into the innards of both libraries to see how each type of layer that you are interested in, translates to tensors in each library.