Hi I am new to deep learning…
I am trying to convert some Tensorflow code into Pytorch for practice
can you please help me understand what will be pytorch equivalent for the following code block:
def tokenize(lang):

  lang_tokenizer = tf.keras.preprocessing.text.Tokenizer(filters='')
  lang_tokenizer.fit_on_texts(lang)

  tensor = lang_tokenizer.texts_to_sequences(lang)

  tensor = tf.keras.preprocessing.sequence.pad_sequences(tensor,padding='post',maxlen=20,dtype='int32')

  return tensor, lang_tokenizer

the code i am trying to convert is the following:

  

      github.com
  

  
    sunilbelde/neural-machine-translation-english-to-hindi-telugu/blob/main/NeuralMachineTranslation_eng_hindi.ipynb


    {
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "NeuralMachineTranslation_eng_hindi.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {



  This file has been truncated. show original