readme: ---
  library_name: sklearn
  tags:
  - sklearn
  - skops
  - text-classification
  ---
  
  # Model description
  
  This is a neural net classifier and distilbert model chained with sklearn Pipeline trained on 20 news groups dataset.
  
  ## Intended uses & limitations
  
  This model is trained for a tutorial and is not ready to be used in production.
  
  ## Training Procedure
  
  ### Hyperparameters
  
  The model is trained with below hyperparameters.
  
  <details>
  <summary> Click to expand </summary>
  
  | Hyperparameter                                 | Value                                                                                                                                   |
  |------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|
  | memory                                         |                                                                                                                                         |
  | steps                                          | [('tokenizer', HuggingfacePretrainedTokenizer(tokenizer='distilbert-base-uncased')), ('net', <class 'skorch.classifier.NeuralNetClassifier'>[initialized](
    module_=BertModule(
      (bert): DistilBertForSequenceClassification(
        (distilbert): DistilBertModel(
          (embeddings): Embeddings(
            (word_embeddings): Embedding(30522, 768, padding_idx=0)
            (position_embeddings): Embedding(512, 768)
            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
          (transformer): Transformer(
            (layer): ModuleList(
              (0): TransformerBlock(
                (attention): MultiHeadSelfAttention(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (q_lin): Linear(in_features=768, out_features=768, bias=True)
                  (k_lin): Linear(in_features=768, out_features=768, bias=True)
                  (v_lin): Linear(in_features=768, out_features=768, bias=True)
                  (out_lin): Linear(in_features=768, out_features=768, bias=True)
                )
                (sa_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (ffn): FFN(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (lin1): Linear(in_features=768, out_features=3072, bias=True)
                  (lin2): Linear(in_features=3072, out_features=768, bias=True)
                  (activation): GELUActivation()
                )
                (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              )
              (1): TransformerBlock(
                (attention): MultiHeadSelfAttention(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (q_lin): Linear(in_features=768, out_features=768, bias=True)
                  (k_lin): Linear(in_features=768, out_features=768, bias=True)
                  (v_lin): Linear(in_features=768, out_features=768, bias=True)
                  (out_lin): Linear(in_features=768, out_features=768, bias=True)
                )
                (sa_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (ffn): FFN(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (lin1): Linear(in_features=768, out_features=3072, bias=True)
                  (lin2): Linear(in_features=3072, out_features=768, bias=True)
                  (activation): GELUActivation()
                )
                (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              )
              (2): TransformerBlock(
                (attention): MultiHeadSelfAttention(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (q_lin): Linear(in_features=768, out_features=768, bias=True)
                  (k_lin): Linear(in_features=768, out_features=768, bias=True)
                  (v_lin): Linear(in_features=768, out_features=768, bias=True)
                  (out_lin): Linear(in_features=768, out_features=768, bias=True)
                )
                (sa_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (ffn): FFN(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (lin1): Linear(in_features=768, out_features=3072, bias=True)
                  (lin2): Linear(in_features=3072, out_features=768, bias=True)
                  (activation): GELUActivation()
                )
                (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              )
              (3): TransformerBlock(
                (attention): MultiHeadSelfAttention(
                  (dropout): Dropout(p=0.1, inplace=False)
                  (q_lin): Linear(in_features=768, out_features=768, bias=True)
                  (k_lin): Linear(in_features=768, out_features=768, bias=True)
                  (v_lin): Linear(in_features=768, out_features=768, bias=True)
                  (out_li