text: Title: how set_transform affects batch size?
  Body: ### Describe the bug
  
  I am trying to fine-tune w2v-bert for ASR task. Since my dataset is so big, I preferred to use the on-the-fly method with set_transform. So i change the preprocessing function to this:
  ```
  def prepare_dataset(batch):
      input_features = processor(batch["audio"], sampling_rate=16000).input_features[0]
      input_length = len(input_features)
      labels = processor.tokenizer(batch["text"], padding=False).input_ids
  
      batch = {
          "input_features": [input_features],
          "input_length": [input_length],
          "labels": [labels]
      }
      
      return batch
  
  
  train_ds.set_transform(prepare_dataset)
  val_ds.set_transform(prepare_dataset)
  ```
  
  After this, I also had to change the DataCollatorCTCWithPadding class like this:
  ```
  @dataclass
  class DataCollatorCTCWithPadding:
      processor: Wav2Vec2BertProcessor
      padding: Union[bool, str] = True
  
      def __call__(self, features: List[Dict[str, Union[List[int], torch.Tensor]]]) -> Dict[str, torch.Tensor]:
          # Separate input_features and labels
          input_features = [{"input_features": feature["input_features"][0]} for feature in features]
          labels = [feature["labels"][0] for feature in features]
  
          # Pad input features
          batch = self.processor.pad(
              input_features,
              padding=self.padding,
              return_tensors="pt",
          )
  
          # Pad and process labels
          label_features = self.processor.tokenizer.pad(
              {"input_ids": labels},
              padding=self.padding,
              return_tensors="pt",
          )
  
          labels = label_features["input_ids"]
          attention_mask = label_features["attention_mask"]
  
          # Replace padding with -100 to ignore these tokens during loss calculation
          labels = labels.masked_fill(attention_mask.ne(1), -100)
  
          batch["labels"] = labels
  
          return batch
  ```
          
  But now a strange thing is happening, no matter how much I increase the batch size, the amount of V-RAM GPU usage does not change, while the number of total steps in the progress-bar (logging) changes. Is this normal or have I made a mistake?
  
  ### Steps to reproduce the bug
  
  i can share my code if needed
  
  ### Expected behavior
  
  Equal to the batch size value, the set_transform function is applied to the dataset and given to the model as a batch.
  
  ### Environment info
  
  all updated versions