hi, @goerlitz Thank you for the reply.
this simpletransformer.ai blog helps me to understand what are class weights and how they will impact the calculation of loss function while having unbalanced samples.
all now I need is programmatic implementation.
I am using (Hugging Face DistilBert & Tensorflow for Custom Text Classification. | by Galina Blokh | Geek Culture | Medium) this blog to create a text classification model.
as well I am using Trainer API to train the model.
sample code:
class Dataset(torch.utils.data.Dataset):
def init(self, encodings, labels=None):
self.encodings = encodings
self.labels = labels
def getitem(self, idx):
item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
if self.labels:
item[“labels”] = torch.tensor(self.labels[idx])
return item
def len(self):
return len(self.encodings[“input_ids”])
train_dataset = Dataset(X_train_tokenized, y_train)
val_dataset = Dataset(X_val_tokenized, y_val)
args = TrainingArguments(
evaluation_strategy=“steps”,
eval_steps=500,
per_device_train_batch_size=8,
per_device_eval_batch_size=8,
num_train_epochs=1,
seed=0,
load_best_model_at_end=True,
)
trainer = Trainer(
model=model,
args=args,
train_dataset=train_dataset,
eval_dataset=val_dataset,
compute_metrics=compute_metrics,)
Can you please help me where to use custom trainer function in my code