I tried look into this similar issue: Custom GPT2 Model won't load after training
but still…
class CustomClass(PreTrainedModel):
    def __init__(self, config, num_labels):
        super().__init__(config, num_labels)
        self.distilbert = DistilBertModel.from_pretrained('distilbert-base-uncased')
        self.pre_classifier = torch.nn.Linear(768, 768)
        self.dropout = torch.nn.Dropout(0.1)
        self.classifier = torch.nn.Linear(768, num_labels)

    def forward(self, input_ids, attention_mask):
        distilbert_output = self.distilbert(input_ids=input_ids, attention_mask=attention_mask)
        hidden_state = distilbert_output[0]
        pooled_output = hidden_state[:, 0]
        pooled_output = self.pre_classifier(pooled_output)
        pooled_output = torch.nn.Tanh()(pooled_output) 
        pooled_output = self.dropout(pooled_output)  
        output = self.classifier(pooled_output)  
        return output

I was able to fine tune with a linear classifier for a classification job.
config = PretrainedConfig(name_or_path='own-model', num_labels=100, output_hidden_states=True)

model = CustomClass(config, 100)

I can also save it with
model.save_pretrained(PATH)

But when I try to load it with
new_model=PreTrainedModel.from_pretrained('./PATH/')


i got 'NoneType' object has no attribute 'from_pretrained' which is really strange
Alternatively, with a config file, this doesnt throw error though the model weights wont load
new_config=PretrainedConfig.from_pretrained('./PATH/') new_model=PreTrainedModel.from_pretrained('./PATH/', config=new_config)

please help. I am running out of idea. Oh and I am at v4.6.0
Thank you!