django-oauth2-provider with custom user model?
As the previous answer suggested, you should extend AbstractUser from django.contrib.auth.models.
The problem with the access token that the OP referring to, occur when changing the setting AUTH_USER_MODEL AFTER django-oauth2-provider was migrated.
When django-oauth2-provider is migrated, it creates a key constrain between the User model and django-oauth2-provider.
The solution is very easy:

Create your new User model and change the AUTH_USER_MODEL setting.
Go to the django_migration table in your database.
Delete all rows of django-oauth2-provider.
run python manage.py makemigrations
run python manage.py migrate

Now, the django-oauth2-provider tables are connected to the RIGHT User model.
I am really stuck in my project right now. I am trying to implement Oauth2 for my app. I found out about django-oauth2-provider a lot and tried it. The only problem is, it uses the User model at django.contrib.auth. The main users of our site are saved in a custom model called User which does not inherit from or extend the model at django.contrib.auth.
Is there any way to use my custom User model for creating clients and token? 
If django-oauth2-provider can't be used for this purpose, can anyone recommend me some oauth2 library with the option to implement oauth2 with my own model.
Sincerely,
Sushant Karki