How to change default django User model to fit my needs?
You face a bit of a dilemma which really has two solutions if you're committed to avoiding the profile-based customization already pointed out.

Change the User model itself, per Daniel's suggestions
Write a CustomUser class, subclassing User or copying its functionality.

The latter suggestion means that you would have to implement some things that User does automatically manually, but I wonder whether that's as bad as it sounds, especially if you're at the beginning of your project.  All you'd have to do is rewrite a middle-ware class and some decorators.
Of course, I don't think this buys you anything that 1 won't get you, except that your project shouldn't break if you svn update your django.  It may avoid some of the compatibility problems with other apps, but my guess is most problems will exist either way.
The default Django's User model has some fields, and validation rules, that I don't really need. I want to make registration as simple as possible, i.e. require either email or username, or phone number - all those being unique, hence good as user identifiers. 
I also don't like default character set for user name that is validated in Django user model. I'd like to allow any character there - why not?
I used user-profile django application before to add a profile to user - but this time I'd rather make the class mimimal. But I still want to use the User class, as it gives me an easy way to have parts of site restricted only for users logged in. 
How do I do it?