Is Django admin difficult to customize?
Personally, if you want a site to look like the admin, why not pull the templates and styles and use them, build your own views for what you need.  Gobs of documentation and forum help is there for that.  I like the idea of customizing the admin, but honestly, I have been doing it for awhile on a project and time and time again I think to myself, if this was built in the standard MVC (or MTV) manner with free templates online, copied admin ones, or some professionally made ones, and built with the plethora of addons and my code, it would be much easier!!! And, when you decide that request/response isn't cutting it, and you want to add lots of JavaScript, you'll be glad.  I know.  I have had to put all sorts of JavaScript on our Admin project.  Mostly because it's impossible to make the admin do what we want, so we fix it with JavaScript once it is on screen.   When you find yourself writing an Ajax based system, you'll wonder why you have the admin at all.  
If I could start this project over, I might not even use Django, but I probably would.  I most certainly won't used the Admin.  
Now, if you are building an basic CRUD type site that doesn't have style-eyed users, then Django with grappelli, and some elbow grease will get the job done.  Remember, Django is a collection of Python scripts.  Just override everything you can, and you'll get there, if you aren't too ambitious.
I have been playing for a couple of days with Django Admin to explore it, but I am still clueless of how it can be customized in the way we need. 
Every time I look for any help for the customization in the admin panel, what I find is, a bunch of articles on various communities and forums, explaining how to customize the template, the lists items, the the column views etc. But that should not be called Django Customization. 
If I need to make even a small change in the User functionality or any modification to the auth module. It takes a lots of efforts even in figuring out how that can be done.
Is Django that difficult to customize or its just lack of the help available over internet for this or its me who is moving in the wrong direction ?