How do I let go the ownership of the mind? If someone offense me on physical things like "You are not beautiful, You are poor, You are not smart", I can bear it. Actually I don't care much.

But, If someone offense me on my thoughts like "You are not kind, You are greedy", It's hard to bear. Sometimes, I got aversion.

How can I overcome this ?

I asked a related question here. 


How Am I not the owner of the mind?
This is due to subtle Upadana. (clinging)
There are three kinds of clinging.
-Kama Upadana (it seems you have less of this)
-Dithi upadana
-Atta upadana. (perhaps this is your problem)
-Silabbatha Upadana

upādāna:
Clinging; attachment; sustenance for becoming and birth — attachment to sensuality, to views, to precepts and practices, and to theories of the self.

The way to eliminate Upadana is to follow the Noble Eightfold Path. It is a gradual training of Sila (virtues), Samadhi (concentration) and Panna (wisdom).