Why should I focus on that which is not mine? I was sitting among my friends when I realised none of them are mine. They will change or perish. I felt detached and disconnected. I no longer focused on what they were saying. I appeared absent minded to them. I appeared as one who doesn't understand or care or love.
My question is : Why should I focus on people or serve people who are not mine ?
There is a Buddhist concept of Kalyāṇa-mittatā, Buddha definitely approved friendship amongst the sangha as well as for householders. As long as it was with right people (meaning friends addicted to gambling, alcohol should be avoided).  

Now to your dilemma,


  when I realised none of them are mine


The better thing to realize was to say, 'I am not' 'anatta'. As long as you have not realized no-self, and as long as you are you can say 'none of them are mine'. Of course, you do not possess them, so in that sense, they are not yours, but they are YOUR friends. As a householder, friendship is imperative, you cannot live in this samsara like an island, you never know when will you need whom. Although this should not be the criteria to make friends, but this is true.


  They will change or perish. I felt detached and disconnected. I no longer focused on what they were saying. 


They will change and perish so will you, so will everybody you know, your wife, your children, your parents, everything is impermanent, 'annicca'. The feeling of detachment is alright, in fact good, but dis-connection shows you lack respect and compassion. Everyone wants to tell their story to friends, share the problems and good and bad times with friends, you should have compassion. You should have listened to them without attachment. Like a lotus in a pond, be in this world but not of it. 


  I appeared absent-minded to them.


Are you not practicing mindfulness? Absent-mindedness is not something for a meditator. You should be aware. Later on, you will regret if only I would have been listening to him...


  I appeared as one who doesn't understand or care or love.


Cultivate kindness and compassion. Metta bhava is important. 


  Why should I focus on people or serve people who are not mine?


Because Samsara is Nirvana (this is something I learned in Zen). Also not focusing on people who are not of profit to you or not yours shows selfishness and will bolster your ego. Remember the Jataka tales and remember all the sacrifice the Buddha did in His previous life for other beings, out of compassion.