In C/C++ like languages, closing zeromq socket explicitly is a must, which I understand. But in some higher level languages, such as php and python, which have garbage collection mechanism, do I need to close the sockets explicitly?
In php, there is no ZMQSocket::close() and in python, pyzmq's doc says socket.close() can be omitted since it will be closed automatically during garbage collection.
So my question is, do I need to manually close it or not?...
You don't. You might close or delete things explicitly in Python when:

Ordering becomes important, such as requiring the connection to be closed before you can proceed.
Your references to the objects will persist for a long time, and the resource will no longer be required after some time. This might happen if you're storing them in lists or as member variables. You should explicitly close the resource, or remove references to it when you are done.

Generally speaking it's pedantic and premature to even think about such things in Python.
Should I close zeromq socket explicitly in python?