Who is responsible for avoiding collisions on the ground; the pilot or the air traffic controller? I may have some false assumptions here, so correct if needed. It is said that while ATC sounds like they are "in charge" of the airspace surrounding a major airport, control and responsibility for any given airliner is (still) in the hands of the Pilot-In-Command (PIC).

What this means is that if an airliner is given instructions to do something by ATC, and they are unable (or unwilling) to follow those instructions, they are able to tell ATC they are unable and ATC has to work it out, with all of the ramifications that implies. If the ATC controller wishes, they require the crew to contact ATC by other means (the dreaded phone call, I suppose) to explain, but the fact of the matter is, the passengers are ultimatley depending on the crew to keep them safe.

Now, let's take an airliner, at the gate, readying itself for a flight. They receive a pushback and are ready to taxi. They contact ATC Ground for instructions. Ground gives them taxi instructions. Who is responsible for the safety of the aircraft at that point?

Why do I ask this, when it should be obvious, the PIC. However, visibility out of the aircraft is limited, especially in some of the larger craft. As far as I know, there is no "check the mirrors" or "objects in the rear view window may be larger than they appear." The PIC simply CANNOT see behind the aircraft, and so any ground vehicles, personnel, or other aircraft (small or large) could cause an accident, which would be out of the control of the PIC.
At all times, whether in the air or on the ground, whether at the gate or taxiing, the pilot-in-command (PIC) has final and ultimate responsibility for the safe operation of the aircraft. Per ICAO Annex 2 (with similar statements/definitions by individual civil authorities):


  The pilot-in-command of an aircraft shall, whether manipulating the controls or not, be responsible for the operation of the aircraft in accordance with the rules of the air, except that the pilot-in-command may depart from these rules in circumstances that render such departure absolutely necessary in the interests of safety.


As you stated, the pilot is allowed to deviate from any ATC instruction or regulation to the extent necessary to ensure the safety of the crew, passengers and property. If necessary, the PIC may enlist additional resources, such as a ground crew, to deal with issues like not being able to see behind the airplane. For airline operations, this is usually spelled out by the airline's operations procedures, which the PIC effectively "approves" (as the ultimate authority for the aircraft's operation) and carries out. 

Air traffic controllers have what you might call "jurisdiction" over certain parcels of airspace and airports. The gates and the area immediately surrounding them are usually not under the jurisdiction of the control tower. They are instead managed by the airport authority. At large airports, the airport may operate a "ramp control" system with its own (smaller) control tower and radio frequencies. Arriving and departing aircraft coordinate with ramp control to get into and out of the gates safely. Ground crews will assist the pilot-in-command with parking and pushback operations, getting the aircraft to a point where the PIC can taxi the aircraft to leave the ramp area and enter the taxiway system in accordance with ATC taxi instructions.

Once on the taxiways, and from there to the runway and into the air, the aircraft is under the jurisdiction of the regular air traffic controllers. The controllers can divide up this area between different positions, usually a "ground controller," a "tower" or "local controller," and one or more approach/departure radar controllers.