In layered LiNixMnyCozO2 cathode material for lithium-ion batteries, the spins of transition metal (TM) ions construct a two-dimensional triangular networks, which can be considered as a simple case of geometrical frustration. By performing neutron powder diffraction experiments and magnetization measurements, we find that long-range magnetic order cannot be established in LiNixMnyCozO2 even at low temperature of 3 K. Remarkably, the frustration parameters of these compounds are estimated to be larger than 30, indicating the existence of strongly frustrated magnetic interactions between spins of TM ions. As frustration will inevitably give rise to lattice instability, the formation of Li/Ni exchange in LiNixMnyCozO2 will help to partially relieve the degeneracy of the frustrated magnetic lattice by forming a stable antiferromagnetic state in hexagonal sublattice with nonmagnetic ions located in centers of the hexagons. Moreover, Li/Ni exchange will introduce 180° superexchange interaction, which further relieves the magnetic frustration through bringing in new exchange paths. Thus, the variation of Li/Ni exchange ratio vs. TM mole fraction in LiNixMnyCozO2 with different compositions can be well understood and predicted in terms of magnetic frustration and superexchange interactions. This provides a unique viewpoint to study the Li/Ni ions exchange in layered Li(NixMnyCoz)O2 cathode materials.