With the increasing technological maturity and economies of scale for solar photovoltaic (PV) and electrical energy storage (EES), there is a potential for mass-scale deployment of both technologies in stand-alone and grid-connected power systems. The challenge arises in analyzing the economic projections on complex hybrid systems utilizing PV and EES. It is well known that PV power is of diurnal and stochastic nature, and surplus electrical energy is generally available in midday during high irradiance levels. EES does not produce energy as it is not a conventional generator source. Commonly, the cost of a generating asset or the power system is evaluated by using levelized cost of electricity (LCOE). In this paper, a new metric levelized cost of delivery (LCOD) is proposed to calculate the LCOE for the EES. A review on definitions in LCOE for PV hybrid energy systems is provided. Four years of solar irradiance data from Johannesburg and the national load data from Kenya are obtained for case studies. The proposed cost calculation methods are evaluated with two types of EES, namely Vanadium redox flow battery (VRB) and Lithium-ion (Li-ion) battery. It shows that the marginal LCOE and LCOD indices can be used to assist policymakers to consider the discount rate, the type of storage technology and sizing of components in a PV-EES hybrid system.