  This article investigates the cache-enabling unmanned aerial vehicle (UAV)
cellular networks with massive access capability supported by non-orthogonal
multiple access (NOMA). The delivery of a large volume of multimedia contents
for ground users is assisted by a mobile UAV base station, which caches some
popular contents for wireless backhaul link traffic offloading. In
cache-enabling UAV NOMA networks, the caching placement of content caching
phase and radio resource allocation of content delivery phase are crucial for
network performance. To cope with the dynamic UAV locations and content
requests in practical scenarios, we formulate the long-term caching placement
and resource allocation optimization problem for content delivery delay
minimization as a Markov decision process (MDP). The UAV acts as an agent to
take actions for caching placement and resource allocation, which includes the
user scheduling of content requests and the power allocation of NOMA users. In
order to tackle the MDP, we propose a Q-learning based caching placement and
resource allocation algorithm, where the UAV learns and selects action with
\emph{soft ${\varepsilon}$-greedy} strategy to search for the optimal match
between actions and states. Since the action-state table size of Q-learning
grows with the number of states in the dynamic networks, we propose a function
approximation based algorithm with combination of stochastic gradient descent
and deep neural networks, which is suitable for large-scale networks. Finally,
the numerical results show that the proposed algorithms provide considerable
performance compared to benchmark algorithms, and obtain a trade-off between
network performance and calculation complexity.

Signal Processing;Machine Learning