  This paper investigates a futuristic spectrum sharing paradigm for
heterogeneous wireless networks with imperfect channels. In the heterogeneous
networks, multiple wireless networks adopt different medium access control
(MAC) protocols to share a common wireless spectrum and each network is unaware
of the MACs of others. This paper aims to design a distributed deep
reinforcement learning (DRL) based MAC protocol for a particular network, and
the objective of this network is to achieve a global $\alpha$-fairness
objective. In the conventional DRL framework, feedback/reward given to the
agent is always correctly received, so that the agent can optimize its strategy
based on the received reward. In our wireless application where the channels
are noisy, the feedback/reward (i.e., the ACK packet) may be lost due to
channel noise and interference. Without correct feedback, the agent (i.e., the
network user) may fail to find a good solution. Moreover, in the distributed
protocol, each agent makes decisions on its own. It is a challenge to guarantee
that the multiple agents will make coherent decisions and work together to
achieve the same objective, particularly in the face of imperfect feedback
channels. To tackle the challenge, we put forth (i) a feedback recovery
mechanism to recover missing feedback information, and (ii) a two-stage action
selection mechanism to aid coherent decision making to reduce transmission
collisions among the agents. Extensive simulation results demonstrate the
effectiveness of these two mechanisms. Last but not least, we believe that the
feedback recovery mechanism and the two-stage action selection mechanism can
also be used in general distributed multi-agent reinforcement learning problems
in which feedback information on rewards can be corrupted.

Networking and Internet Architecture