HSVI-based Online Minimax Strategies for Partially Observable Stochastic Games with Neural Perception Mechanisms
<|reference_start|>HSVI-based Online Minimax Strategies for Partially Observable Stochastic Games with Neural Perception Mechanisms: We consider a variant of continuous-state partially-observable stochastic games with neural perception mechanisms and an asymmetric information structure. One agent has partial information, with the observation function implemented as a neural network, while the other agent is assumed to have full knowledge of the state. We present, for the first time, an efficient online method to compute an $\varepsilon$-minimax strategy profile, which requires only one linear program to be solved for each agent at every stage, instead of a complex estimation of opponent counterfactual values. For the partially-informed agent, we propose a continual resolving approach which uses lower bounds, pre-computed offline with heuristic search value iteration (HSVI), instead of opponent counterfactual values. This inherits the soundness of continual resolving at the cost of pre-computing the bound. For the fully-informed agent, we propose an inferred-belief strategy, where the agent maintains an inferred belief about the belief of the partially-informed agent based on (offline) upper bounds from HSVI, guaranteeing $\varepsilon$-distance to the value of the game at the initial belief known to both agents.<|reference_end|>
@article{yan2024hsvi-based,
    title={HSVI-based Online Minimax Strategies for Partially Observable Stochastic
  Games with Neural Perception Mechanisms},
    author={Rui Yan, Gabriel Santos, Gethin Norman, David Parker and Marta
  Kwiatkowska},
    journal={arXiv preprint arXiv:2404.10679},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.10679},
    primaryClass={cs.GT cs.AI}
}