A nucleus for Bayesian Partially Observable Markov Games: Joint observer and mechanism design

An intelligent agent suggests an autonomous entity, which manages and learns actions to be taken towards achieving goals. The problem, reported as common knowledge in the literature in Artificial Intelligence (AI), is that it is a challenge to develop an approach able to compute efficient decisions that maximize the total reward of interacting agents upon an environment with unknown, incomplete, and uncertain information. To address these shortcomings, this paper provides a step forward: a nucleus for Bayesian Partially Observable Markov Games (BPOMGs) supported by an AI approach. Three fundamental topics conform the structure of the nucleus: game theory, learning and inference. First, we present a novel general Bayesian approach which is conceptualized for games that considered both, the incomplete information of the Bayesian model and the incomplete information over the states of the Markov system. In this new model, execution uncertainty is handled by using a Partially Observable Markov Game (POMG). Second, we extend the design theory, which now involves the mechanism design and the joint observer design (both unknown). The mechanism design results from the fact that agents act in their own individuals’ self-interest, and to induce agents to not reveal their private information and create a particular outcome. The joint observer design goal is related to represent the fact that agents may not be interested to provide accurate information of their states. In addition, agents follow a model that employs a Reinforcement Learning (RL) approach for estimating the transition matrices (also unknown) at each time step. Hence, as our final contribution, is an extended model of POMGs by introducing a new variable and proposing an analytical solution to compute both the observer design and the mechanism design (the two unknown). The proposed extension makes the game theory problem computationally tractable. We derive relations to recover analytically the variables of interest for each agent, i.e. observation kernels, joint observers, mechanism, strategies, and distribution vectors. The usefulness and effectiveness of the proposed nucleus is validated in simulation on a game-theoretic analysis of the patrolling problem designing the mechanism, computing the observers, and employing an RL approach.