• p7-katt

    Final published version, 1 MB, PDF-document


Model-based Bayesian Reinforcement Learning (BRL) provides a principled solution to dealing with the exploration-exploitation trade-off, but such methods typically assume a fully observable environments. The few Bayesian RL methods that are applicable in partially observable domains, such as the Bayes-Adaptive POMDP (BA-POMDP), scale poorly. To address this issue, we introduce the Factored BA-POMDP model (FBA-POMDP), a framework that is able to learn a compact model of the dynamics by exploiting the underlying structure of a POMDP. The FBA-POMDP framework casts the problem as a planning task, for which we adapt the Monte-Carlo Tree Search planning algorithm and develop a belief tracking method to approximate the joint posterior over the state and model variables. Our empirical results show that this method outperforms a number of BRL baselines and is able to learn efficiently when the factorization is known, as well as learn both the factorization and the model parameters simultaneously.
Original languageEnglish
Title of host publicationAAMAS'19
Subtitle of host publicationProceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Number of pages9
ISBN (Print)978-1-4503-6309-9
Publication statusPublished - 2019
EventAAMAS 2019: The 18th International Conference on Autonomous Agents and MultiAgent Systems - Montreal, Canada
Duration: 13 May 201917 May 2019


ConferenceAAMAS 2019

    Research areas

  • Bayesian reinforcement learning, POMDPs, Monte-Chain Monte Carlo, Monte-Carlo Tree Search, Bayes Networks

ID: 67208707