Abstract
A common solution criterion for partially observable Markov decision processes (POMDPs) is to maximize the expected sum of exponentially discounted rewards, for which a variety of approximate methods have been proposed. Those that plan in the belief space typically provide tighter performance guarantees, but those that plan over the state space (e.g., QMDP and FIB) often require much less memory and computation. This paper presents an encouraging result that shows that reducing the discount factor while planning in the state space can actually improve performance significantly when evaluated on the original problem. This phenomenon is confirmed by both a theoretical analysis as well as a series of empirical studies on benchmark problems. As predicted by the theory and confirmed empirically, the phenomenon is most prominent when the observation model is noisy or rewards are sparse.
Original language | English |
---|---|
Title of host publication | 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 |
Editors | Carlos Balaguer, Hajime Asama, Danica Kragic, Kevin Lynch |
Place of Publication | Piscataway, NJ, USA |
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
Pages | 3531-3536 |
Number of pages | 6 |
ISBN (Electronic) | 978-1-5386-8094-0 |
DOIs | |
Publication status | Published - 2018 |
Event | 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 - Madrid, Spain Duration: 1 Oct 2018 → 5 Oct 2018 |
Conference
Conference | 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 |
---|---|
Country/Territory | Spain |
City | Madrid |
Period | 1/10/18 → 5/10/18 |