Goal-HSVI: Heuristic search value iteration for goal-POMDPs

K. Horák, B. Bošanský, K. Chatterjee, in:, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI, 2018, pp. 4764–4770.

Download
No fulltext has been uploaded. References only!

Conference Paper | Published | English
Author
; ;
Department
Abstract
Partially observable Markov decision processes (POMDPs) are the standard models for planning under uncertainty with both finite and infinite horizon. Besides the well-known discounted-sum objective, indefinite-horizon objective (aka Goal-POMDPs) is another classical objective for POMDPs. In this case, given a set of target states and a positive cost for each transition, the optimization objective is to minimize the expected total cost until a target state is reached. In the literature, RTDP-Bel or heuristic search value iteration (HSVI) have been used for solving Goal-POMDPs. Neither of these algorithms has theoretical convergence guarantees, and HSVI may even fail to terminate its trials. We give the following contributions: (1) We discuss the challenges introduced in Goal-POMDPs and illustrate how they prevent the original HSVI from converging. (2) We present a novel algorithm inspired by HSVI, termed Goal-HSVI, and show that our algorithm has convergence guarantees. (3) We show that Goal-HSVI outperforms RTDP-Bel on a set of well-known examples.
Publishing Year
Date Published
2018-07-01
Proceedings Title
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Volume
2018-July
Page
4764 - 4770
Conference
IJCAI: International Joint Conference on Artificial Intelligence
Conference Location
Stockholm, Sweden
Conference Date
2018-07-13 – 2018-07-19
IST-REx-ID

Cite this

Horák K, Bošanský B, Chatterjee K. Goal-HSVI: Heuristic search value iteration for goal-POMDPs. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. Vol 2018-July. IJCAI; 2018:4764-4770. doi:10.24963/ijcai.2018/662
Horák, K., Bošanský, B., & Chatterjee, K. (2018). Goal-HSVI: Heuristic search value iteration for goal-POMDPs. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (Vol. 2018–July, pp. 4764–4770). Stockholm, Sweden: IJCAI. https://doi.org/10.24963/ijcai.2018/662
Horák, Karel, Branislav Bošanský, and Krishnendu Chatterjee. “Goal-HSVI: Heuristic Search Value Iteration for Goal-POMDPs.” In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018–July:4764–70. IJCAI, 2018. https://doi.org/10.24963/ijcai.2018/662 .
K. Horák, B. Bošanský, and K. Chatterjee, “Goal-HSVI: Heuristic search value iteration for goal-POMDPs,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018, vol. 2018–July, pp. 4764–4770.
Horák K, Bošanský B, Chatterjee K. 2018. Goal-HSVI: Heuristic search value iteration for goal-POMDPs. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. IJCAI: International Joint Conference on Artificial Intelligence vol. 2018–July. 4764–4770.
Horák, Karel, et al. “Goal-HSVI: Heuristic Search Value Iteration for Goal-POMDPs.” Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, vol. 2018–July, IJCAI, 2018, pp. 4764–70, doi:10.24963/ijcai.2018/662 .

Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar