Optimizing expectation with guarantees in POMDPs

K. Chatterjee, P. Novotny, G. Pérez, J. Raskin, D. Zikelic, in:, Proceedings of the 31st AAAI Conference on Artificial Intelligence, AAAI Press, 2017, pp. 3725–3732.

Conference Paper | Published | English
Author
; ; ; ;
Department
Abstract
A standard objective in partially-observable Markov decision processes (POMDPs) is to find a policy that maximizes the expected discounted-sum payoff. However, such policies may still permit unlikely but highly undesirable outcomes, which is problematic especially in safety-critical applications. Recently, there has been a surge of interest in POMDPs where the goal is to maximize the probability to ensure that the payoff is at least a given threshold, but these approaches do not consider any optimization beyond satisfying this threshold constraint. In this work we go beyond both the “expectation” and “threshold” approaches and consider a “guaranteed payoff optimization (GPO)” problem for POMDPs, where we are given a threshold t and the objective is to find a policy σ such that a) each possible outcome of σ yields a discounted-sum payoff of at least t, and b) the expected discounted-sum payoff of σ is optimal (or near-optimal) among all policies satisfying a). We present a practical approach to tackle the GPO problem and evaluate it on standard POMDP benchmarks.
Publishing Year
Date Published
2017-01-01
Proceedings Title
Proceedings of the 31st AAAI Conference on Artificial Intelligence
Acknowledgement
he research leading to these results was supported by the Austrian Science Fund (FWF) NFN Grant no. S11407-N23 (RiSE/SHiNE); two ERC Starting grants (279307: Graph Games, 279499: inVEST); the Vienna Science and Tech- nology Fund (WWTF) through project ICT15-003; and the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no. [291734].
Volume
5
Page
3725 - 3732
Conference
AAAI: Conference on Artificial Intelligence
Conference Location
San Francisco, CA, United States
Conference Date
2017-02-04 – 2017-02-10
IST-REx-ID

Cite this

Chatterjee K, Novotny P, Pérez G, Raskin J, Zikelic D. Optimizing expectation with guarantees in POMDPs. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. Vol 5. AAAI Press; 2017:3725-3732.
Chatterjee, K., Novotny, P., Pérez, G., Raskin, J., & Zikelic, D. (2017). Optimizing expectation with guarantees in POMDPs. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (Vol. 5, pp. 3725–3732). San Francisco, CA, United States: AAAI Press.
Chatterjee, Krishnendu, Petr Novotny, Guillermo Pérez, Jean Raskin, and Djordje Zikelic. “Optimizing Expectation with Guarantees in POMDPs.” In Proceedings of the 31st AAAI Conference on Artificial Intelligence, 5:3725–32. AAAI Press, 2017.
K. Chatterjee, P. Novotny, G. Pérez, J. Raskin, and D. Zikelic, “Optimizing expectation with guarantees in POMDPs,” in Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, CA, United States, 2017, vol. 5, pp. 3725–3732.
Chatterjee K, Novotny P, Pérez G, Raskin J, Zikelic D. 2017. Optimizing expectation with guarantees in POMDPs. Proceedings of the 31st AAAI Conference on Artificial Intelligence. AAAI: Conference on Artificial Intelligence vol. 5. 3725–3732.
Chatterjee, Krishnendu, et al. “Optimizing Expectation with Guarantees in POMDPs.” Proceedings of the 31st AAAI Conference on Artificial Intelligence, vol. 5, AAAI Press, 2017, pp. 3725–32.

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar