Trading performance for stability in Markov decision processes

T. Brázdil, K. Chatterjee, V. Forejt, A. Kučera, Journal of Computer and System Sciences 84 (2017) 144–170.

Download
OA IST-2016-717-v1+1_1-s2.0-S0022000016300897-main.pdf 708.66 KB

Journal Article | Published | English

Scopus indexed
Author
Brázdil, Tomáš; Chatterjee, KrishnenduIST Austria ; Forejt, Vojtěch; Kučera, Antonín
Department
Abstract
We study controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize the expected mean-payoff performance and stability (also known as variability in the literature). We argue that the basic notion of expressing the stability using the statistical variance of the mean payoff is sometimes insufficient, and propose an alternative definition. We show that a strategy ensuring both the expected mean payoff and the variance below given bounds requires randomization and memory, under both the above definitions. We then show that the problem of finding such a strategy can be expressed as a set of constraints.
Publishing Year
Date Published
2017-03-01
Journal Title
Journal of Computer and System Sciences
Volume
84
Page
144 - 170
IST-REx-ID

Cite this

Brázdil T, Chatterjee K, Forejt V, Kučera A. Trading performance for stability in Markov decision processes. Journal of Computer and System Sciences. 2017;84:144-170. doi:10.1016/j.jcss.2016.09.009
Brázdil, T., Chatterjee, K., Forejt, V., & Kučera, A. (2017). Trading performance for stability in Markov decision processes. Journal of Computer and System Sciences, 84, 144–170. https://doi.org/10.1016/j.jcss.2016.09.009
Brázdil, Tomáš, Krishnendu Chatterjee, Vojtěch Forejt, and Antonín Kučera. “Trading Performance for Stability in Markov Decision Processes.” Journal of Computer and System Sciences 84 (2017): 144–70. https://doi.org/10.1016/j.jcss.2016.09.009.
T. Brázdil, K. Chatterjee, V. Forejt, and A. Kučera, “Trading performance for stability in Markov decision processes,” Journal of Computer and System Sciences, vol. 84, pp. 144–170, 2017.
Brázdil T, Chatterjee K, Forejt V, Kučera A. 2017. Trading performance for stability in Markov decision processes. Journal of Computer and System Sciences. 84, 144–170.
Brázdil, Tomáš, et al. “Trading Performance for Stability in Markov Decision Processes.” Journal of Computer and System Sciences, vol. 84, Elsevier, 2017, pp. 144–70, doi:10.1016/j.jcss.2016.09.009.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
Access Level
OA Open Access
Date Uploaded
2018-12-12
MD5 Checksum
91271b23cf884d7c06d33bef0cd623b1


Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar