{"abstract":[{"lang":"eng","text":"Multiple-environment Markov decision processes (MEMDPs) are MDPs equipped with not one, but multiple probabilistic transition functions, which represent the various possible unknown environments. While the previous research on MEMDPs focused on theoretical properties for long-run average payoff, we study them with discounted-sum payoff and focus on their practical advantages and applications. MEMDPs can be viewed as a special case of Partially observable and Mixed observability MDPs: the state of the system is perfectly observable, but not the environment. We show that the specific structure of MEMDPs allows for more efficient algorithmic analysis, in particular for faster belief updates. We demonstrate the applicability of MEMDPs in several domains. In particular, we formalize the sequential decision-making approach to contextual recommendation systems as MEMDPs and substantially improve over the previous MDP approach."}],"year":"2020","title":"Multiple-environment Markov decision processes: Efficient analysis and applications","_id":"8193","month":"06","related_material":{"record":[{"relation":"dissertation_contains","status":"public","id":"8390"}]},"oa_version":"None","conference":{"start_date":"2020-10-26","name":"ICAPS: International Conference on Automated Planning and Scheduling","end_date":"2020-10-30","location":"Nancy, France"},"status":"public","project":[{"_id":"25863FF4-B435-11E9-9278-68D0E5697425","grant_number":"S11407","name":"Game Theory","call_identifier":"FWF"}],"department":[{"_id":"KrCh"}],"publication":"Proceedings of the 30th International Conference on Automated Planning and Scheduling","publisher":"Association for the Advancement of Artificial Intelligence","page":"48-56","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","citation":{"chicago":"Chatterjee, Krishnendu, Martin Chmelik, Deep Karkhanis, Petr Novotný, and Amélie Royer. “Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications.” In <i>Proceedings of the 30th International Conference on Automated Planning and Scheduling</i>, 30:48–56. Association for the Advancement of Artificial Intelligence, 2020.","mla":"Chatterjee, Krishnendu, et al. “Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications.” <i>Proceedings of the 30th International Conference on Automated Planning and Scheduling</i>, vol. 30, Association for the Advancement of Artificial Intelligence, 2020, pp. 48–56.","short":"K. Chatterjee, M. Chmelik, D. Karkhanis, P. Novotný, A. Royer, in:, Proceedings of the 30th International Conference on Automated Planning and Scheduling, Association for the Advancement of Artificial Intelligence, 2020, pp. 48–56.","ista":"Chatterjee K, Chmelik M, Karkhanis D, Novotný P, Royer A. 2020. Multiple-environment Markov decision processes: Efficient analysis and applications. Proceedings of the 30th International Conference on Automated Planning and Scheduling. ICAPS: International Conference on Automated Planning and Scheduling vol. 30, 48–56.","apa":"Chatterjee, K., Chmelik, M., Karkhanis, D., Novotný, P., &#38; Royer, A. (2020). Multiple-environment Markov decision processes: Efficient analysis and applications. In <i>Proceedings of the 30th International Conference on Automated Planning and Scheduling</i> (Vol. 30, pp. 48–56). Nancy, France: Association for the Advancement of Artificial Intelligence.","ieee":"K. Chatterjee, M. Chmelik, D. Karkhanis, P. Novotný, and A. Royer, “Multiple-environment Markov decision processes: Efficient analysis and applications,” in <i>Proceedings of the 30th International Conference on Automated Planning and Scheduling</i>, Nancy, France, 2020, vol. 30, pp. 48–56.","ama":"Chatterjee K, Chmelik M, Karkhanis D, Novotný P, Royer A. Multiple-environment Markov decision processes: Efficient analysis and applications. In: <i>Proceedings of the 30th International Conference on Automated Planning and Scheduling</i>. Vol 30. Association for the Advancement of Artificial Intelligence; 2020:48-56."},"publication_status":"published","language":[{"iso":"eng"}],"volume":30,"date_published":"2020-06-01T00:00:00Z","publication_identifier":{"issn":["23340835"],"eissn":["23340843"]},"date_updated":"2023-09-07T13:16:18Z","author":[{"last_name":"Chatterjee","orcid":"0000-0002-4561-241X","full_name":"Chatterjee, Krishnendu","first_name":"Krishnendu","id":"2E5DCA20-F248-11E8-B48F-1D18A9856A87"},{"id":"3624234E-F248-11E8-B48F-1D18A9856A87","first_name":"Martin","full_name":"Chmelik, Martin","last_name":"Chmelik"},{"last_name":"Karkhanis","full_name":"Karkhanis, Deep","first_name":"Deep"},{"id":"3CC3B868-F248-11E8-B48F-1D18A9856A87","first_name":"Petr","full_name":"Novotný, Petr","last_name":"Novotný"},{"last_name":"Royer","id":"3811D890-F248-11E8-B48F-1D18A9856A87","first_name":"Amélie","full_name":"Royer, Amélie","orcid":"0000-0002-8407-0705"}],"date_created":"2020-08-02T22:00:58Z","quality_controlled":"1","type":"conference","acknowledgement":"Krishnendu Chatterjee is supported by the Austrian ScienceFund (FWF) NFN Grant No. S11407-N23 (RiSE/SHiNE),and COST Action GAMENET. Petr Novotn ́y is supported bythe Czech Science Foundation grant No. GJ19-15134Y.","scopus_import":"1","day":"01","article_processing_charge":"No","intvolume":"        30"}