--- _id: '12738' abstract: - lang: eng text: We study turn-based stochastic zero-sum games with lexicographic preferences over objectives. Stochastic games are standard models in control, verification, and synthesis of stochastic reactive systems that exhibit both randomness as well as controllable and adversarial non-determinism. Lexicographic order allows one to consider multiple objectives with a strict preference order. To the best of our knowledge, stochastic games with lexicographic objectives have not been studied before. For a mixture of reachability and safety objectives, we show that deterministic lexicographically optimal strategies exist and memory is only required to remember the already satisfied and violated objectives. For a constant number of objectives, we show that the relevant decision problem is in NP∩coNP, matching the current known bound for single objectives; and in general the decision problem is PSPACE-hard and can be solved in NEXPTIME∩coNEXPTIME. We present an algorithm that computes the lexicographically optimal strategies via a reduction to the computation of optimal strategies in a sequence of single-objectives games. For omega-regular objectives, we restrict our analysis to one-player games, also known as Markov decision processes. We show that lexicographically optimal strategies exist and need either randomization or finite memory. We present an algorithm that solves the relevant decision problem in polynomial time. We have implemented our algorithms and report experimental results on various case studies. acknowledgement: Tobias Winkler and Joost-Pieter Katoen are supported by the DFG RTG 2236 UnRAVeL and the innovation programme under the Marie Skłodowska-Curie grant agreement No. 101008233 (Mission). Krishnendu Chatterjee is supported by the ERC CoG 863818 (ForM-SMArt) and the Vienna Science and Technology Fund (WWTF) Project ICT15-003. Maximilian Weininger is supported by the DFG projects 383882557 Statistical Unbounded Verification (SUV) and 427755713 Group-By Objectives in Probabilistic Verification (GOPro). Stefanie Mohr is supported by the DFG RTG 2428 CONVEY. Open Access funding enabled and organized by Projekt DEAL. article_processing_charge: No article_type: original author: - first_name: Krishnendu full_name: Chatterjee, Krishnendu id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-4561-241X - first_name: Joost P full_name: Katoen, Joost P id: 4524F760-F248-11E8-B48F-1D18A9856A87 last_name: Katoen - first_name: Stefanie full_name: Mohr, Stefanie last_name: Mohr - first_name: Maximilian full_name: Weininger, Maximilian last_name: Weininger - first_name: Tobias full_name: Winkler, Tobias last_name: Winkler citation: ama: Chatterjee K, Katoen JP, Mohr S, Weininger M, Winkler T. Stochastic games with lexicographic objectives. Formal Methods in System Design. 2023. doi:10.1007/s10703-023-00411-4 apa: Chatterjee, K., Katoen, J. P., Mohr, S., Weininger, M., & Winkler, T. (2023). Stochastic games with lexicographic objectives. Formal Methods in System Design. Springer Nature. https://doi.org/10.1007/s10703-023-00411-4 chicago: Chatterjee, Krishnendu, Joost P Katoen, Stefanie Mohr, Maximilian Weininger, and Tobias Winkler. “Stochastic Games with Lexicographic Objectives.” Formal Methods in System Design. Springer Nature, 2023. https://doi.org/10.1007/s10703-023-00411-4. ieee: K. Chatterjee, J. P. Katoen, S. Mohr, M. Weininger, and T. Winkler, “Stochastic games with lexicographic objectives,” Formal Methods in System Design. Springer Nature, 2023. ista: Chatterjee K, Katoen JP, Mohr S, Weininger M, Winkler T. 2023. Stochastic games with lexicographic objectives. Formal Methods in System Design. mla: Chatterjee, Krishnendu, et al. “Stochastic Games with Lexicographic Objectives.” Formal Methods in System Design, Springer Nature, 2023, doi:10.1007/s10703-023-00411-4. short: K. Chatterjee, J.P. Katoen, S. Mohr, M. Weininger, T. Winkler, Formal Methods in System Design (2023). date_created: 2023-03-19T23:00:59Z date_published: 2023-03-08T00:00:00Z date_updated: 2023-10-03T11:36:13Z day: '08' ddc: - '000' department: - _id: KrCh doi: 10.1007/s10703-023-00411-4 ec_funded: 1 external_id: isi: - '000946174300001' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.1007/s10703-023-00411-4 month: '03' oa: 1 oa_version: Published Version project: - _id: 0599E47C-7A3F-11EA-A408-12923DDC885E call_identifier: H2020 grant_number: '863818' name: 'Formal Methods for Stochastic Models: Algorithms and Applications' - _id: 25892FC0-B435-11E9-9278-68D0E5697425 grant_number: ICT15-003 name: Efficient Algorithms for Computer Aided Verification publication: Formal Methods in System Design publication_identifier: eissn: - 1572-8102 publication_status: epub_ahead publisher: Springer Nature quality_controlled: '1' related_material: record: - id: '8272' relation: earlier_version status: public scopus_import: '1' status: public title: Stochastic games with lexicographic objectives tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2023' ... --- _id: '8272' abstract: - lang: eng text: We study turn-based stochastic zero-sum games with lexicographic preferences over reachability and safety objectives. Stochastic games are standard models in control, verification, and synthesis of stochastic reactive systems that exhibit both randomness as well as angelic and demonic non-determinism. Lexicographic order allows to consider multiple objectives with a strict preference order over the satisfaction of the objectives. To the best of our knowledge, stochastic games with lexicographic objectives have not been studied before. We establish determinacy of such games and present strategy and computational complexity results. For strategy complexity, we show that lexicographically optimal strategies exist that are deterministic and memory is only required to remember the already satisfied and violated objectives. For a constant number of objectives, we show that the relevant decision problem is in NP∩coNP , matching the current known bound for single objectives; and in general the decision problem is PSPACE -hard and can be solved in NEXPTIME∩coNEXPTIME . We present an algorithm that computes the lexicographically optimal strategies via a reduction to computation of optimal strategies in a sequence of single-objectives games. We have implemented our algorithm and report experimental results on various case studies. alternative_title: - LNCS article_processing_charge: No author: - first_name: Krishnendu full_name: Chatterjee, Krishnendu id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-4561-241X - first_name: Joost P full_name: Katoen, Joost P id: 4524F760-F248-11E8-B48F-1D18A9856A87 last_name: Katoen - first_name: Maximilian full_name: Weininger, Maximilian last_name: Weininger - first_name: Tobias full_name: Winkler, Tobias last_name: Winkler citation: ama: 'Chatterjee K, Katoen JP, Weininger M, Winkler T. Stochastic games with lexicographic reachability-safety objectives. In: International Conference on Computer Aided Verification. Vol 12225. Springer Nature; 2020:398-420. doi:10.1007/978-3-030-53291-8_21' apa: Chatterjee, K., Katoen, J. P., Weininger, M., & Winkler, T. (2020). Stochastic games with lexicographic reachability-safety objectives. In International Conference on Computer Aided Verification (Vol. 12225, pp. 398–420). Springer Nature. https://doi.org/10.1007/978-3-030-53291-8_21 chicago: Chatterjee, Krishnendu, Joost P Katoen, Maximilian Weininger, and Tobias Winkler. “Stochastic Games with Lexicographic Reachability-Safety Objectives.” In International Conference on Computer Aided Verification, 12225:398–420. Springer Nature, 2020. https://doi.org/10.1007/978-3-030-53291-8_21. ieee: K. Chatterjee, J. P. Katoen, M. Weininger, and T. Winkler, “Stochastic games with lexicographic reachability-safety objectives,” in International Conference on Computer Aided Verification, 2020, vol. 12225, pp. 398–420. ista: 'Chatterjee K, Katoen JP, Weininger M, Winkler T. 2020. Stochastic games with lexicographic reachability-safety objectives. International Conference on Computer Aided Verification. CAV: Computer Aided Verification, LNCS, vol. 12225, 398–420.' mla: Chatterjee, Krishnendu, et al. “Stochastic Games with Lexicographic Reachability-Safety Objectives.” International Conference on Computer Aided Verification, vol. 12225, Springer Nature, 2020, pp. 398–420, doi:10.1007/978-3-030-53291-8_21. short: K. Chatterjee, J.P. Katoen, M. Weininger, T. Winkler, in:, International Conference on Computer Aided Verification, Springer Nature, 2020, pp. 398–420. conference: name: 'CAV: Computer Aided Verification' date_created: 2020-08-16T22:00:58Z date_published: 2020-07-14T00:00:00Z date_updated: 2023-10-03T11:36:13Z day: '14' ddc: - '000' department: - _id: KrCh doi: 10.1007/978-3-030-53291-8_21 ec_funded: 1 external_id: arxiv: - '2005.04018' isi: - '000695272500021' file: - access_level: open_access checksum: 093d4788d7d5b2ce0ffe64fbe7820043 content_type: application/pdf creator: dernst date_created: 2020-08-17T11:32:44Z date_updated: 2020-08-17T11:32:44Z file_id: '8276' file_name: 2020_LNCS_CAV_Chatterjee.pdf file_size: 625056 relation: main_file success: 1 file_date_updated: 2020-08-17T11:32:44Z has_accepted_license: '1' intvolume: ' 12225' isi: 1 language: - iso: eng month: '07' oa: 1 oa_version: Published Version page: 398-420 project: - _id: 0599E47C-7A3F-11EA-A408-12923DDC885E call_identifier: H2020 grant_number: '863818' name: 'Formal Methods for Stochastic Models: Algorithms and Applications' - _id: 25892FC0-B435-11E9-9278-68D0E5697425 grant_number: ICT15-003 name: Efficient Algorithms for Computer Aided Verification publication: International Conference on Computer Aided Verification publication_identifier: eissn: - '16113349' isbn: - '9783030532901' issn: - '03029743' publication_status: published publisher: Springer Nature quality_controlled: '1' related_material: record: - id: '12738' relation: later_version status: public scopus_import: '1' status: public title: Stochastic games with lexicographic reachability-safety objectives tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: conference user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8 volume: 12225 year: '2020' ... --- _id: '79' abstract: - lang: eng text: 'Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition probabilities to account for stochastic uncertainties of the environment such as noise or input disturbances. We study pMDPs with reachability objectives where the parameter values are unknown and impossible to measure directly during execution, but there is a probability distribution known over the parameter values. We study for the first time computing parameter-independent strategies that are expectation optimal, i.e., optimize the expected reachability probability under the probability distribution over the parameters. We present an encoding of our problem to partially observable MDPs (POMDPs), i.e., a reduction of our problem to computing optimal strategies in POMDPs. We evaluate our method experimentally on several benchmarks: a motivating (repeated) learner model; a series of benchmarks of varying configurations of a robot moving on a grid; and a consensus protocol.' alternative_title: - LNCS article_processing_charge: No author: - first_name: Sebastian full_name: Arming, Sebastian last_name: Arming - first_name: Ezio full_name: Bartocci, Ezio last_name: Bartocci - first_name: Krishnendu full_name: Chatterjee, Krishnendu id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-4561-241X - first_name: Joost P full_name: Katoen, Joost P id: 4524F760-F248-11E8-B48F-1D18A9856A87 last_name: Katoen - first_name: Ana full_name: Sokolova, Ana last_name: Sokolova citation: ama: 'Arming S, Bartocci E, Chatterjee K, Katoen JP, Sokolova A. Parameter-independent strategies for pMDPs via POMDPs. In: Vol 11024. Springer; 2018:53-70. doi:10.1007/978-3-319-99154-2_4' apa: 'Arming, S., Bartocci, E., Chatterjee, K., Katoen, J. P., & Sokolova, A. (2018). Parameter-independent strategies for pMDPs via POMDPs (Vol. 11024, pp. 53–70). Presented at the QEST: Quantitative Evaluation of Systems, Beijing, China: Springer. https://doi.org/10.1007/978-3-319-99154-2_4' chicago: Arming, Sebastian, Ezio Bartocci, Krishnendu Chatterjee, Joost P Katoen, and Ana Sokolova. “Parameter-Independent Strategies for PMDPs via POMDPs,” 11024:53–70. Springer, 2018. https://doi.org/10.1007/978-3-319-99154-2_4. ieee: 'S. Arming, E. Bartocci, K. Chatterjee, J. P. Katoen, and A. Sokolova, “Parameter-independent strategies for pMDPs via POMDPs,” presented at the QEST: Quantitative Evaluation of Systems, Beijing, China, 2018, vol. 11024, pp. 53–70.' ista: 'Arming S, Bartocci E, Chatterjee K, Katoen JP, Sokolova A. 2018. Parameter-independent strategies for pMDPs via POMDPs. QEST: Quantitative Evaluation of Systems, LNCS, vol. 11024, 53–70.' mla: Arming, Sebastian, et al. Parameter-Independent Strategies for PMDPs via POMDPs. Vol. 11024, Springer, 2018, pp. 53–70, doi:10.1007/978-3-319-99154-2_4. short: S. Arming, E. Bartocci, K. Chatterjee, J.P. Katoen, A. Sokolova, in:, Springer, 2018, pp. 53–70. conference: end_date: 2018-09-07 location: Beijing, China name: 'QEST: Quantitative Evaluation of Systems' start_date: 2018-09-04 date_created: 2018-12-11T11:44:31Z date_published: 2018-08-15T00:00:00Z date_updated: 2023-09-13T09:38:28Z day: '15' department: - _id: KrCh - _id: ToHe doi: 10.1007/978-3-319-99154-2_4 external_id: arxiv: - '1806.05126' isi: - '000548912200004' intvolume: ' 11024' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1806.05126 month: '08' oa: 1 oa_version: Preprint page: 53-70 publication_status: published publisher: Springer publist_id: '7975' quality_controlled: '1' scopus_import: '1' status: public title: Parameter-independent strategies for pMDPs via POMDPs type: conference user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 volume: 11024 year: '2018' ... --- _id: '10418' abstract: - lang: eng text: We present a new proof rule for proving almost-sure termination of probabilistic programs, including those that contain demonic non-determinism. An important question for a probabilistic program is whether the probability mass of all its diverging runs is zero, that is that it terminates "almost surely". Proving that can be hard, and this paper presents a new method for doing so. It applies directly to the program's source code, even if the program contains demonic choice. Like others, we use variant functions (a.k.a. "super-martingales") that are real-valued and decrease randomly on each loop iteration; but our key innovation is that the amount as well as the probability of the decrease are parametric. We prove the soundness of the new rule, indicate where its applicability goes beyond existing rules, and explain its connection to classical results on denumerable (non-demonic) Markov chains. acknowledgement: "McIver and Morgan are grateful to David Basin and the Information Security Group at ETH Zürich for hosting a six-month stay in Switzerland, during part of which this work began. And thanks particularly to Andreas Lochbihler, who shared with us the probabilistic termination problem that led to it. They acknowledge the support of ARC grant DP140101119. Part of this work was carried out during the Workshop on Probabilistic Programming Semantics\r\nat McGill University’s Bellairs Research Institute on Barbados organised by Alexandra Silva and\r\nPrakash Panangaden. Kaminski and Katoen are grateful to Sebastian Junges for spotting a flaw in §5.4." article_number: '33' article_processing_charge: No article_type: original author: - first_name: Annabelle full_name: Mciver, Annabelle last_name: Mciver - first_name: Carroll full_name: Morgan, Carroll last_name: Morgan - first_name: Benjamin Lucien full_name: Kaminski, Benjamin Lucien last_name: Kaminski - first_name: Joost P full_name: Katoen, Joost P id: 4524F760-F248-11E8-B48F-1D18A9856A87 last_name: Katoen citation: ama: Mciver A, Morgan C, Kaminski BL, Katoen JP. A new proof rule for almost-sure termination. Proceedings of the ACM on Programming Languages. 2017;2(POPL). doi:10.1145/3158121 apa: 'Mciver, A., Morgan, C., Kaminski, B. L., & Katoen, J. P. (2017). A new proof rule for almost-sure termination. Proceedings of the ACM on Programming Languages. Los Angeles, CA, United States: Association for Computing Machinery. https://doi.org/10.1145/3158121' chicago: Mciver, Annabelle, Carroll Morgan, Benjamin Lucien Kaminski, and Joost P Katoen. “A New Proof Rule for Almost-Sure Termination.” Proceedings of the ACM on Programming Languages. Association for Computing Machinery, 2017. https://doi.org/10.1145/3158121. ieee: A. Mciver, C. Morgan, B. L. Kaminski, and J. P. Katoen, “A new proof rule for almost-sure termination,” Proceedings of the ACM on Programming Languages, vol. 2, no. POPL. Association for Computing Machinery, 2017. ista: Mciver A, Morgan C, Kaminski BL, Katoen JP. 2017. A new proof rule for almost-sure termination. Proceedings of the ACM on Programming Languages. 2(POPL), 33. mla: Mciver, Annabelle, et al. “A New Proof Rule for Almost-Sure Termination.” Proceedings of the ACM on Programming Languages, vol. 2, no. POPL, 33, Association for Computing Machinery, 2017, doi:10.1145/3158121. short: A. Mciver, C. Morgan, B.L. Kaminski, J.P. Katoen, Proceedings of the ACM on Programming Languages 2 (2017). conference: end_date: 2018-01-13 location: Los Angeles, CA, United States name: 'POPL: Programming Languages' start_date: 2018-01-07 date_created: 2021-12-05T23:01:49Z date_published: 2017-12-07T00:00:00Z date_updated: 2021-12-07T08:04:14Z day: '07' department: - _id: KrCh - _id: ToHe doi: 10.1145/3158121 external_id: arxiv: - '1711.03588' intvolume: ' 2' issue: POPL language: - iso: eng main_file_link: - open_access: '1' url: https://dl.acm.org/doi/10.1145/3158121 month: '12' oa: 1 oa_version: Published Version publication: Proceedings of the ACM on Programming Languages publication_identifier: eissn: - 2475-1421 publication_status: published publisher: Association for Computing Machinery quality_controlled: '1' scopus_import: '1' status: public title: A new proof rule for almost-sure termination type: journal_article user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 volume: 2 year: '2017' ...