--- _id: '1325' abstract: - lang: eng text: We study graphs and two-player games in which rewards are assigned to states, and the goal of the players is to satisfy or dissatisfy certain property of the generated outcome, given as a mean payoff property. Since the notion of mean-payoff does not reflect possible fluctuations from the mean-payoff along a run, we propose definitions and algorithms for capturing the stability of the system, and give algorithms for deciding if a given mean payoff and stability objective can be ensured in the system. acknowledgement: "The work has been supported by the Czech Science Foundation, grant No. 15-17564S, by EPSRC grant\r\nEP/M023656/1, and by the People Programme (Marie Curie Actions) of the European Union’s Seventh\r\nFramework Programme (FP7/2007-2013) under REA grant agreement no [291734]" alternative_title: - LIPIcs article_number: '10' author: - first_name: Tomáš full_name: Brázdil, Tomáš last_name: Brázdil - first_name: Vojtěch full_name: Forejt, Vojtěch last_name: Forejt - first_name: Antonín full_name: Kučera, Antonín last_name: Kučera - first_name: Petr full_name: Novotny, Petr id: 3CC3B868-F248-11E8-B48F-1D18A9856A87 last_name: Novotny citation: ama: 'Brázdil T, Forejt V, Kučera A, Novotný P. Stability in graphs and games. In: Vol 59. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2016. doi:10.4230/LIPIcs.CONCUR.2016.10' apa: 'Brázdil, T., Forejt, V., Kučera, A., & Novotný, P. (2016). Stability in graphs and games (Vol. 59). Presented at the CONCUR: Concurrency Theory, Quebec City, Canada: Schloss Dagstuhl - Leibniz-Zentrum für Informatik. https://doi.org/10.4230/LIPIcs.CONCUR.2016.10' chicago: Brázdil, Tomáš, Vojtěch Forejt, Antonín Kučera, and Petr Novotný. “Stability in Graphs and Games,” Vol. 59. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2016. https://doi.org/10.4230/LIPIcs.CONCUR.2016.10. ieee: 'T. Brázdil, V. Forejt, A. Kučera, and P. Novotný, “Stability in graphs and games,” presented at the CONCUR: Concurrency Theory, Quebec City, Canada, 2016, vol. 59.' ista: 'Brázdil T, Forejt V, Kučera A, Novotný P. 2016. Stability in graphs and games. CONCUR: Concurrency Theory, LIPIcs, vol. 59, 10.' mla: Brázdil, Tomáš, et al. Stability in Graphs and Games. Vol. 59, 10, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2016, doi:10.4230/LIPIcs.CONCUR.2016.10. short: T. Brázdil, V. Forejt, A. Kučera, P. Novotný, in:, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2016. conference: end_date: 2016-08-26 location: Quebec City, Canada name: 'CONCUR: Concurrency Theory' start_date: 2016-08-23 date_created: 2018-12-11T11:51:23Z date_published: 2016-08-01T00:00:00Z date_updated: 2021-01-12T06:49:53Z day: '01' ddc: - '004' department: - _id: KrCh doi: 10.4230/LIPIcs.CONCUR.2016.10 ec_funded: 1 file: - access_level: open_access checksum: 3c2dc6ab0358f8aa8f7aa7d6c1293159 content_type: application/pdf creator: system date_created: 2018-12-12T10:16:40Z date_updated: 2020-07-14T12:44:44Z file_id: '5229' file_name: IST-2016-665-v1+1_Forejt_et_al__Stability_in_graphs_and_games.pdf file_size: 553648 relation: main_file file_date_updated: 2020-07-14T12:44:44Z has_accepted_license: '1' intvolume: ' 59' language: - iso: eng license: https://creativecommons.org/licenses/by/4.0/ month: '08' oa: 1 oa_version: Published Version project: - _id: 25681D80-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '291734' name: International IST Postdoc Fellowship Programme publication_status: published publisher: Schloss Dagstuhl - Leibniz-Zentrum für Informatik publist_id: '5944' pubrep_id: '665' quality_controlled: '1' scopus_import: 1 status: public title: Stability in graphs and games tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: conference user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 volume: 59 year: '2016' ... --- _id: '1324' abstract: - lang: eng text: 'DEC-POMDPs extend POMDPs to a multi-agent setting, where several agents operate in an uncertain environment independently to achieve a joint objective. DEC-POMDPs have been studied with finite-horizon and infinite-horizon discounted-sum objectives, and there exist solvers both for exact and approximate solutions. In this work we consider Goal-DEC-POMDPs, where given a set of target states, the objective is to ensure that the target set is reached with minimal cost. We consider the indefinite-horizon (infinite-horizon with either discounted-sum, or undiscounted-sum, where absorbing goal states have zero-cost) problem. We present a new and novel method to solve the problem that extends methods for finite-horizon DEC-POMDPs and the RTDP-Bel approach for POMDPs. We present experimental results on several examples, and show that our approach presents promising results. Copyright ' author: - first_name: Krishnendu full_name: Chatterjee, Krishnendu id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-4561-241X - first_name: Martin full_name: Chmelik, Martin id: 3624234E-F248-11E8-B48F-1D18A9856A87 last_name: Chmelik citation: ama: 'Chatterjee K, Chmelik M. Indefinite-horizon reachability in Goal-DEC-POMDPs. In: Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling. Vol 2016-January. AAAI Press; 2016:88-96.' apa: 'Chatterjee, K., & Chmelik, M. (2016). Indefinite-horizon reachability in Goal-DEC-POMDPs. In Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling (Vol. 2016–January, pp. 88–96). London, United Kingdom: AAAI Press.' chicago: Chatterjee, Krishnendu, and Martin Chmelik. “Indefinite-Horizon Reachability in Goal-DEC-POMDPs.” In Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling, 2016–January:88–96. AAAI Press, 2016. ieee: K. Chatterjee and M. Chmelik, “Indefinite-horizon reachability in Goal-DEC-POMDPs,” in Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling, London, United Kingdom, 2016, vol. 2016–January, pp. 88–96. ista: 'Chatterjee K, Chmelik M. 2016. Indefinite-horizon reachability in Goal-DEC-POMDPs. Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling. ICAPS: International Conference on Automated Planning and Scheduling vol. 2016–January, 88–96.' mla: Chatterjee, Krishnendu, and Martin Chmelik. “Indefinite-Horizon Reachability in Goal-DEC-POMDPs.” Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling, vol. 2016–January, AAAI Press, 2016, pp. 88–96. short: K. Chatterjee, M. Chmelik, in:, Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling, AAAI Press, 2016, pp. 88–96. conference: end_date: 2016-06-17 location: London, United Kingdom name: 'ICAPS: International Conference on Automated Planning and Scheduling' start_date: 2016-06-12 date_created: 2018-12-11T11:51:22Z date_published: 2016-01-01T00:00:00Z date_updated: 2021-01-12T06:49:53Z day: '01' department: - _id: KrCh ec_funded: 1 language: - iso: eng main_file_link: - url: http://www.aaai.org/ocs/index.php/ICAPS/ICAPS16/paper/view/12999 month: '01' oa_version: None page: 88 - 96 project: - _id: 2584A770-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: P 23499-N23 name: Modern Graph Algorithmic Techniques in Formal Verification - _id: 2581B60A-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '279307' name: 'Quantitative Graph Games: Theory and Applications' - _id: 25832EC2-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: S 11407_N23 name: Rigorous Systems Engineering publication: Proceedings of the Twenty-Sixth International Conference on International Conference on Automated Planning and Scheduling publication_status: published publisher: AAAI Press publist_id: '5946' quality_controlled: '1' scopus_import: 1 status: public title: Indefinite-horizon reachability in Goal-DEC-POMDPs type: conference user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 volume: 2016-January year: '2016' ... --- _id: '1327' abstract: - lang: eng text: 'We consider partially observable Markov decision processes (POMDPs) with a set of target states and positive integer costs associated with every transition. The traditional optimization objective (stochastic shortest path) asks to minimize the expected total cost until the target set is reached. We extend the traditional framework of POMDPs to model energy consumption, which represents a hard constraint. The energy levels may increase and decrease with transitions, and the hard constraint requires that the energy level must remain positive in all steps till the target is reached. First, we present a novel algorithm for solving POMDPs with energy levels, developing on existing POMDP solvers and using RTDP as its main method. Our second contribution is related to policy representation. For larger POMDP instances the policies computed by existing solvers are too large to be understandable. We present an automated procedure based on machine learning techniques that automatically extracts important decisions of the policy allowing us to compute succinct human readable policies. Finally, we show experimentally that our algorithm performs well and computes succinct policies on a number of POMDP instances from the literature that were naturally enhanced with energy levels. ' author: - first_name: Tomáš full_name: Brázdil, Tomáš last_name: Brázdil - first_name: Krishnendu full_name: Chatterjee, Krishnendu id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87 last_name: Chatterjee orcid: 0000-0002-4561-241X - first_name: Martin full_name: Chmelik, Martin id: 3624234E-F248-11E8-B48F-1D18A9856A87 last_name: Chmelik - first_name: Anchit full_name: Gupta, Anchit last_name: Gupta - first_name: Petr full_name: Novotny, Petr id: 3CC3B868-F248-11E8-B48F-1D18A9856A87 last_name: Novotny citation: ama: 'Brázdil T, Chatterjee K, Chmelik M, Gupta A, Novotný P. Stochastic shortest path with energy constraints in POMDPs. In: Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems. ACM; 2016:1465-1466.' apa: 'Brázdil, T., Chatterjee, K., Chmelik, M., Gupta, A., & Novotný, P. (2016). Stochastic shortest path with energy constraints in POMDPs. In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (pp. 1465–1466). Singapore: ACM.' chicago: Brázdil, Tomáš, Krishnendu Chatterjee, Martin Chmelik, Anchit Gupta, and Petr Novotný. “Stochastic Shortest Path with Energy Constraints in POMDPs.” In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems, 1465–66. ACM, 2016. ieee: T. Brázdil, K. Chatterjee, M. Chmelik, A. Gupta, and P. Novotný, “Stochastic shortest path with energy constraints in POMDPs,” in Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems, Singapore, 2016, pp. 1465–1466. ista: 'Brázdil T, Chatterjee K, Chmelik M, Gupta A, Novotný P. 2016. Stochastic shortest path with energy constraints in POMDPs. Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems. AAMAS: Autonomous Agents & Multiagent Systems, 1465–1466.' mla: Brázdil, Tomáš, et al. “Stochastic Shortest Path with Energy Constraints in POMDPs.” Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems, ACM, 2016, pp. 1465–66. short: T. Brázdil, K. Chatterjee, M. Chmelik, A. Gupta, P. Novotný, in:, Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems, ACM, 2016, pp. 1465–1466. conference: end_date: 2016-05-13 location: Singapore name: 'AAMAS: Autonomous Agents & Multiagent Systems' start_date: 2016-05-09 date_created: 2018-12-11T11:51:23Z date_published: 2016-01-01T00:00:00Z date_updated: 2021-01-12T06:49:54Z day: '01' department: - _id: KrCh ec_funded: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1602.07565 month: '01' oa: 1 oa_version: Preprint page: 1465 - 1466 project: - _id: 2584A770-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: P 23499-N23 name: Modern Graph Algorithmic Techniques in Formal Verification - _id: 25832EC2-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: S 11407_N23 name: Rigorous Systems Engineering - _id: 25681D80-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '291734' name: International IST Postdoc Fellowship Programme - _id: 2581B60A-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '279307' name: 'Quantitative Graph Games: Theory and Applications' publication: Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems publication_status: published publisher: ACM publist_id: '5942' quality_controlled: '1' scopus_import: 1 status: public title: Stochastic shortest path with energy constraints in POMDPs type: conference user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 year: '2016' ... --- _id: '1326' abstract: - lang: eng text: 'Energy Markov Decision Processes (EMDPs) are finite-state Markov decision processes where each transition is assigned an integer counter update and a rational payoff. An EMDP configuration is a pair s(n), where s is a control state and n is the current counter value. The configurations are changed by performing transitions in the standard way. We consider the problem of computing a safe strategy (i.e., a strategy that keeps the counter non-negative) which maximizes the expected mean payoff. ' acknowledgement: The research was funded by the Czech Science Foundation Grant No. P202/12/G061 and by the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no [291734]. alternative_title: - LNCS author: - first_name: Tomáš full_name: Brázdil, Tomáš last_name: Brázdil - first_name: Antonín full_name: Kučera, Antonín last_name: Kučera - first_name: Petr full_name: Novotny, Petr id: 3CC3B868-F248-11E8-B48F-1D18A9856A87 last_name: Novotny citation: ama: 'Brázdil T, Kučera A, Novotný P. Optimizing the expected mean payoff in Energy Markov Decision Processes. In: Vol 9938. Springer; 2016:32-49. doi:10.1007/978-3-319-46520-3_3' apa: 'Brázdil, T., Kučera, A., & Novotný, P. (2016). Optimizing the expected mean payoff in Energy Markov Decision Processes (Vol. 9938, pp. 32–49). Presented at the ATVA: Automated Technology for Verification and Analysis, Chiba, Japan: Springer. https://doi.org/10.1007/978-3-319-46520-3_3' chicago: Brázdil, Tomáš, Antonín Kučera, and Petr Novotný. “Optimizing the Expected Mean Payoff in Energy Markov Decision Processes,” 9938:32–49. Springer, 2016. https://doi.org/10.1007/978-3-319-46520-3_3. ieee: 'T. Brázdil, A. Kučera, and P. Novotný, “Optimizing the expected mean payoff in Energy Markov Decision Processes,” presented at the ATVA: Automated Technology for Verification and Analysis, Chiba, Japan, 2016, vol. 9938, pp. 32–49.' ista: 'Brázdil T, Kučera A, Novotný P. 2016. Optimizing the expected mean payoff in Energy Markov Decision Processes. ATVA: Automated Technology for Verification and Analysis, LNCS, vol. 9938, 32–49.' mla: Brázdil, Tomáš, et al. Optimizing the Expected Mean Payoff in Energy Markov Decision Processes. Vol. 9938, Springer, 2016, pp. 32–49, doi:10.1007/978-3-319-46520-3_3. short: T. Brázdil, A. Kučera, P. Novotný, in:, Springer, 2016, pp. 32–49. conference: end_date: 2016-10-20 location: Chiba, Japan name: 'ATVA: Automated Technology for Verification and Analysis' start_date: 2016-10-17 date_created: 2018-12-11T11:51:23Z date_published: 2016-09-22T00:00:00Z date_updated: 2021-01-12T06:49:53Z day: '22' department: - _id: KrCh doi: 10.1007/978-3-319-46520-3_3 ec_funded: 1 intvolume: ' 9938' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1607.00678 month: '09' oa: 1 oa_version: Preprint page: 32 - 49 project: - _id: 25681D80-B435-11E9-9278-68D0E5697425 call_identifier: FP7 grant_number: '291734' name: International IST Postdoc Fellowship Programme publication_status: published publisher: Springer publist_id: '5943' quality_controlled: '1' scopus_import: 1 status: public title: Optimizing the expected mean payoff in Energy Markov Decision Processes type: conference user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 volume: 9938 year: '2016' ... --- _id: '1333' abstract: - lang: eng text: Social dilemmas force players to balance between personal and collective gain. In many dilemmas, such as elected governments negotiating climate-change mitigation measures, the decisions are made not by individual players but by their representatives. However, the behaviour of representatives in social dilemmas has not been investigated experimentally. Here inspired by the negotiations for greenhouse-gas emissions reductions, we experimentally study a collective-risk social dilemma that involves representatives deciding on behalf of their fellow group members. Representatives can be re-elected or voted out after each consecutive collective-risk game. Selfish players are preferentially elected and are hence found most frequently in the "representatives" treatment. Across all treatments, we identify the selfish players as extortioners. As predicted by our mathematical model, their steadfast strategies enforce cooperation from fair players who finally compensate almost completely the deficit caused by the extortionate co-players. Everybody gains, but the extortionate representatives and their groups gain the most. acknowledgement: We thank the students for participation; H.-J. Krambeck for writing the software for the game; H. Arndt, T. Bakker, L. Becks, H. Brendelberger, S. Dobler and T. Reusch for support; and the Max Planck Society for the Advancement of Science for funding. article_number: '10915' author: - first_name: Manfred full_name: Milinski, Manfred last_name: Milinski - first_name: Christian full_name: Hilbe, Christian id: 2FDF8F3C-F248-11E8-B48F-1D18A9856A87 last_name: Hilbe orcid: 0000-0001-5116-955X - first_name: Dirk full_name: Semmann, Dirk last_name: Semmann - first_name: Ralf full_name: Sommerfeld, Ralf last_name: Sommerfeld - first_name: Jochem full_name: Marotzke, Jochem last_name: Marotzke citation: ama: Milinski M, Hilbe C, Semmann D, Sommerfeld R, Marotzke J. Humans choose representatives who enforce cooperation in social dilemmas through extortion. Nature Communications. 2016;7. doi:10.1038/ncomms10915 apa: Milinski, M., Hilbe, C., Semmann, D., Sommerfeld, R., & Marotzke, J. (2016). Humans choose representatives who enforce cooperation in social dilemmas through extortion. Nature Communications. Nature Publishing Group. https://doi.org/10.1038/ncomms10915 chicago: Milinski, Manfred, Christian Hilbe, Dirk Semmann, Ralf Sommerfeld, and Jochem Marotzke. “Humans Choose Representatives Who Enforce Cooperation in Social Dilemmas through Extortion.” Nature Communications. Nature Publishing Group, 2016. https://doi.org/10.1038/ncomms10915. ieee: M. Milinski, C. Hilbe, D. Semmann, R. Sommerfeld, and J. Marotzke, “Humans choose representatives who enforce cooperation in social dilemmas through extortion,” Nature Communications, vol. 7. Nature Publishing Group, 2016. ista: Milinski M, Hilbe C, Semmann D, Sommerfeld R, Marotzke J. 2016. Humans choose representatives who enforce cooperation in social dilemmas through extortion. Nature Communications. 7, 10915. mla: Milinski, Manfred, et al. “Humans Choose Representatives Who Enforce Cooperation in Social Dilemmas through Extortion.” Nature Communications, vol. 7, 10915, Nature Publishing Group, 2016, doi:10.1038/ncomms10915. short: M. Milinski, C. Hilbe, D. Semmann, R. Sommerfeld, J. Marotzke, Nature Communications 7 (2016). date_created: 2018-12-11T11:51:25Z date_published: 2016-03-07T00:00:00Z date_updated: 2021-01-12T06:49:57Z day: '07' ddc: - '519' - '530' - '599' department: - _id: KrCh doi: 10.1038/ncomms10915 file: - access_level: open_access checksum: 9ea0d7ce59a555a1cb8353d5559407cb content_type: application/pdf creator: system date_created: 2018-12-12T10:10:44Z date_updated: 2020-07-14T12:44:44Z file_id: '4834' file_name: IST-2016-661-v1+1_ncomms10915.pdf file_size: 1432577 relation: main_file file_date_updated: 2020-07-14T12:44:44Z has_accepted_license: '1' intvolume: ' 7' language: - iso: eng month: '03' oa: 1 oa_version: Published Version publication: Nature Communications publication_status: published publisher: Nature Publishing Group publist_id: '5935' pubrep_id: '661' quality_controlled: '1' scopus_import: 1 status: public title: Humans choose representatives who enforce cooperation in social dilemmas through extortion tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87 volume: 7 year: '2016' ...