---
_id: '1325'
abstract:
- lang: eng
text: We study graphs and two-player games in which rewards are assigned to states,
and the goal of the players is to satisfy or dissatisfy certain property of the
generated outcome, given as a mean payoff property. Since the notion of mean-payoff
does not reflect possible fluctuations from the mean-payoff along a run, we propose
definitions and algorithms for capturing the stability of the system, and give
algorithms for deciding if a given mean payoff and stability objective can be
ensured in the system.
acknowledgement: "The work has been supported by the Czech Science Foundation, grant
No. 15-17564S, by EPSRC grant\r\nEP/M023656/1, and by the People Programme (Marie
Curie Actions) of the European Union’s Seventh\r\nFramework Programme (FP7/2007-2013)
under REA grant agreement no [291734]"
alternative_title:
- LIPIcs
article_number: '10'
author:
- first_name: Tomáš
full_name: Brázdil, Tomáš
last_name: Brázdil
- first_name: Vojtěch
full_name: Forejt, Vojtěch
last_name: Forejt
- first_name: Antonín
full_name: Kučera, Antonín
last_name: Kučera
- first_name: Petr
full_name: Novotny, Petr
id: 3CC3B868-F248-11E8-B48F-1D18A9856A87
last_name: Novotny
citation:
ama: 'Brázdil T, Forejt V, Kučera A, Novotný P. Stability in graphs and games. In:
Vol 59. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2016. doi:10.4230/LIPIcs.CONCUR.2016.10'
apa: 'Brázdil, T., Forejt, V., Kučera, A., & Novotný, P. (2016). Stability in
graphs and games (Vol. 59). Presented at the CONCUR: Concurrency Theory, Quebec
City, Canada: Schloss Dagstuhl - Leibniz-Zentrum für Informatik. https://doi.org/10.4230/LIPIcs.CONCUR.2016.10'
chicago: Brázdil, Tomáš, Vojtěch Forejt, Antonín Kučera, and Petr Novotný. “Stability
in Graphs and Games,” Vol. 59. Schloss Dagstuhl - Leibniz-Zentrum für Informatik,
2016. https://doi.org/10.4230/LIPIcs.CONCUR.2016.10.
ieee: 'T. Brázdil, V. Forejt, A. Kučera, and P. Novotný, “Stability in graphs and
games,” presented at the CONCUR: Concurrency Theory, Quebec City, Canada, 2016,
vol. 59.'
ista: 'Brázdil T, Forejt V, Kučera A, Novotný P. 2016. Stability in graphs and games.
CONCUR: Concurrency Theory, LIPIcs, vol. 59, 10.'
mla: Brázdil, Tomáš, et al. Stability in Graphs and Games. Vol. 59, 10, Schloss
Dagstuhl - Leibniz-Zentrum für Informatik, 2016, doi:10.4230/LIPIcs.CONCUR.2016.10.
short: T. Brázdil, V. Forejt, A. Kučera, P. Novotný, in:, Schloss Dagstuhl - Leibniz-Zentrum
für Informatik, 2016.
conference:
end_date: 2016-08-26
location: Quebec City, Canada
name: 'CONCUR: Concurrency Theory'
start_date: 2016-08-23
date_created: 2018-12-11T11:51:23Z
date_published: 2016-08-01T00:00:00Z
date_updated: 2021-01-12T06:49:53Z
day: '01'
ddc:
- '004'
department:
- _id: KrCh
doi: 10.4230/LIPIcs.CONCUR.2016.10
ec_funded: 1
file:
- access_level: open_access
checksum: 3c2dc6ab0358f8aa8f7aa7d6c1293159
content_type: application/pdf
creator: system
date_created: 2018-12-12T10:16:40Z
date_updated: 2020-07-14T12:44:44Z
file_id: '5229'
file_name: IST-2016-665-v1+1_Forejt_et_al__Stability_in_graphs_and_games.pdf
file_size: 553648
relation: main_file
file_date_updated: 2020-07-14T12:44:44Z
has_accepted_license: '1'
intvolume: ' 59'
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
month: '08'
oa: 1
oa_version: Published Version
project:
- _id: 25681D80-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '291734'
name: International IST Postdoc Fellowship Programme
publication_status: published
publisher: Schloss Dagstuhl - Leibniz-Zentrum für Informatik
publist_id: '5944'
pubrep_id: '665'
quality_controlled: '1'
scopus_import: 1
status: public
title: Stability in graphs and games
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
volume: 59
year: '2016'
...
---
_id: '1324'
abstract:
- lang: eng
text: 'DEC-POMDPs extend POMDPs to a multi-agent setting, where several agents operate
in an uncertain environment independently to achieve a joint objective. DEC-POMDPs
have been studied with finite-horizon and infinite-horizon discounted-sum objectives,
and there exist solvers both for exact and approximate solutions. In this work
we consider Goal-DEC-POMDPs, where given a set of target states, the objective
is to ensure that the target set is reached with minimal cost. We consider the
indefinite-horizon (infinite-horizon with either discounted-sum, or undiscounted-sum,
where absorbing goal states have zero-cost) problem. We present a new and novel
method to solve the problem that extends methods for finite-horizon DEC-POMDPs
and the RTDP-Bel approach for POMDPs. We present experimental results on several
examples, and show that our approach presents promising results. Copyright '
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Martin
full_name: Chmelik, Martin
id: 3624234E-F248-11E8-B48F-1D18A9856A87
last_name: Chmelik
citation:
ama: 'Chatterjee K, Chmelik M. Indefinite-horizon reachability in Goal-DEC-POMDPs.
In: Proceedings of the Twenty-Sixth International Conference on International
Conference on Automated Planning and Scheduling. Vol 2016-January. AAAI Press;
2016:88-96.'
apa: 'Chatterjee, K., & Chmelik, M. (2016). Indefinite-horizon reachability
in Goal-DEC-POMDPs. In Proceedings of the Twenty-Sixth International Conference
on International Conference on Automated Planning and Scheduling (Vol. 2016–January,
pp. 88–96). London, United Kingdom: AAAI Press.'
chicago: Chatterjee, Krishnendu, and Martin Chmelik. “Indefinite-Horizon Reachability
in Goal-DEC-POMDPs.” In Proceedings of the Twenty-Sixth International Conference
on International Conference on Automated Planning and Scheduling, 2016–January:88–96.
AAAI Press, 2016.
ieee: K. Chatterjee and M. Chmelik, “Indefinite-horizon reachability in Goal-DEC-POMDPs,”
in Proceedings of the Twenty-Sixth International Conference on International
Conference on Automated Planning and Scheduling, London, United Kingdom, 2016,
vol. 2016–January, pp. 88–96.
ista: 'Chatterjee K, Chmelik M. 2016. Indefinite-horizon reachability in Goal-DEC-POMDPs.
Proceedings of the Twenty-Sixth International Conference on International Conference
on Automated Planning and Scheduling. ICAPS: International Conference on Automated
Planning and Scheduling vol. 2016–January, 88–96.'
mla: Chatterjee, Krishnendu, and Martin Chmelik. “Indefinite-Horizon Reachability
in Goal-DEC-POMDPs.” Proceedings of the Twenty-Sixth International Conference
on International Conference on Automated Planning and Scheduling, vol. 2016–January,
AAAI Press, 2016, pp. 88–96.
short: K. Chatterjee, M. Chmelik, in:, Proceedings of the Twenty-Sixth International
Conference on International Conference on Automated Planning and Scheduling, AAAI
Press, 2016, pp. 88–96.
conference:
end_date: 2016-06-17
location: London, United Kingdom
name: 'ICAPS: International Conference on Automated Planning and Scheduling'
start_date: 2016-06-12
date_created: 2018-12-11T11:51:22Z
date_published: 2016-01-01T00:00:00Z
date_updated: 2021-01-12T06:49:53Z
day: '01'
department:
- _id: KrCh
ec_funded: 1
language:
- iso: eng
main_file_link:
- url: http://www.aaai.org/ocs/index.php/ICAPS/ICAPS16/paper/view/12999
month: '01'
oa_version: None
page: 88 - 96
project:
- _id: 2584A770-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: P 23499-N23
name: Modern Graph Algorithmic Techniques in Formal Verification
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
- _id: 25832EC2-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S 11407_N23
name: Rigorous Systems Engineering
publication: Proceedings of the Twenty-Sixth International Conference on International
Conference on Automated Planning and Scheduling
publication_status: published
publisher: AAAI Press
publist_id: '5946'
quality_controlled: '1'
scopus_import: 1
status: public
title: Indefinite-horizon reachability in Goal-DEC-POMDPs
type: conference
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
volume: 2016-January
year: '2016'
...
---
_id: '1327'
abstract:
- lang: eng
text: 'We consider partially observable Markov decision processes (POMDPs) with
a set of target states and positive integer costs associated with every transition.
The traditional optimization objective (stochastic shortest path) asks to minimize
the expected total cost until the target set is reached. We extend the traditional
framework of POMDPs to model energy consumption, which represents a hard constraint.
The energy levels may increase and decrease with transitions, and the hard constraint
requires that the energy level must remain positive in all steps till the target
is reached. First, we present a novel algorithm for solving POMDPs with energy
levels, developing on existing POMDP solvers and using RTDP as its main method.
Our second contribution is related to policy representation. For larger POMDP
instances the policies computed by existing solvers are too large to be understandable.
We present an automated procedure based on machine learning techniques that automatically
extracts important decisions of the policy allowing us to compute succinct human
readable policies. Finally, we show experimentally that our algorithm performs
well and computes succinct policies on a number of POMDP instances from the literature
that were naturally enhanced with energy levels. '
author:
- first_name: Tomáš
full_name: Brázdil, Tomáš
last_name: Brázdil
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Martin
full_name: Chmelik, Martin
id: 3624234E-F248-11E8-B48F-1D18A9856A87
last_name: Chmelik
- first_name: Anchit
full_name: Gupta, Anchit
last_name: Gupta
- first_name: Petr
full_name: Novotny, Petr
id: 3CC3B868-F248-11E8-B48F-1D18A9856A87
last_name: Novotny
citation:
ama: 'Brázdil T, Chatterjee K, Chmelik M, Gupta A, Novotný P. Stochastic shortest
path with energy constraints in POMDPs. In: Proceedings of the 15th International
Conference on Autonomous Agents and Multiagent Systems. ACM; 2016:1465-1466.'
apa: 'Brázdil, T., Chatterjee, K., Chmelik, M., Gupta, A., & Novotný, P. (2016).
Stochastic shortest path with energy constraints in POMDPs. In Proceedings
of the 15th International Conference on Autonomous Agents and Multiagent Systems
(pp. 1465–1466). Singapore: ACM.'
chicago: Brázdil, Tomáš, Krishnendu Chatterjee, Martin Chmelik, Anchit Gupta, and
Petr Novotný. “Stochastic Shortest Path with Energy Constraints in POMDPs.” In
Proceedings of the 15th International Conference on Autonomous Agents and Multiagent
Systems, 1465–66. ACM, 2016.
ieee: T. Brázdil, K. Chatterjee, M. Chmelik, A. Gupta, and P. Novotný, “Stochastic
shortest path with energy constraints in POMDPs,” in Proceedings of the 15th
International Conference on Autonomous Agents and Multiagent Systems, Singapore,
2016, pp. 1465–1466.
ista: 'Brázdil T, Chatterjee K, Chmelik M, Gupta A, Novotný P. 2016. Stochastic
shortest path with energy constraints in POMDPs. Proceedings of the 15th International
Conference on Autonomous Agents and Multiagent Systems. AAMAS: Autonomous Agents
& Multiagent Systems, 1465–1466.'
mla: Brázdil, Tomáš, et al. “Stochastic Shortest Path with Energy Constraints in
POMDPs.” Proceedings of the 15th International Conference on Autonomous Agents
and Multiagent Systems, ACM, 2016, pp. 1465–66.
short: T. Brázdil, K. Chatterjee, M. Chmelik, A. Gupta, P. Novotný, in:, Proceedings
of the 15th International Conference on Autonomous Agents and Multiagent Systems,
ACM, 2016, pp. 1465–1466.
conference:
end_date: 2016-05-13
location: Singapore
name: 'AAMAS: Autonomous Agents & Multiagent Systems'
start_date: 2016-05-09
date_created: 2018-12-11T11:51:23Z
date_published: 2016-01-01T00:00:00Z
date_updated: 2021-01-12T06:49:54Z
day: '01'
department:
- _id: KrCh
ec_funded: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1602.07565
month: '01'
oa: 1
oa_version: Preprint
page: 1465 - 1466
project:
- _id: 2584A770-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: P 23499-N23
name: Modern Graph Algorithmic Techniques in Formal Verification
- _id: 25832EC2-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S 11407_N23
name: Rigorous Systems Engineering
- _id: 25681D80-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '291734'
name: International IST Postdoc Fellowship Programme
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
publication: Proceedings of the 15th International Conference on Autonomous Agents
and Multiagent Systems
publication_status: published
publisher: ACM
publist_id: '5942'
quality_controlled: '1'
scopus_import: 1
status: public
title: Stochastic shortest path with energy constraints in POMDPs
type: conference
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
year: '2016'
...
---
_id: '1326'
abstract:
- lang: eng
text: 'Energy Markov Decision Processes (EMDPs) are finite-state Markov decision
processes where each transition is assigned an integer counter update and a rational
payoff. An EMDP configuration is a pair s(n), where s is a control state and n
is the current counter value. The configurations are changed by performing transitions
in the standard way. We consider the problem of computing a safe strategy (i.e.,
a strategy that keeps the counter non-negative) which maximizes the expected mean
payoff. '
acknowledgement: The research was funded by the Czech Science Foundation Grant No.
P202/12/G061 and by the People Programme (Marie Curie Actions) of the European Union’s
Seventh Framework Programme (FP7/2007-2013) under REA grant agreement no [291734].
alternative_title:
- LNCS
author:
- first_name: Tomáš
full_name: Brázdil, Tomáš
last_name: Brázdil
- first_name: Antonín
full_name: Kučera, Antonín
last_name: Kučera
- first_name: Petr
full_name: Novotny, Petr
id: 3CC3B868-F248-11E8-B48F-1D18A9856A87
last_name: Novotny
citation:
ama: 'Brázdil T, Kučera A, Novotný P. Optimizing the expected mean payoff in Energy
Markov Decision Processes. In: Vol 9938. Springer; 2016:32-49. doi:10.1007/978-3-319-46520-3_3'
apa: 'Brázdil, T., Kučera, A., & Novotný, P. (2016). Optimizing the expected
mean payoff in Energy Markov Decision Processes (Vol. 9938, pp. 32–49). Presented
at the ATVA: Automated Technology for Verification and Analysis, Chiba, Japan:
Springer. https://doi.org/10.1007/978-3-319-46520-3_3'
chicago: Brázdil, Tomáš, Antonín Kučera, and Petr Novotný. “Optimizing the Expected
Mean Payoff in Energy Markov Decision Processes,” 9938:32–49. Springer, 2016.
https://doi.org/10.1007/978-3-319-46520-3_3.
ieee: 'T. Brázdil, A. Kučera, and P. Novotný, “Optimizing the expected mean payoff
in Energy Markov Decision Processes,” presented at the ATVA: Automated Technology
for Verification and Analysis, Chiba, Japan, 2016, vol. 9938, pp. 32–49.'
ista: 'Brázdil T, Kučera A, Novotný P. 2016. Optimizing the expected mean payoff
in Energy Markov Decision Processes. ATVA: Automated Technology for Verification
and Analysis, LNCS, vol. 9938, 32–49.'
mla: Brázdil, Tomáš, et al. Optimizing the Expected Mean Payoff in Energy Markov
Decision Processes. Vol. 9938, Springer, 2016, pp. 32–49, doi:10.1007/978-3-319-46520-3_3.
short: T. Brázdil, A. Kučera, P. Novotný, in:, Springer, 2016, pp. 32–49.
conference:
end_date: 2016-10-20
location: Chiba, Japan
name: 'ATVA: Automated Technology for Verification and Analysis'
start_date: 2016-10-17
date_created: 2018-12-11T11:51:23Z
date_published: 2016-09-22T00:00:00Z
date_updated: 2021-01-12T06:49:53Z
day: '22'
department:
- _id: KrCh
doi: 10.1007/978-3-319-46520-3_3
ec_funded: 1
intvolume: ' 9938'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1607.00678
month: '09'
oa: 1
oa_version: Preprint
page: 32 - 49
project:
- _id: 25681D80-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '291734'
name: International IST Postdoc Fellowship Programme
publication_status: published
publisher: Springer
publist_id: '5943'
quality_controlled: '1'
scopus_import: 1
status: public
title: Optimizing the expected mean payoff in Energy Markov Decision Processes
type: conference
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
volume: 9938
year: '2016'
...
---
_id: '1333'
abstract:
- lang: eng
text: Social dilemmas force players to balance between personal and collective gain.
In many dilemmas, such as elected governments negotiating climate-change mitigation
measures, the decisions are made not by individual players but by their representatives.
However, the behaviour of representatives in social dilemmas has not been investigated
experimentally. Here inspired by the negotiations for greenhouse-gas emissions
reductions, we experimentally study a collective-risk social dilemma that involves
representatives deciding on behalf of their fellow group members. Representatives
can be re-elected or voted out after each consecutive collective-risk game. Selfish
players are preferentially elected and are hence found most frequently in the
"representatives" treatment. Across all treatments, we identify the
selfish players as extortioners. As predicted by our mathematical model, their
steadfast strategies enforce cooperation from fair players who finally compensate
almost completely the deficit caused by the extortionate co-players. Everybody
gains, but the extortionate representatives and their groups gain the most.
acknowledgement: We thank the students for participation; H.-J. Krambeck for writing
the software for the game; H. Arndt, T. Bakker, L. Becks, H. Brendelberger, S. Dobler
and T. Reusch for support; and the Max Planck Society for the Advancement of Science
for funding.
article_number: '10915'
author:
- first_name: Manfred
full_name: Milinski, Manfred
last_name: Milinski
- first_name: Christian
full_name: Hilbe, Christian
id: 2FDF8F3C-F248-11E8-B48F-1D18A9856A87
last_name: Hilbe
orcid: 0000-0001-5116-955X
- first_name: Dirk
full_name: Semmann, Dirk
last_name: Semmann
- first_name: Ralf
full_name: Sommerfeld, Ralf
last_name: Sommerfeld
- first_name: Jochem
full_name: Marotzke, Jochem
last_name: Marotzke
citation:
ama: Milinski M, Hilbe C, Semmann D, Sommerfeld R, Marotzke J. Humans choose representatives
who enforce cooperation in social dilemmas through extortion. Nature Communications.
2016;7. doi:10.1038/ncomms10915
apa: Milinski, M., Hilbe, C., Semmann, D., Sommerfeld, R., & Marotzke, J. (2016).
Humans choose representatives who enforce cooperation in social dilemmas through
extortion. Nature Communications. Nature Publishing Group. https://doi.org/10.1038/ncomms10915
chicago: Milinski, Manfred, Christian Hilbe, Dirk Semmann, Ralf Sommerfeld, and
Jochem Marotzke. “Humans Choose Representatives Who Enforce Cooperation in Social
Dilemmas through Extortion.” Nature Communications. Nature Publishing Group,
2016. https://doi.org/10.1038/ncomms10915.
ieee: M. Milinski, C. Hilbe, D. Semmann, R. Sommerfeld, and J. Marotzke, “Humans
choose representatives who enforce cooperation in social dilemmas through extortion,”
Nature Communications, vol. 7. Nature Publishing Group, 2016.
ista: Milinski M, Hilbe C, Semmann D, Sommerfeld R, Marotzke J. 2016. Humans choose
representatives who enforce cooperation in social dilemmas through extortion.
Nature Communications. 7, 10915.
mla: Milinski, Manfred, et al. “Humans Choose Representatives Who Enforce Cooperation
in Social Dilemmas through Extortion.” Nature Communications, vol. 7, 10915,
Nature Publishing Group, 2016, doi:10.1038/ncomms10915.
short: M. Milinski, C. Hilbe, D. Semmann, R. Sommerfeld, J. Marotzke, Nature Communications
7 (2016).
date_created: 2018-12-11T11:51:25Z
date_published: 2016-03-07T00:00:00Z
date_updated: 2021-01-12T06:49:57Z
day: '07'
ddc:
- '519'
- '530'
- '599'
department:
- _id: KrCh
doi: 10.1038/ncomms10915
file:
- access_level: open_access
checksum: 9ea0d7ce59a555a1cb8353d5559407cb
content_type: application/pdf
creator: system
date_created: 2018-12-12T10:10:44Z
date_updated: 2020-07-14T12:44:44Z
file_id: '4834'
file_name: IST-2016-661-v1+1_ncomms10915.pdf
file_size: 1432577
relation: main_file
file_date_updated: 2020-07-14T12:44:44Z
has_accepted_license: '1'
intvolume: ' 7'
language:
- iso: eng
month: '03'
oa: 1
oa_version: Published Version
publication: Nature Communications
publication_status: published
publisher: Nature Publishing Group
publist_id: '5935'
pubrep_id: '661'
quality_controlled: '1'
scopus_import: 1
status: public
title: Humans choose representatives who enforce cooperation in social dilemmas through
extortion
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
volume: 7
year: '2016'
...