---
_id: '25'
abstract:
- lang: eng
text: 'Partially observable Markov decision processes (POMDPs) are the standard
models for planning under uncertainty with both finite and infinite horizon. Besides
the well-known discounted-sum objective, indefinite-horizon objective (aka Goal-POMDPs)
is another classical objective for POMDPs. In this case, given a set of target
states and a positive cost for each transition, the optimization objective is
to minimize the expected total cost until a target state is reached. In the literature,
RTDP-Bel or heuristic search value iteration (HSVI) have been used for solving
Goal-POMDPs. Neither of these algorithms has theoretical convergence guarantees,
and HSVI may even fail to terminate its trials. We give the following contributions:
(1) We discuss the challenges introduced in Goal-POMDPs and illustrate how they
prevent the original HSVI from converging. (2) We present a novel algorithm inspired
by HSVI, termed Goal-HSVI, and show that our algorithm has convergence guarantees.
(3) We show that Goal-HSVI outperforms RTDP-Bel on a set of well-known examples.'
acknowledgement: '∗This work has been supported by Vienna Science and Technology Fund
(WWTF) Project ICT15-003, Austrian Science Fund (FWF) NFN Grant No S11407-N23 (RiSE/SHiNE),
and ERC Starting grant (279307: Graph Games). This research was sponsored by the
Army Research Laboratory and was accomplished under Cooperative Agreement Number
W911NF-13-2-0045 (ARL Cyber Security CRA). '
article_processing_charge: No
author:
- first_name: Karel
full_name: Horák, Karel
last_name: Horák
- first_name: Branislav
full_name: Bošanský, Branislav
last_name: Bošanský
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
citation:
ama: 'Horák K, Bošanský B, Chatterjee K. Goal-HSVI: Heuristic search value iteration
for goal-POMDPs. In: Proceedings of the Twenty-Seventh International Joint
Conference on Artificial Intelligence. Vol 2018-July. IJCAI; 2018:4764-4770.
doi:10.24963/ijcai.2018/662'
apa: 'Horák, K., Bošanský, B., & Chatterjee, K. (2018). Goal-HSVI: Heuristic
search value iteration for goal-POMDPs. In Proceedings of the Twenty-Seventh
International Joint Conference on Artificial Intelligence (Vol. 2018–July,
pp. 4764–4770). Stockholm, Sweden: IJCAI. https://doi.org/10.24963/ijcai.2018/662'
chicago: 'Horák, Karel, Branislav Bošanský, and Krishnendu Chatterjee. “Goal-HSVI:
Heuristic Search Value Iteration for Goal-POMDPs.” In Proceedings of the Twenty-Seventh
International Joint Conference on Artificial Intelligence, 2018–July:4764–70.
IJCAI, 2018. https://doi.org/10.24963/ijcai.2018/662.'
ieee: 'K. Horák, B. Bošanský, and K. Chatterjee, “Goal-HSVI: Heuristic search value
iteration for goal-POMDPs,” in Proceedings of the Twenty-Seventh International
Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018, vol.
2018–July, pp. 4764–4770.'
ista: 'Horák K, Bošanský B, Chatterjee K. 2018. Goal-HSVI: Heuristic search value
iteration for goal-POMDPs. Proceedings of the Twenty-Seventh International Joint
Conference on Artificial Intelligence. IJCAI: International Joint Conference on
Artificial Intelligence vol. 2018–July, 4764–4770.'
mla: 'Horák, Karel, et al. “Goal-HSVI: Heuristic Search Value Iteration for Goal-POMDPs.”
Proceedings of the Twenty-Seventh International Joint Conference on Artificial
Intelligence, vol. 2018–July, IJCAI, 2018, pp. 4764–70, doi:10.24963/ijcai.2018/662.'
short: K. Horák, B. Bošanský, K. Chatterjee, in:, Proceedings of the Twenty-Seventh
International Joint Conference on Artificial Intelligence, IJCAI, 2018, pp. 4764–4770.
conference:
end_date: 2018-07-19
location: Stockholm, Sweden
name: 'IJCAI: International Joint Conference on Artificial Intelligence'
start_date: 2018-07-13
date_created: 2018-12-11T11:44:13Z
date_published: 2018-07-01T00:00:00Z
date_updated: 2023-09-19T14:44:59Z
day: '01'
department:
- _id: KrCh
doi: 10.24963/ijcai.2018/662
ec_funded: 1
external_id:
isi:
- '000764175404127'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.24963/ijcai.2018/662
month: '07'
oa: 1
oa_version: Published Version
page: 4764 - 4770
project:
- _id: 25892FC0-B435-11E9-9278-68D0E5697425
grant_number: ICT15-003
name: Efficient Algorithms for Computer Aided Verification
- _id: 25832EC2-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S 11407_N23
name: Rigorous Systems Engineering
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
publication: Proceedings of the Twenty-Seventh International Joint Conference on Artificial
Intelligence
publication_status: published
publisher: IJCAI
publist_id: '8030'
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'Goal-HSVI: Heuristic search value iteration for goal-POMDPs'
type: conference
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 2018-July
year: '2018'
...
---
_id: '24'
abstract:
- lang: eng
text: Partially-observable Markov decision processes (POMDPs) with discounted-sum
payoff are a standard framework to model a wide range of problems related to decision
making under uncertainty. Traditionally, the goal has been to obtain policies
that optimize the expectation of the discounted-sum payoff. A key drawback of
the expectation measure is that even low probability events with extreme payoff
can significantly affect the expectation, and thus the obtained policies are not
necessarily risk-averse. An alternate approach is to optimize the probability
that the payoff is above a certain threshold, which allows obtaining risk-averse
policies, but ignores optimization of the expectation. We consider the expectation
optimization with probabilistic guarantee (EOPG) problem, where the goal is to
optimize the expectation ensuring that the payoff is above a given threshold with
at least a specified probability. We present several results on the EOPG problem,
including the first algorithm to solve it.
acknowledgement: "This research was supported by the Vienna Science and Technology
Fund (WWTF) grant ICT15-003; Austrian Science Fund (FWF): S11407-N23(RiSE/SHiNE);and
an ERC Start Grant (279307:Graph Games).\r\n"
article_processing_charge: No
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Adrian
full_name: Elgyütt, Adrian
id: 4A2E9DBA-F248-11E8-B48F-1D18A9856A87
last_name: Elgyütt
- first_name: Petr
full_name: Novotny, Petr
id: 3CC3B868-F248-11E8-B48F-1D18A9856A87
last_name: Novotny
- first_name: Owen
full_name: Rouillé, Owen
last_name: Rouillé
citation:
ama: 'Chatterjee K, Elgyütt A, Novotný P, Rouillé O. Expectation optimization with
probabilistic guarantees in POMDPs with discounted-sum objectives. In: Vol 2018.
IJCAI; 2018:4692-4699. doi:10.24963/ijcai.2018/652'
apa: 'Chatterjee, K., Elgyütt, A., Novotný, P., & Rouillé, O. (2018). Expectation
optimization with probabilistic guarantees in POMDPs with discounted-sum objectives
(Vol. 2018, pp. 4692–4699). Presented at the IJCAI: International Joint Conference
on Artificial Intelligence, Stockholm, Sweden: IJCAI. https://doi.org/10.24963/ijcai.2018/652'
chicago: Chatterjee, Krishnendu, Adrian Elgyütt, Petr Novotný, and Owen Rouillé.
“Expectation Optimization with Probabilistic Guarantees in POMDPs with Discounted-Sum
Objectives,” 2018:4692–99. IJCAI, 2018. https://doi.org/10.24963/ijcai.2018/652.
ieee: 'K. Chatterjee, A. Elgyütt, P. Novotný, and O. Rouillé, “Expectation optimization
with probabilistic guarantees in POMDPs with discounted-sum objectives,” presented
at the IJCAI: International Joint Conference on Artificial Intelligence, Stockholm,
Sweden, 2018, vol. 2018, pp. 4692–4699.'
ista: 'Chatterjee K, Elgyütt A, Novotný P, Rouillé O. 2018. Expectation optimization
with probabilistic guarantees in POMDPs with discounted-sum objectives. IJCAI:
International Joint Conference on Artificial Intelligence vol. 2018, 4692–4699.'
mla: Chatterjee, Krishnendu, et al. Expectation Optimization with Probabilistic
Guarantees in POMDPs with Discounted-Sum Objectives. Vol. 2018, IJCAI, 2018,
pp. 4692–99, doi:10.24963/ijcai.2018/652.
short: K. Chatterjee, A. Elgyütt, P. Novotný, O. Rouillé, in:, IJCAI, 2018, pp.
4692–4699.
conference:
end_date: 2018-07-19
location: Stockholm, Sweden
name: 'IJCAI: International Joint Conference on Artificial Intelligence'
start_date: 2018-07-13
date_created: 2018-12-11T11:44:13Z
date_published: 2018-07-01T00:00:00Z
date_updated: 2023-09-19T14:45:48Z
day: '01'
department:
- _id: KrCh
- _id: ToHe
doi: 10.24963/ijcai.2018/652
ec_funded: 1
external_id:
arxiv:
- '1804.10601'
isi:
- '000764175404117'
intvolume: ' 2018'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1804.10601
month: '07'
oa: 1
oa_version: Preprint
page: 4692 - 4699
project:
- _id: 25892FC0-B435-11E9-9278-68D0E5697425
grant_number: ICT15-003
name: Efficient Algorithms for Computer Aided Verification
- _id: 25832EC2-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S 11407_N23
name: Rigorous Systems Engineering
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
publication_status: published
publisher: IJCAI
publist_id: '8031'
quality_controlled: '1'
scopus_import: '1'
status: public
title: Expectation optimization with probabilistic guarantees in POMDPs with discounted-sum
objectives
type: conference
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 2018
year: '2018'
...
---
_id: '34'
abstract:
- lang: eng
text: Partially observable Markov decision processes (POMDPs) are widely used in
probabilistic planning problems in which an agent interacts with an environment
using noisy and imprecise sensors. We study a setting in which the sensors are
only partially defined and the goal is to synthesize “weakest” additional sensors,
such that in the resulting POMDP, there is a small-memory policy for the agent
that almost-surely (with probability 1) satisfies a reachability objective. We
show that the problem is NP-complete, and present a symbolic algorithm by encoding
the problem into SAT instances. We illustrate trade-offs between the amount of
memory of the policy and the number of additional sensors on a simple example.
We have implemented our approach and consider three classical POMDP examples from
the literature, and show that in all the examples the number of sensors can be
significantly decreased (as compared to the existing solutions in the literature)
without increasing the complexity of the policies.
alternative_title:
- ICAPS
article_processing_charge: No
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Martin
full_name: Chemlík, Martin
last_name: Chemlík
- first_name: Ufuk
full_name: Topcu, Ufuk
last_name: Topcu
citation:
ama: 'Chatterjee K, Chemlík M, Topcu U. Sensor synthesis for POMDPs with reachability
objectives. In: Vol 2018. AAAI Press; 2018:47-55.'
apa: 'Chatterjee, K., Chemlík, M., & Topcu, U. (2018). Sensor synthesis for
POMDPs with reachability objectives (Vol. 2018, pp. 47–55). Presented at the ICAPS:
International Conference on Automated Planning and Scheduling, Delft, Netherlands:
AAAI Press.'
chicago: Chatterjee, Krishnendu, Martin Chemlík, and Ufuk Topcu. “Sensor Synthesis
for POMDPs with Reachability Objectives,” 2018:47–55. AAAI Press, 2018.
ieee: 'K. Chatterjee, M. Chemlík, and U. Topcu, “Sensor synthesis for POMDPs with
reachability objectives,” presented at the ICAPS: International Conference on
Automated Planning and Scheduling, Delft, Netherlands, 2018, vol. 2018, pp. 47–55.'
ista: 'Chatterjee K, Chemlík M, Topcu U. 2018. Sensor synthesis for POMDPs with
reachability objectives. ICAPS: International Conference on Automated Planning
and Scheduling, ICAPS, vol. 2018, 47–55.'
mla: Chatterjee, Krishnendu, et al. Sensor Synthesis for POMDPs with Reachability
Objectives. Vol. 2018, AAAI Press, 2018, pp. 47–55.
short: K. Chatterjee, M. Chemlík, U. Topcu, in:, AAAI Press, 2018, pp. 47–55.
conference:
end_date: 2018-06-29
location: Delft, Netherlands
name: 'ICAPS: International Conference on Automated Planning and Scheduling'
start_date: 2018-06-24
date_created: 2018-12-11T11:44:16Z
date_published: 2018-06-01T00:00:00Z
date_updated: 2023-09-19T14:44:14Z
day: '01'
department:
- _id: KrCh
ec_funded: 1
external_id:
arxiv:
- '1710.00675'
isi:
- '000492986200006'
intvolume: ' 2018'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1710.00675
month: '06'
oa: 1
oa_version: Preprint
page: 47 - 55
project:
- _id: 2584A770-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: P 23499-N23
name: Modern Graph Algorithmic Techniques in Formal Verification
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
- _id: 25832EC2-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S 11407_N23
name: Rigorous Systems Engineering
- _id: 2587B514-B435-11E9-9278-68D0E5697425
name: Microsoft Research Faculty Fellowship
publication_status: published
publisher: AAAI Press
publist_id: '8021'
quality_controlled: '1'
scopus_import: '1'
status: public
title: Sensor synthesis for POMDPs with reachability objectives
type: conference
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 2018
year: '2018'
...
---
_id: '18'
abstract:
- lang: eng
text: An N-superconcentrator is a directed, acyclic graph with N input nodes and
N output nodes such that every subset of the inputs and every subset of the outputs
of same cardinality can be connected by node-disjoint paths. It is known that
linear-size and bounded-degree superconcentrators exist. We prove the existence
of such superconcentrators with asymptotic density 25.3 (where the density is
the number of edges divided by N). The previously best known densities were 28
[12] and 27.4136 [17].
article_processing_charge: No
author:
- first_name: Vladimir
full_name: Kolmogorov, Vladimir
id: 3D50B0BA-F248-11E8-B48F-1D18A9856A87
last_name: Kolmogorov
- first_name: Michal
full_name: Rolinek, Michal
id: 3CB3BC06-F248-11E8-B48F-1D18A9856A87
last_name: Rolinek
citation:
ama: Kolmogorov V, Rolinek M. Superconcentrators of density 25.3. Ars Combinatoria.
2018;141(10):269-304.
apa: Kolmogorov, V., & Rolinek, M. (2018). Superconcentrators of density 25.3.
Ars Combinatoria. Charles Babbage Research Centre.
chicago: Kolmogorov, Vladimir, and Michal Rolinek. “Superconcentrators of Density
25.3.” Ars Combinatoria. Charles Babbage Research Centre, 2018.
ieee: V. Kolmogorov and M. Rolinek, “Superconcentrators of density 25.3,” Ars
Combinatoria, vol. 141, no. 10. Charles Babbage Research Centre, pp. 269–304,
2018.
ista: Kolmogorov V, Rolinek M. 2018. Superconcentrators of density 25.3. Ars Combinatoria.
141(10), 269–304.
mla: Kolmogorov, Vladimir, and Michal Rolinek. “Superconcentrators of Density 25.3.”
Ars Combinatoria, vol. 141, no. 10, Charles Babbage Research Centre, 2018,
pp. 269–304.
short: V. Kolmogorov, M. Rolinek, Ars Combinatoria 141 (2018) 269–304.
date_created: 2018-12-11T11:44:11Z
date_published: 2018-10-01T00:00:00Z
date_updated: 2023-09-19T14:46:18Z
day: '01'
department:
- _id: VlKo
external_id:
arxiv:
- '1405.7828'
isi:
- '000446809500022'
intvolume: ' 141'
isi: 1
issue: '10'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1405.7828
month: '10'
oa: 1
oa_version: Preprint
page: 269 - 304
publication: Ars Combinatoria
publication_identifier:
issn:
- 0381-7032
publication_status: published
publisher: Charles Babbage Research Centre
publist_id: '8037'
quality_controlled: '1'
scopus_import: '1'
status: public
title: Superconcentrators of density 25.3
type: journal_article
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 141
year: '2018'
...
---
_id: '6355'
abstract:
- lang: eng
text: We prove that any cyclic quadrilateral can be inscribed in any closed convex
C1-curve. The smoothness condition is not required if the quadrilateral is a
rectangle.
article_number: e7
article_processing_charge: No
author:
- first_name: Arseniy
full_name: Akopyan, Arseniy
id: 430D2C90-F248-11E8-B48F-1D18A9856A87
last_name: Akopyan
orcid: 0000-0002-2548-617X
- first_name: Sergey
full_name: Avvakumov, Sergey
id: 3827DAC8-F248-11E8-B48F-1D18A9856A87
last_name: Avvakumov
citation:
ama: Akopyan A, Avvakumov S. Any cyclic quadrilateral can be inscribed in any closed
convex smooth curve. Forum of Mathematics, Sigma. 2018;6. doi:10.1017/fms.2018.7
apa: Akopyan, A., & Avvakumov, S. (2018). Any cyclic quadrilateral can be inscribed
in any closed convex smooth curve. Forum of Mathematics, Sigma. Cambridge
University Press. https://doi.org/10.1017/fms.2018.7
chicago: Akopyan, Arseniy, and Sergey Avvakumov. “Any Cyclic Quadrilateral Can Be
Inscribed in Any Closed Convex Smooth Curve.” Forum of Mathematics, Sigma.
Cambridge University Press, 2018. https://doi.org/10.1017/fms.2018.7.
ieee: A. Akopyan and S. Avvakumov, “Any cyclic quadrilateral can be inscribed in
any closed convex smooth curve,” Forum of Mathematics, Sigma, vol. 6. Cambridge
University Press, 2018.
ista: Akopyan A, Avvakumov S. 2018. Any cyclic quadrilateral can be inscribed in
any closed convex smooth curve. Forum of Mathematics, Sigma. 6, e7.
mla: Akopyan, Arseniy, and Sergey Avvakumov. “Any Cyclic Quadrilateral Can Be Inscribed
in Any Closed Convex Smooth Curve.” Forum of Mathematics, Sigma, vol. 6,
e7, Cambridge University Press, 2018, doi:10.1017/fms.2018.7.
short: A. Akopyan, S. Avvakumov, Forum of Mathematics, Sigma 6 (2018).
date_created: 2019-04-30T06:09:57Z
date_published: 2018-05-31T00:00:00Z
date_updated: 2023-09-19T14:50:12Z
day: '31'
ddc:
- '510'
department:
- _id: UlWa
- _id: HeEd
- _id: JaMa
doi: 10.1017/fms.2018.7
ec_funded: 1
external_id:
arxiv:
- '1712.10205'
isi:
- '000433915500001'
file:
- access_level: open_access
checksum: 5a71b24ba712a3eb2e46165a38fbc30a
content_type: application/pdf
creator: dernst
date_created: 2019-04-30T06:14:58Z
date_updated: 2020-07-14T12:47:28Z
file_id: '6356'
file_name: 2018_ForumMahtematics_Akopyan.pdf
file_size: 249246
relation: main_file
file_date_updated: 2020-07-14T12:47:28Z
has_accepted_license: '1'
intvolume: ' 6'
isi: 1
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
month: '05'
oa: 1
oa_version: Published Version
project:
- _id: 256E75B8-B435-11E9-9278-68D0E5697425
call_identifier: H2020
grant_number: '716117'
name: Optimal Transport and Stochastic Dynamics
publication: Forum of Mathematics, Sigma
publication_identifier:
issn:
- 2050-5094
publication_status: published
publisher: Cambridge University Press
quality_controlled: '1'
related_material:
record:
- id: '8156'
relation: dissertation_contains
status: public
status: public
title: Any cyclic quadrilateral can be inscribed in any closed convex smooth curve
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 6
year: '2018'
...