Earlier Version

# Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives

K. Chatterjee, M. Henzinger, M. Joglekar, S. Nisarg, in:, G. Gopalakrishnan, S. Qadeer (Eds.), Springer, 2011, pp. 260–276.

Download (ext.)

*Conference Paper*|

*Published*|

*English*

Author

Editor

Gopalakrishnan, Ganesh
;
Qadeer, Shaz

Department

Series Title

LNCS

Abstract

We consider Markov decision processes (MDPs) with ω-regular specifications given as parity objectives. We consider the problem of computing the set of almost-sure winning states from where the objective can be ensured with probability 1. The algorithms for the computation of the almost-sure winning set for parity objectives iteratively use the solutions for the almost-sure winning set for Büchi objectives (a special case of parity objectives). Our contributions are as follows: First, we present the first subquadratic symbolic algorithm to compute the almost-sure winning set for MDPs with Büchi objectives; our algorithm takes O(nm) symbolic steps as compared to the previous known algorithm that takes O(n 2) symbolic steps, where n is the number of states and m is the number of edges of the MDP. In practice MDPs often have constant out-degree, and then our symbolic algorithm takes O(nn) symbolic steps, as compared to the previous known O(n 2) symbolic steps algorithm. Second, we present a new algorithm, namely win-lose algorithm, with the following two properties: (a) the algorithm iteratively computes subsets of the almost-sure winning set and its complement, as compared to all previous algorithms that discover the almost-sure winning set upon termination; and (b) requires O(nK) symbolic steps, where K is the maximal number of edges of strongly connected components (scc’s) of the MDP. The win-lose algorithm requires symbolic computation of scc’s. Third, we improve the algorithm for symbolic scc computation; the previous known algorithm takes linear symbolic steps, and our new algorithm improves the constants associated with the linear number of steps. In the worst case the previous known algorithm takes 5·n symbolic steps, whereas our new algorithm takes 4 ·n symbolic steps.

Publishing Year

Date Published

2011-08-11

Volume

6806

Page

260 - 276

Conference

CAV: Computer Aided Verification

Conference Location

Snowbird, USA

Conference Date

2011-07-14 – 2011-07-20

IST-REx-ID

### Cite this

Chatterjee K, Henzinger M, Joglekar M, Nisarg S. Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives. In: Gopalakrishnan G, Qadeer S, eds. Vol 6806. Springer; 2011:260-276. doi:10.1007/978-3-642-22110-1_21

Chatterjee, K., Henzinger, M., Joglekar, M., & Nisarg, S. (2011). Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives. In G. Gopalakrishnan & S. Qadeer (Eds.) (Vol. 6806, pp. 260–276). Presented at the CAV: Computer Aided Verification, Snowbird, USA: Springer. https://doi.org/10.1007/978-3-642-22110-1_21

Chatterjee, Krishnendu, Monika Henzinger, Manas Joglekar, and Shah Nisarg. “Symbolic Algorithms for Qualitative Analysis of Markov Decision Processes with Büchi Objectives.” edited by Ganesh Gopalakrishnan and Shaz Qadeer, 6806:260–76. Springer, 2011. https://doi.org/10.1007/978-3-642-22110-1_21.

K. Chatterjee, M. Henzinger, M. Joglekar, and S. Nisarg, “Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives,” presented at the CAV: Computer Aided Verification, Snowbird, USA, 2011, vol. 6806, pp. 260–276.

Chatterjee K, Henzinger M, Joglekar M, Nisarg S. 2011. Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives. CAV: Computer Aided Verification, LNCS, vol. 6806. 260–276.

Chatterjee, Krishnendu, et al.

*Symbolic Algorithms for Qualitative Analysis of Markov Decision Processes with Büchi Objectives*. Edited by Ganesh Gopalakrishnan and Shaz Qadeer, vol. 6806, Springer, 2011, pp. 260–76, doi:10.1007/978-3-642-22110-1_21.**Material in IST:**

**Later Version**

### Export

Marked PublicationsOpen Data IST Research Explorer

### Sources

arXiv 1104.3348