Scalable verification of quantized neural networks

Henzinger TA, Lechner M, Zikelic D. 2021. Scalable verification of quantized neural networks. Proceedings of the AAAI Conference on Artificial Intelligence. AAAI: Association for the Advancement of Artificial Intelligence, Technical Tracks, vol. 35, 3787–3795.

Download
OA 16496-Article Text-19990-1-2-20210518 (1).pdf 137.24 KB
Conference Paper | Published | English

Scopus indexed
Series Title
Technical Tracks
Abstract
Formal verification of neural networks is an active topic of research, and recent advances have significantly increased the size of the networks that verification tools can handle. However, most methods are designed for verification of an idealized model of the actual network which works over real arithmetic and ignores rounding imprecisions. This idealization is in stark contrast to network quantization, which is a technique that trades numerical precision for computational efficiency and is, therefore, often applied in practice. Neglecting rounding errors of such low-bit quantized neural networks has been shown to lead to wrong conclusions about the network’s correctness. Thus, the desired approach for verifying quantized neural networks would be one that takes these rounding errors into account. In this paper, we show that verifying the bitexact implementation of quantized neural networks with bitvector specifications is PSPACE-hard, even though verifying idealized real-valued networks and satisfiability of bit-vector specifications alone are each in NP. Furthermore, we explore several practical heuristics toward closing the complexity gap between idealized and bit-exact verification. In particular, we propose three techniques for making SMT-based verification of quantized neural networks more scalable. Our experiments demonstrate that our proposed methods allow a speedup of up to three orders of magnitude over existing approaches.
Publishing Year
Date Published
2021-05-28
Proceedings Title
Proceedings of the AAAI Conference on Artificial Intelligence
Acknowledgement
This research was supported in part by the Austrian Science Fund (FWF) under grant Z211-N23 (Wittgenstein Award), ERC CoG 863818 (FoRM-SMArt), and the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 665385.
Volume
35
Issue
5A
Page
3787-3795
Conference
AAAI: Association for the Advancement of Artificial Intelligence
Conference Location
Virtual
Conference Date
2021-02-02 – 2021-02-09
ISSN
eISSN
IST-REx-ID

Cite this

Henzinger TA, Lechner M, Zikelic D. Scalable verification of quantized neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol 35. AAAI Press; 2021:3787-3795.
Henzinger, T. A., Lechner, M., & Zikelic, D. (2021). Scalable verification of quantized neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, pp. 3787–3795). Virtual: AAAI Press.
Henzinger, Thomas A, Mathias Lechner, and Dorde Zikelic. “Scalable Verification of Quantized Neural Networks.” In Proceedings of the AAAI Conference on Artificial Intelligence, 35:3787–95. AAAI Press, 2021.
T. A. Henzinger, M. Lechner, and D. Zikelic, “Scalable verification of quantized neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence, Virtual, 2021, vol. 35, no. 5A, pp. 3787–3795.
Henzinger TA, Lechner M, Zikelic D. 2021. Scalable verification of quantized neural networks. Proceedings of the AAAI Conference on Artificial Intelligence. AAAI: Association for the Advancement of Artificial Intelligence, Technical Tracks, vol. 35, 3787–3795.
Henzinger, Thomas A., et al. “Scalable Verification of Quantized Neural Networks.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 5A, AAAI Press, 2021, pp. 3787–95.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Main File(s)
Access Level
OA Open Access
Date Uploaded
2022-01-26
MD5 Checksum
2bc8155b2526a70fba5b7301bc89dbd1


Link(s) to Main File(s)
Access Level
OA Open Access
Material in ISTA:
Dissertation containing ISTA record

Export

Marked Publications

Open Data ISTA Research Explorer

Sources

arXiv 2012.08185

Search this title in

Google Scholar
ISBN Search