Into the unknown: active monitoring of neural networks

Lukina A, Schilling C, Henzinger TA. 2021. Into the unknown: active monitoring of neural networks. 21st International Conference on Runtime Verification. RV: Runtime Verification, LNCS, vol. 12974, 42–61.


Conference Paper | Published | English

Scopus indexed
Department
Series Title
LNCS
Abstract
Neural-network classifiers achieve high accuracy when predicting the class of an input that they were trained to identify. Maintaining this accuracy in dynamic environments, where inputs frequently fall outside the fixed set of initially known classes, remains a challenge. The typical approach is to detect inputs from novel classes and retrain the classifier on an augmented dataset. However, not only the classifier but also the detection mechanism needs to adapt in order to distinguish between newly learned and yet unknown input classes. To address this challenge, we introduce an algorithmic framework for active monitoring of a neural network. A monitor wrapped in our framework operates in parallel with the neural network and interacts with a human user via a series of interpretable labeling queries for incremental adaptation. In addition, we propose an adaptive quantitative monitor to improve precision. An experimental evaluation on a diverse set of benchmarks with varying numbers of classes confirms the benefits of our active monitoring framework in dynamic scenarios.
Publishing Year
Date Published
2021-10-06
Proceedings Title
21st International Conference on Runtime Verification
Acknowledgement
We thank Christoph Lampert and Alex Greengold for fruitful discussions. This research was supported in part by the Simons Institute for the Theory of Computing, the Austrian Science Fund (FWF) under grant Z211-N23 (Wittgenstein Award), and the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 754411.
Volume
12974
Page
42-61
Conference
RV: Runtime Verification
Conference Location
Virtual
Conference Date
2021-10-11 – 2021-10-14
ISSN
eISSN
IST-REx-ID

Cite this

Lukina A, Schilling C, Henzinger TA. Into the unknown: active monitoring of neural networks. In: 21st International Conference on Runtime Verification. Vol 12974. Cham: Springer Nature; 2021:42-61. doi:10.1007/978-3-030-88494-9_3
Lukina, A., Schilling, C., & Henzinger, T. A. (2021). Into the unknown: active monitoring of neural networks. In 21st International Conference on Runtime Verification (Vol. 12974, pp. 42–61). Cham: Springer Nature. https://doi.org/10.1007/978-3-030-88494-9_3
Lukina, Anna, Christian Schilling, and Thomas A Henzinger. “Into the Unknown: Active Monitoring of Neural Networks.” In 21st International Conference on Runtime Verification, 12974:42–61. Cham: Springer Nature, 2021. https://doi.org/10.1007/978-3-030-88494-9_3.
A. Lukina, C. Schilling, and T. A. Henzinger, “Into the unknown: active monitoring of neural networks,” in 21st International Conference on Runtime Verification, Virtual, 2021, vol. 12974, pp. 42–61.
Lukina A, Schilling C, Henzinger TA. 2021. Into the unknown: active monitoring of neural networks. 21st International Conference on Runtime Verification. RV: Runtime Verification, LNCS, vol. 12974, 42–61.
Lukina, Anna, et al. “Into the Unknown: Active Monitoring of Neural Networks.” 21st International Conference on Runtime Verification, vol. 12974, Springer Nature, 2021, pp. 42–61, doi:10.1007/978-3-030-88494-9_3.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 2009.06429

Search this title in

Google Scholar
ISBN Search