Response characterization for auditing cell dynamics in long short-term memory networks

R. Hasani, A. Amini, M. Lechner, F. Naser, R. Grosu, D. Rus, in:, Proceedings of the International Joint Conference on Neural Networks, IEEE, 2019.


Conference Paper | Published | English
Author
; ; ; ; ;
Department
Abstract
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate the generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.
Publishing Year
Date Published
2019-09-30
Proceedings Title
Proceedings of the International Joint Conference on Neural Networks
Article Number
8851954
Conference
IJCNN: International Joint Conference on Neural Networks
Conference Location
Budapest, Hungary
Conference Date
2019-07-14 – 2019-07-19
IST-REx-ID

Cite this

Hasani R, Amini A, Lechner M, Naser F, Grosu R, Rus D. Response characterization for auditing cell dynamics in long short-term memory networks. In: Proceedings of the International Joint Conference on Neural Networks. IEEE; 2019. doi:10.1109/ijcnn.2019.8851954
Hasani, R., Amini, A., Lechner, M., Naser, F., Grosu, R., & Rus, D. (2019). Response characterization for auditing cell dynamics in long short-term memory networks. In Proceedings of the International Joint Conference on Neural Networks. Budapest, Hungary: IEEE. https://doi.org/10.1109/ijcnn.2019.8851954
Hasani, Ramin, Alexander Amini, Mathias Lechner, Felix Naser, Radu Grosu, and Daniela Rus. “Response Characterization for Auditing Cell Dynamics in Long Short-Term Memory Networks.” In Proceedings of the International Joint Conference on Neural Networks. IEEE, 2019. https://doi.org/10.1109/ijcnn.2019.8851954.
R. Hasani, A. Amini, M. Lechner, F. Naser, R. Grosu, and D. Rus, “Response characterization for auditing cell dynamics in long short-term memory networks,” in Proceedings of the International Joint Conference on Neural Networks, Budapest, Hungary, 2019.
Hasani R, Amini A, Lechner M, Naser F, Grosu R, Rus D. 2019. Response characterization for auditing cell dynamics in long short-term memory networks. Proceedings of the International Joint Conference on Neural Networks. IJCNN: International Joint Conference on Neural Networks
Hasani, Ramin, et al. “Response Characterization for Auditing Cell Dynamics in Long Short-Term Memory Networks.” Proceedings of the International Joint Conference on Neural Networks, 8851954, IEEE, 2019, doi:10.1109/ijcnn.2019.8851954.

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 1809.03864

Search this title in

Google Scholar
ISBN Search