Nardin, MicheleIST Austria ; Phillips, James W; Podlaski, William F; Keemink, Sander W
The brain performs many nonlinear computations through intricate spiking neural networks (SNNs). How neural network dynamics relate to arbitrary computations under these constraints is still an open question. As a strong constraint, these networks are hypothesized to be robust to perturbations and use minimal energy. The theory of Spike Coding Networks (SCNs) derives the required connectivity and dynamics for both information representation and linear dynamical systems from first principles, and achieves robustness and efficiency. Nonlinear dynamical systems have thus far only been implemented in SCNs by filtering neural inputs through sets of nonlinear dendritic basis functions. While this approach works well, it relies on providing a rich enough basis set as well as supervised training of the connectivity weights. Another way to implement nonlinear computations is through multiplicatively interacting synapses. However, there is currently no principled way to implement such synapses in SCNs. Here, we extend the core SCN derivations to implement polynomial dynamical systems, from which also the need for such multiplicatively interacting synapses arises. We demonstrate our approach with a highly accurate Lorenz attractor implementation, as well as a second-order approximation of a double pendulum. We additionally demonstrate how to implement higher-order polynomials using sequential networks with only pair-wise synapses. Finally, we derive upper bounds and expected numbers of connections based on the sparsity of the underlying representation. Overall, our work provides an alternative way to directly implement nonlinear computations in spike coding networks, and expands our understanding about the potential functions of multiplicative synapses. Furthermore, due to the high accuracy and low energy usage of our approach, this work may be of interest for neuromorphic computing.
We thank Christian Machens for useful discussions on the project. This report came out of a collaboration started at the CAJAL Advanced Neuroscience Training Programme in Computational Neuroscience in Lisbon, Portugal, during the 2019 summer. The authors would like to thank the participants, TAs, lecturers, and organizers of the summer school. SWK was supported by the Simons Collaboration on the Global Brain (543009). WFP was supported by FCT (032077). MN was supported by European Union Horizon 2020 (665385).
Nardin M, Phillips JW, Podlaski WF, Keemink SW. Nonlinear computations in spiking neural networks through multiplicative synapses. arXiv.
Nardin, M., Phillips, J. W., Podlaski, W. F., & Keemink, S. W. (n.d.). Nonlinear computations in spiking neural networks through multiplicative synapses. arXiv.
Nardin, Michele, James W Phillips, William F Podlaski, and Sander W Keemink. “Nonlinear Computations in Spiking Neural Networks through Multiplicative Synapses.” ArXiv, n.d.
M. Nardin, J. W. Phillips, W. F. Podlaski, and S. W. Keemink, “Nonlinear computations in spiking neural networks through multiplicative synapses,” arXiv. .
Nardin M, Phillips JW, Podlaski WF, Keemink SW. Nonlinear computations in spiking neural networks through multiplicative synapses. arXiv, 2009.03857.
Nardin, Michele, et al. “Nonlinear Computations in Spiking Neural Networks through Multiplicative Synapses.” ArXiv, 2009.03857.