Bayesian NN (BayNN) using Dropout-based approximation provides a systematic approach for estimating the uncertainty of predictions. Despite such merit, they are not suitable for implementation in an embedded device and in most cases cannot meet high-performance demands for certain applications. Computation in-memory (CiM) architecture with spintronics emerging non-volatile memories (NVMs) is a great candidate for high-performance and low-power acceleration BayNNs in hardware.
Neural Networks (NNs) are in the center of modern computing thanks to their ability to solve complex and difficult tasks in numerous application domains, including autonomous driving and medical applications. Nevertheless, their predictions can be incorrect if the input sample is outside of the training distribution or affected by noise. Consequently, quantifying the uncertainty of the NN prediction allows the system to make more insightful decisions by avoiding blind predictions. Therefore, uncertainty quantification is crucial for a variety of applications, including safety-critical applications.
Spindrop Module Schematic and Inference architecture for Bayesian Binary Neural Network.
We propose for the first time a Bayesian Binary Neural Network (BayBNN) using spintronic-based Dropout as a Gaussian approximation and develop a complete solution and flow spanning from the training algorithm to the circuit-level hardware implementation. We propose a hardware-based Dropout method and present its hardware implementation using spintronic devices, specifically STT-MTJs. We demonstrate the concept of Bayesian Binary Neural Networks through holistic, extensive analysis from device and circuit level evaluations up to the algorithmic level. We show that the proposed approach can be used to detect out-of-distribution and noisy data, being robust against variability issues of the Dropout generation mechanism.
Our proposed approach embraces the randomness of the STT-spintronic-based CiM architecture, considering it as a feature instead of as an issue. For this purpose, the stochastic and deterministic aspects of STT-MRAM have been combined in a crossbar array-based architecture. Moreover, at the architecture level, we show that it does not imply changes in the bit-cell design, leading to the re-utilization of the bit-cell array designed for classic MRAM memory or for classic Binary NN. The results show up to 100% detection capabilities for out-of-distribution data, and up to 15% improvement in accuracy for poisoned data. Furthermore, our results show the high resilience of the proposed concept to process and thermal variations.
Teams: Artificial Intelligence, IC Design
Collaboration: KIT, Karlsruhe, Germany
Funding: ANR DFG Grand NEUSPIN, ANR-21-FAI1-0008 ,and partially by ANR-19-PI3IA-0
Further reading: SpinDrop: Dropout-Based Bayesian Binary Neural Networks With Spintronic Implementationr, S. Ahmed; et al., IEEE Journal on Emerging and Selected Topics in Circuits and Systems (Volume: 13, Issue: 1, March 2023). DOI: 10.1109/JETCAS.2023.3242146.
Contacts: Lorena Anghel