A Deep Neural Network Deployment Based on Resistive Memory Accelerator
  Simulation
The objective of this study is to illustrate the process of training a Deep
Neural Network (DNN) within a Resistive RAM (ReRAM) Crossbar-based simulation
environment using CrossSim, an Application Programming Interface (API)
developed for this purpose. The CrossSim API is designed to simulate neural
networks while taking into account factors that may affect the accuracy of
solutions during training on non-linear and noisy ReRAM devices. ReRAM-based
neural cores that serve as memory accelerators for digital cores on a chip can
significantly reduce energy consumption by minimizing data transfers between
the processor and SRAM and DRAM. CrossSim employs lookup tables obtained from
experimentally derived datasets of real fabricated ReRAM devices to digitally
reproduce noisy weight updates to the neural network. The CrossSim directory
comprises eight device configurations that operate at different temperatures
and are made of various materials. This study aims to analyse the results of
training a Neural Network on the Breast Cancer Wisconsin (Diagnostic) dataset
using CrossSim, plotting the innercore weight updates and average training and
validation loss to investigate the outcomes of all the devices.
Tejaswanth Reddy Maram, Ria Barnwal, Bindu B
# A Deep Neural Network Deployment Based on Resistive Memory Accelerator Simulation

###### Abstract

The objective of this study is to illustrate the process of training a Deep Neural Network (DNN) within a Resistive RAM (ReRAM) Crossbar-based simulation environment using CrossSim, an Application Programming Interface (API) developed for this purpose. The CrossSim API is designed to simulate neural networks while taking into account factors that may affect the accuracy of solutions during training on non-linear and noisy ReRAM devices. ReRAM-based neural cores that serve as memory accelerators for digital cores on a chip can significantly reduce energy consumption by minimizing data transfers between the processor and SRAM and DRAM. CrossSim employs lookup tables obtained from experimentally derived datasets of real fabricated ReRAM devices to digitally reproduce noisy weight updates to the neural network. The CrossSim directory comprises eight device configurations that operate at different temperatures and are made of various materials. This study aims to analyze the results of training a Neural Network on the Breast Cancer Wisconsin (Diagnostic) dataset using CrossSim, plotting the innercore weight updates and average training and validation loss to investigate the outcomes of all the devices.

## I Introduction

When we talk about the existing memory technologies, we mainly talk about SRAM, DRAM and flash memory. While all of them are charge based storage devices, they are used in different places according to their varied performances in different areas. SRAM devices are high speed, low power consuming devices which are often used as small cache memory units in the computers because of it's complexity to scale in size and costlier to manufacture. DRAM being a cheaper technology and better scalable, it is used for bigger RAM units which stores the temporary data that is required by the computer for the current program at volatile level. Flash memory is a highly scalable and low-power-consuming non-volatile memory technology, which makes it an ideal candidate for use as a secondary storage unit in computer systems.

It is desired by the industry to create an ideal memory technology which has most of the upsides and very few if not none of the downsides of the current memory technologies. Hence with the aim of reinventing the current memory storage techniques, the industry is actively researching the development of a new hierarchy of memory often known as emerging memory technologies. The purpose of these emerging memory technologies is to enable the strengths of existing memory technologies like the rapid toggling capability of SRAMs, high storage capacity equivalent to DRAMs, and non-volatile capability of Flash memory. thus having the potential to be highly appealing alternatives to the next-generation memory hierarchy. [1]

In contrast to charge-based memories technologies, these memory technologies works on the principle of making changes in the resistance of the cells to store the information. Few examples of this emerging memory technologies are : (i) PCM or phase change memory, (ii) STT-MRAM or spin-transfer torque magnetoresistive random access memory, and (iii) ReRAM or resistive random access memory. ReRAM is characterized as a non-volatile memory type that has the ability to preserve data even during the power absence. ReRAM stores the data in form of Low and High resistive states. The Metal Insulator Metal (MIM) structure of this memory type allows the fabrication to be simple. It comprises an insulating layer (I) positioned between two metal (M) electrodes. The storage and retrieval of data is dependent on the creation and breaking of a conductive filament between the electrodes, resulting in the low resistance state (LRS) and high resistance state (HRS) respectively. These states will be retained until it is changed with a writing voltage thus making the ReRAMS non volatile.

A ReRAM uses a set voltage (Vs) to form a conductive filament between the electrodes representing the Low Resistive State(LRS). And a reset voltage (Vr) to rupture the filament representing the High Reistive State (HRS).

In comparison with a conventional CPU, resistive memory crossbars can reduce the energy required to perform computations in neural algorithms by five orders of magnitude or

Fig. 1: Existing memory technologies and their properties.

Fig. 2: Advantages of ReRAM [2]