On the Way to Artifical Intelligence

The Neuromorphic Survey

What is neuromorphic hardware?

Icon image - neuromorphic hardware
© zapp2photo - stock.adobe.com

When traditional computers (with von Neumann architectures) calculate Deep Neural Networks (DNNs), they suffer from the von Neumann bottleneck. This means that the computing performance of the computing systems is limited by the data rate which can be transferred between an external memory unit and the processing unit.

Neuromorphic hardware refers to brain-inspired computers or components modeling neurological or artificial neural networks consisting of highly connected parallel synthetic neurons and synapses. Current neural network architectures like DNNs require high computational complexity and power consumption. Efficient architectures for neuromorphic hardware with respect to computational performance, power consumption and chip area are therefore a key element for a widespread deployment of neural networks in embedded devices.

High-performance neuromorphic hardware architectures tailored for efficient computation of neural networks are applying massive parallel processing and colocation of memory and processing. By applying these approaches, calculations required by complex neural networks can be performed faster and with less power compared to von-Neumann architectures. As neural networks exhibit highly regular structures, massive parallel processing can be applied by using the same type of computational units/cells in parallel. Architectural approaches known from parallel computing architectures can therefore be applied:

  • Single-Instruction Multiple Data (SIMD): multiple parallel processing elements (e.g. MAC units) perform the same operation on different pieces of distributed data simultaneously (data-level parallelism).
  • Very Long Instruction Word (VLIW): several not necessarily equal instructions are performed in parallel (instruction-level parallelism).
  • Systolic arrays: dataflow architectures based on a network of tightly-coupled homogeneous processing elements. Computations are performed in a pipelined manner by passing data through the systolic array.

Overview of neuromorphic hardware

A taxonomy of neuromorphic hardware platforms is provided in the following table with their strengths and drawbacks with respect to neuromorphic computing.

PLATFORM STRENGTHS DRAWBACKS
General Purpose
CPU
(GP CPU)
  • Highly programmable
  • Low energy efficiency
  • Low computational throughput compared to other architectures
GP CPU with
accelerators
  • Accelerators can be tailored to a dedicated task
  • Computational throughput depends on number of attached accelerators
General Purpose
Graphics Pro-
cessing Unit

(GP-GPU)
  • Good for training large data sets on server-based DL
  • Highly parallel computational resources
  • Lack of efficiency for inference
  • Power consumption
Dedicated
Hardware

(Application
specific ICs, ASICs)
  • Low power
  • Highest performance for targeted applications / NN architectures / frameworks
  • Chip architecture possibly inflexible to adapt to rapid evolution of Deep Learning topologies/frameworks
  • In most cases restricted to inference-only

Concerning the ASIC manufacturer:

  • High costs for ASIC development and production
  • Large market volume required to justify development costs
  • Slow Time to Market (TTM)
FPGA
  • Reconfigurability allows adaptation to evolving NN architectures/frameworks
  • Custom NN architectures can be implemented
  • HW acceleration for specific operations on-chip (MAC) and on-chip memory (SRAM) available
  • TTM less restrictive than for ASICs as firmware can be altered
  • Not suitable for low power applications
  • Limited on-FPGA memory (SRAM)
  • Limited data rate to external memory (DDR)
DSPs
  • Acceleration of the most important operations (MAC)
  • Similar memory bottleneck as for CPUs due to von Neumann architecture
  • Typically higher performance as CPU but lower than GPU

GP CPU: General Purpose Central Processing Unit; GP-GPU: General Purpose Graphics Processing Unit; ASIC: Application Specific Integrated Circuit; FPGA: Field-Programmable Gate Array; DSP: Digital Signal Processor

Overview of deep
learning inference
accelerators

 

AI chips and IP cores

Source: Elektronik, 9/2019

Spiking Neural Networks

Spiking Neural Networks (SNN) are a form of brain-inspired neural networks where communication between the neurons is modeled in form of spikes or pulses similar to the communication in a real human brain. The computation in a neuron depends not only on the amplitude of the spikes but also on the time of arrival and the shape of the pulses. A simple spiking neuron model exhibiting these properties is the (leaky) integrate-and-fire neuron. Due to their brain-like mode of operation SNNs have the potential to be highly power efficient. However, still open issues for research and practical application of SNNs do exist (e.g. optimal network and neuron models, methods for training, input processing).

SNN chips are not yet commercially available but research and test chips have been implemented by companies as well as in public research projects.

Consulting, design and implementation

Fraunhofer IIS has long-standing experience in the application of Machine Learning to various use cases. Our experts employ neuromorphic hardware to speed up the computation in embedded devices.

Our offer includes consulting for your Machine Learning use case and suitable neuromorphic hardware as well as the design and implementation of modules for your devices that employ neuromorphic hardware.

Further information

 

Embedded Machine Learning

Implementation and integration of machine learning algorithms on embedded devices

 

Machine Learning at Fraunhofer IIS

Overview of the topic "Machine Learning" at Fraunhofer IIS