Spiking Neural Networks and Neuromorphic Computing

Problem Definition

Neuromorphic computing describes a relatively new class of computing devices. In the conventional Von-Neumman model of computing, the design of most computers to date, the memory and processing is separate. Neuromorphic computing takes a more biologically inspired route, where the memory and processing are together, in a structure of neurons emitting spikes, similar to the brain or some artificial neural networks. It has been shown neuromorphic computing has the potential for vastly more energy-efficient computation than conventional computers, requiring orders of magnitude less energy, and as a side effect also producing less excess heat. The problem is designing algorithms to effectively take advantage of these benefits. This remains a largely open problem. DVS cameras, also called an event-camera or silicon retina, are a class of neuromorphic sensor. Rather than recording an image at a constant frame-rate like a conventional camera recording a video, an event-camera only records an event when the luminace of a pixel changes. This has many benefits such as a high dynamic range, no motion blur, and very high effective frame-rates. However it also has its drawbacks, because it can be difficult to reconstruct a full video stream from the sequence of events.

Problem Solution

Neuromorphic computation can be simulated with spiking neural networks on conventional computers. This can be used to learn how to design algorithms for neuromorphic hardware. Spiking neural networks, having much in common with event-camera, are a suitable choice of algorithm to reconstruct the video data from event-camera data.

Progress to Date

Presently, I have implemented a simulation of a spiking neural network and designed algorithms to train it. I applied this to N-MNIST, or neuromorphic MNIST, data. This is a collection of handwritten digits (MNIST), recorded moving around with an event-camera. The simulated spiking neural network is able to reconstruct the image of the handwritten digit from the event-camera spike data. I have also constructed a simulated spiking neural network and trained it to classify the digit being moved. I then added random noise to the data, and train a spiking neural network to see through the data and classify the digit despite the noise. I tested several different architectures of spiking neural network to see which are able to do these tasks best.

Expected Results

I will continue by testing different architectures of spiking network on the tasks I apply them to. I expect to improve the accuracy of the classifier by using different architectures. I also plan to see how well a spiking neural network can reconstruct a natural image from event-camera data of the image moving.

Team Members: Robert Strauss
Sponsoring Teacher: Charlie Strauss