CWI researchers take an important step towards artificial intelligence (AI) that can be used in local devices such as smartphones and in VR-like applications, while at the same time protecting privacy. Brain-like neurons combined with new learning methods make it possible to train fast and energy-efficient spiking neural networks on a large scale.
The study is the work of CWI researchers Bojian Yin and Sander Bohté. They present the results of their study in Nature Machine Intelligence. According to CWI, the development opens the door to potential applications ranging from wearable AI to ubiquitous speech recognition and augmented reality.
From voice recognition to drone navigation
Artificial neural networks are the backbone of the current AI revolution. But they are only loosely based on networks of real, biological neurons like our brains. The human brain encompasses a much larger network, is much more energy-efficient and can respond ultra-fast when triggered by external events.
Spiking neural networks are a special kind of artificial neural networks that come closer to the way our brain works. Thus, they are able to more closely mimic the functioning of biological neurons. Like their biological counterpart, spiking neural networks communicate by exchanging electrical pulses. This without using a large amount of energy in the process.
Bringing AI programmes closer to users
If you manage to integrate spiking neural networks into a chip (called neuromorphic hardware), you can bring AI programmes closer to users on their own devices. Think smartphones, tablets and smartwatches. CWI calls such local solutions beneficial for the privacy, robustness and responsiveness of these programmes. It foresees applications ranging from voice recognition in toys and devices, healthcare monitoring and drone navigation to local surveillance.
Like 'normal' artificial neural networks, spiking neural networks need to be trained to perform their tasks properly. However, communication between such networks poses a serious challenge. "The algorithms required for this require a lot of computer memory, so we can usually only train small network models for smaller tasks. This holds back many practical AI applications so far," explains Bohté of CWI's Machine Learning group.
Mimicking the learning brain
These algorithms cannot match the learning ability of our brain, the researchers report. For instance, the brain learns directly from new experiences by changing connections or even creating entirely new ones. The human brain requires far fewer examples to do this and works more energy-efficiently. "We wanted to develop something closer to the way our brains learn," says Yin.
If you make a mistake during a driving lesson, you immediately learn from it. You correct your behaviour while you are driving, not an hour later. Yin explains, "You learn as you absorb the new information, so to speak. We wanted to mimic that by giving each neuron of the neural network a piece of information that is constantly updated. That way, the network learns how the information changes, and does not have to remember all the previous information. This is the big difference from current networks, which have to work with all previous changes. The current way of learning requires huge computing power, and thus a lot of memory and energy."
Six million neurons
The new learning algorithm allows learning directly from data. This enables much larger spiking neural networks. Together with researchers from TU Eindhoven and research partner Holst Centre, Bohté and Yin demonstrated this in a system designed for recognising and localising objects. For their study, they used real-time images of a busy street in Amsterdam. The underlying spiking neural network, SPYv4, was trained to distinguish cyclists, pedestrians and cars and indicate exactly where they are.
"Previously, we could train neural networks with up to 10,000 neurons; with the current algorithm, we can do so quite easily with networks with more than six million neurons," says Bohté. "This allows us to train advanced spiking neural networks, such as our SPYv4."
Future
"Now that we have such powerful AI solutions based on spiking neural networks, chips are being developed that can run these AI programmes while consuming very little power," Bohté explains. The researchers expect the networks to eventually find their way into numerous smart devices. These include hearing aids, but also virtual reality glasses.
Author: Wouter Hoeffnagel
Photo: Pixabay / Seanbatty
