Fraunhofer-project SEC-Learn

The SEC-Learn (Sensor Edge Cloud for Federated Learning) project, involving eleven Fraunhofer Institutes, promises a major technological leap in the field of neuromorphic hardware: for the first time, a chip is being developed to accelerate spiking neural networks (SNN) in conjunction with what is known as federated learning. This would offer practical benefits for companies and individuals.

Currently, machine learning is still very dependent on big data centers. Implementing artificial intelligence directly on edge devices, by contrast, brings advantages in terms of both data protection and efficiency. In general, however, two problems stand in the way: for one thing, battery-powered devices have a rather limited energy budget; having a powerful graphics card active in the background would quickly exceed it. This is especially true for devices and applications that are permanently in operation, such as a voice recognition system

The second problem is that machine learning requires huge data sets that simply don't fit on the available memory of a normal edge device. At present, only cloud providers can store datasets that are big enough for machine learning. However, this leads to privacy and security concerns: no one wants to send raw audio files, for instance, directly to the cloud of one of the major speech recognition providers.

SEC-Learn is designed to overcome these problems. To meet energy efficiency requirements, devices need to be able to handle data processing locally while making use of energy efficient dedicated neural network circuits. To this end, Fraunhofer IIS is developing a neuromorphic chip that is much more energy efficient than conventional chips. The other problem, i.e., how to take data that originates locally and pool it in a cloud while maintaining high security standards, is solved by federated learning. This means that no raw data has to be passed on, only the changes to the models.

Three phases to the goal

The project started with SEC-Learn-Takeoff: during this six-month phase, the eleven institutes came together to develop a common vision for the project. Tasks were apportioned, and the final product critically considered: Can combining an SNN with federated learning even work in the first place? As a result, concepts were first drawn up for how to achieve this goal.

This was followed by SEC-Learn-Fly. This phase involved developing the individual components for the previously designed concepts. Fraunhofer IIS took on responsibility for developing a test chip with synaptic weights and spiking neurons, with the goal of arriving at a tape-out – the final result of the chip design process – by the end of SEC-Learn-Fly. This chip is currently in production. Thereafter, researchers will explore the extent to which the mixed-signal test chip can be more energy efficient than (software-based) solutions in off-the-shelf hardware. This also marks the beginning of the final phase, which SEC-Learn is now in: SEC-Learn-Arrival.

The aim of this two and a half year project is to finally develop a working neuromorphic hardware accelerator and to integrate it in a hardware platform in conjunction with federated learning. While the previous phases saw tasks defined and initial components developed, the task now is to bring these components together. Fraunhofer IIS, together with Fraunhofer EMFT, is providing an important building block here for the further success of the project through its test chip. The tests carried out with the chip will form the basis for the project’s further development.

Spin-offs likely after project ends

While there are already many companies and start-ups working on either neuromorphic hardware or federated learning, to date this effort to combine the two technologies is unique. The use of a mixed-signal SNN chip – i.e., using the combination of analog and digital circuits in such a chip – is also breaking new technological ground. In this respect, Fraunhofer IIS, together with the other institutes, is doing pioneering work at a level that could fundamentally and sustainably change future technology. It is likely that SEC-Learn will result in numerous spin-off projects.