Optical tracking - CNN

Optical Tracking

At a glance

CNNLok provides a camera-based, infrastructure-free location of mobile objects.

At CNNLok, mobile computers with cameras use a special deep learning architecture to localize themselves in a learned environment. This can be simple Android smartphones to embedded computers with multiple cameras are used. By continuously re-exploring the environment and using an application-specific motion model, reliable positions are calculated.

The technology

CNNLok can also be used when the environment changes, such as when the loading of high racks changes in the logistics sector. Localization in areas where infrastructure cannot be deployed is also something that can be learned. In contrast to standard radio-based or optical technologies, no sensor infrastructure is required for self-positioning; nor do any optical markers have to be installed in the environment, as other self-localizing systems often require. Static elements in the existing environment are all that is needed to calculate positions. As a result, CNNLok is self-sufficient and can be used very flexibly in a wide variety of application fields. As the technical feasibility of this idea has now been confirmed, a team from Fraunhofer IIS is working on a practical solution for CNNLok. Although there are some hurdles still to be overcome, development of the self-learning technology is progressing rapidly.

© iStock

How it works

The mobile processing unit uses a so-called convolutional neural network to calculate the current position based on a camera image. First of all, data is automatically captured from thousands of camera images and the corresponding positions. Using this data, an existing network is further trained for the purpose of adapting it to the target environment. The finished network is used on the target platforms to determine the positions of new images. To prevent the system from degenerating over time, updated information is continuously collected on a central processing unit for further training of the network and distribution to the mobile processors.


System components of CNNLok

The mobile processing unit is typically either a simple smartphone or an ARM- or Intel-based single-board computer with a standard camera. Thanks to the high flexibility of the platform, many application scenarios are possible that cannot be accomplished by standard, infrastructure-based solutions. Adapted motion models and special pre-processing of the collected data allow new data to populate an existing positioning system. Because continuously learning more about the environment is a very computing-intensive task, the system needs to be connected up to other hardware via a network or docking station or similar means. Depending on the dynamics of the area, a central processor with powerful standard deep learning hardware – such as graphics cards or special vector processors – may be additionally required. This takes over the duties of the mobile processing units at appropriate times – for example, when they need to be charged. 

Added value for industry

By virtue of its autonomous self-positioning, CNNLok offers major potential for the optimization of workflows in logistics, even in dynamic environments. Downstream analytical applications could use the localization to achieve efficiencies such as increased throughput of goods.

Added value for indoor localization

For various indoor applications that are unsuitable for infrastructure and are situated in a dynamic environment, indoor localization could serve as the basis for developing specific positioning solutions. For example, it allows the creation of navigation services in the retail sector that can be implemented on a purely local basis on a smartphone. 

Your partner from the research sector

Various further fields of application are conceivable for CNNLok. The positioning device’s autonomy, the fact that it needs no positioning infrastructure, and its tolerance of dynamic changes together open up a wealth of new possibilities. 

We would be glad to further develope CNNLok with you and adapt the technology precisely to your wishes and requirements