X-Ray technology develops into a cognitive sensor


Of the systems capable of monitoring large components such as whole car bodies, none have so far been suitable for an application for series production. Now, with RoboCT, we can inspect these components quickly and accurately. At the same time, this robot-based computed tomography (CT) system also represents the first step towards cognitive sensor systems in this field. In addition to applications in the automotive and aerospace industries, the system is to be used, for example, to inspect the condition and completeness of returned goods ordered online – without having to open them.


The door to the X-ray cabinet closes slowly, a quiet click signaling that the cabinet is securely locked. Wolfgang Holub, an engineer and scientist at our Development Center for X-ray Technology EZRT takes his place at the computer connected to the system and starts an X-ray measurement process with a few clicks of the mouse. As the X-ray cabinet has no windows, he keeps a watchful eye on a monitor showing a live image from inside the locked cabinet. The sizable component we see on the screen – a section of vehicle bodywork – measures around two meters wide and one and a half meters tall and deep and is securely tied to a pallet using ratchet straps. Two robotic arms, equipped with an X-ray source and detector, carefully approach the large component, one from the left and one from the right of the bodywork. After a second’s pause, the robotic arms move further around the part in total synchrony and with the utmost precision. »Right now, the robots are taking 2D X-ray images,« says Holub. »This already gives us a clear picture of whether there are any irregularities in the structure of the bodywork. If part of the component shows any such irregularities, the system can automatically perform a highly specific CT scan in 3D.« Holub stops the measurement process and makes a few manual adjustments in order to demonstrate the process involved in 3D CT. Shortly afterwards, the monitor shows the robotic arms adopting a new position altogether. The robots are now revolving around the point where we suspect there to be an irregularity. During their 180° rotation, the arms pause about once a second to capture an image. »This approach allows us to create 3D CT scans that give us far more information than two-dimensional X-ray images,« Wolfgang Holub explains.

»We won't measure material data indiscriminately or in its entirety. Instead, we'll only record the relevant data.«

© Fraunhofer IIS
3D CT – a volume rendering of the nested structure in the lock area of the tailgate.

Flexibility on a completely new level

First things first, however: What is so special or novel about this technology? After all, two-dimensional X-ray inspections have been the state-of-the-art tool of choice in industry for many years. X-ray techniques can even be applied to relatively large objects, such as alloy wheels or cylinder heads, by running through a series of inspection positions and perspectives using a device known as a manipulator.

In practice, however, inspection methods of this kind could only be applied to objects of limited size and geometric complexity. For larger components, such as the vehicle bodywork in our example, none of the existing systems are suitable for series inspection, let alone integration into the production process – and certainly not for complete 3D CT scans. The tests can only be conducted under laboratory conditions, at considerable expense, and at a small number of institutions, such as the laboratories of the Fraunhofer Development Center for X-ray Technology EZRT. It is only thanks to the capabilities of RoboCT – which uses large industrial robots to move the X-ray hardware – that it is possible to plot an automatic inspection procedure in which the component is inspected in a few seconds or minutes, depending on the inspection task, and hard-to-image sections or regions with unclear findings are analyzed in detail using computed tomography. In the example application of a cast aluminum tailgate for a car, such regions might include the area around the door lock, which has a nested structure.

A focus on intelligent monitoring

However, the robot-based CT scanner described here is just the tip of the iceberg. Led by Professor Randolf Hanke, the team at the Fraunhofer Development Center for X-ray Technology EZRT still has a lot of work to do – and the key concept is »intelligent monitoring«. In the future, this will no longer simply be about making an OK or not OK decision. Rather, our aim is to provide customers with a monitoring system that helps them to optimize their processes.

The term »process« by no means refers exclusively to the classical production process. It also encompasses material development, design, maintenance, trading, and recycling processes. »Based on this, the focus of our research efforts is shifting. We’re now increasingly focusing on the development of cognitive and auto-adaptive sensor systems,« Professor Hanke explains. »Advances in the area of robot-based computed tomography make these aims considerably more accessible,« he adds.

»Big data« under control

Nowadays, faced with huge volumes of available data, the aim is to use learning algorithms to extract information that can be used to achieve something – that is, to better understand, observe, or design processes. However, when we talk about big data today, we are almost always referring to factory data, logistics data, cost data, machine data, and so on. Until now, what is termed »smart materials data« has barely featured in discussions about big data. »In the future, materials and products will be monitored throughout the value chain – that is, over their entire life cycle from raw material to use and finally recycling – in order to examine the changes that take place whenever humans, machines, or the environment alter the material or product in some way. Moreover, in the future, we won’t measure material data indiscriminately or in its entirety. Instead, we’ll only record the relevant data, and it’ll be for the intelligent measurement system – the cognitive sensor system – to decide which data is relevant or smart. These are the research and development tasks for non-destructive monitoring that we’re very familiar with as experts in non-destructive testing and that we know harbor enormous potential,« Professor Randolf Hanke explains.

The intelligent black box

In the future, we can imagine a scenario in which an intelligent monitoring system – a type of »black box« – is delivered to customers, who need not give it much thought or indeed have any knowledge of non-destructive testing whatsoever. The box might contain robots with access to different sensor systems, for example, and these robots would decide for themselves, in the broadest sense, which method to use. The robot would then select an X-ray system, an airborne ultrasound system, or even a thermal imaging system to solve a highly specific problem ¬– instead of simply carrying out an inspection.

»We won't measure material data indiscriminately or in its entirety. Instead, we'll only record the relevant data.«

© Jimmy Lopes - Fotolia.com
Percentage rates of returns in the mid-double figures are presenting e-commerce businesses with an increasingly difficult challenge.

Protecting the environment and saving money – robot-based monitoring in retail

At the Fraunhofer Application Center for CT in Metrology, which is part of Fraunhofer IIS, we are demonstrating that this vision is anything but far-fetched. In collaboration with Deggendorf Institute of Technology (DIT), we are planning, as early as 2018, to establish a robot-based digitalization center to improve the efficiency of returns management and the e-commerce sector.

The reasons for this new development are obvious: A study by the German Retail Federation shows that online sales rose by 10 percent overall to reach 48.7 billion euros in 2017, with electronic goods and clothing being particularly dominant among the most popular product groups. To be on the safe side, people often order items in two or more versions, as they can return them easily – and often without paying shipping costs – if they don’t fit, if they don’t like them, or if the product is damaged. This leads to percentage rates of return in the middouble figures, presenting e-commerce businesses with an increasingly difficult challenge: At present, most of the returned packages have to be opened by hand in order to check whether they are complete, in good condition, damaged, or in working order, and then to sort them accordingly. Research into product returns shows that returning a package costs mail-order businesses an average of 15 euros due to processing costs and depreciation.

Fraunhofer’s Package Return Station is intended to put an end to this situation. This self-learning, robot-based cognitive sensor system consists of a combination of various nondestructive measurement systems, such as X-ray CT, ultrasonic sensors, or thermal imaging sensors, and allows returned packages to be imaged in three dimensions along with their contents. The special thing about is this system is that the package need not be opened during this procedure. The volume data is reconstructed, visualized, and analyzed by intelligent software systems and image-processing algorithms – including techniques based on mordern artificial intelligence (AI). This fully automatic process identifies wear and imperfections in the goods based on the digital image. In addition, the recorded information is compared with geometric target models, parts lists, and other important product-specific information so that further steps in the returns process, such as automated removal and sorting of the goods, can be initiated in the shortest possible time independently of the operator. This will speed up the entire returns process in the future, allowing e-commerce businesses not only to make considerable savings but also to respond faster to individual customers’ requests.

The Package Return Station is not the first project in which we have successfully worked on the digitalization of consumer goods for the e-commerce sector. In collaboration with Mifitto GmbH, a start-up from the Ruhr region, we have also developed the fastest commercially available 3D scanner for shoes, which provides customers with a unique, personal foot ID in a few simple steps. This personal 3D scan can then be compared with previously digitalized shoe models from an existing – and constantly growing – database to find a shoe that is a perfect fit for the foot. In this way, the number of shoe models ordered can be minimized in advance. In addition to the economic consequences, the environmental impact of returns is considerable. With CO2 emissions of 500 grams per package, transporting Germany’s 300 million annual returns produces 150,000 tons of CO2 emissions – not to mention the congestion on the roads.

In collaboration with Fraunhofer IIS, the Research Center for Modern Mobility at Deggendorf Institute of Technology (DIT) will construct the Package Return Station, put it into operation, and expand it with additional sensors to create a universal digitalization street based on cognitive sensor networks. This will allow the development of novel applications in the areas of mobility, trade, transport, and recycling.

Privacy warning

With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacy