Simulating human sensory perceptions with digital systems is the focus of research at the Campus of the Senses. In the future, how will the Campus be able to help people who have no sense of taste or smell, like Mr. Winter?
Andrea Büttner: Well, by developing various tools, the Campus of the Senses can help Mr. Winter to actually realize that he has a smell and taste disorder in the first place. The tools can then also tell him that his food has gone bad. People are often not aware that they have such a disorder, as it can happen quite gradually or may be caused by a “normal” illness, as in our example. With these diagnostic tools, we can also detect at an early stage if Mr. Winter has symptoms of Alzheimer’s or Parkinson’s, which are often associated with smelling disorders. Furthermore, we could develop assistance systems and sensors at the Campus that can then function as an artificial sense of smell to determine if the pasta sauce is still OK to eat.
Where did the idea come from to pursue this line of research? What senses are to be digitalized?
Andrea Büttner: The idea itself was obvious enough: we’ve already developed devices to support the functions of three senses – seeing, hearing and touch – with hearing aids and glasses being prime examples. The Campus of the Senses takes this a step further and concentrates primarily on the two “chemical” senses of smell and taste. What I’d love to be able to do would be to “record” the smell of maquis, the characteristic shrubland of Corsica, and “play back” the smell at home in my living room.
The research in this area is still in its infancy. What difficulties does it pose?
Jens-Uwe Garbas: There are several dimensions to the digitalization of the senses. It’s not just about digitalizing sensory perceptions, but also about how people interpret them, what they associate with them, and what feelings are evoked. Sticking with our maquis example, the challenge is to capture and evaluate this smell with a machine-based system, so that it can be, say, transported to another place.
What are the first steps for the Campus?
Albert Heuberger: The first step is to teach machines to understand the complex world of molecules. Currently, humans are much more sensitive in many respects than machines and have a much wider range of perception. There are very many different substances that produce odors. In certain cases, machines are able to detect even very small concentrations. Things become tricky, however, when the substances are present in a complex mixture and the machines are tasked with simulating how humans would actually perceive it.
The Campus of the Senses comprises Fraunhofer Institutes IVV and ISS and the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) as a university partner. These are all research institutes. How can private-sector companies benefit from the research?
Albert Heuberger: The Campus brings together the various kinds of expertise present within the three research institutions, which companies will often possess only in part – for example, we would expect companies in the food industry to know a lot about food, but they may have limited experience with machine learning. We offer support to such companies in line with their business models, working with them to develop new methods, services or technologies for simulating and interpreting human perceptions.
What do you personally find the most exciting thing about researching human perceptions?
Andrea Büttner: For me, the most exciting thing is how the research helps me understand myself better as a person and how we become aware of many human phenomena only when we try to translate them into a machine context.
Jens-Uwe Garbas: What I particularly like is that we have to think and work across disciplinary boundaries at the Campus. Innovations can quickly arise wherever two research fields overlap, provided that people approach the work with openness and a willingness to cross the “language barrier.” In my view, it’s precisely this element that makes the Campus so special, with its researchers from all different fields.
Albert Heuberger: We’re not yet able to capture taste and smell digitally. Creating new technologies that would also allow us to transport these impressions and make them available in other places is a fascinating challenge.
If you had one wish, what breakthrough would you most like to see?
Andrea Büttner: As the research into sensory perceptions is very complex, I’d like to fully understand my own perceptions and create greater awareness of our various senses. Our sense of smell is very underappreciated, for example, because we often respond to smells only at the unconscious level.
Albert Heuberger: I wish I had a cookbook that didn’t just have pretty pictures in it, but also supplied the aroma of the prepared meal before I decide whether to cook it.
Jens-Uwe Garbas: I want computers and machines that adapt to my needs in the future – and not the other way around.
Fraunhofer Institute for Integrated Circuits IIS