Multimodal sensor-based assistance systems

© MatBeyoNDT
Possible visualization for the overall condition of a fish farm

The industrial and production landscape in Germany is changing. Manual and handcrafted activities are increasingly being replaced by technically sophisticated processes with a high degree of automation and networking. This change in the world of work makes it necessary to rethink the interfaces to production facilities.

The technologies of the future represent complex systems, essentially characterized by a constantly growing flood of influencing variables and sensor data. These multidimensional data spaces require simple, intuitive access to the control and regulating variables: An assistance system that is able to tangibly visualize multifactorial influencing variables, the status of individual components or the overall system and enable the user to take control by means of target-oriented instructions and options for action.

A current research focus at Fraunhofer IZFP is therefore on the development of intuitive multimodal sensor-based assistance systems in order to do justice to this change in the industrial landscape. The latest methods of artificial intelligence and data processing are used here to create solutions that make it possible to adapt even the most complex systems quickly, easily and reliably to customer-specific problems.

A large number of sensors and measured variables are used in a wide range of applications. The systems developed at Fraunhofer IZFP are able to cover a wide range of functions, from the evaluation of a general "system status" to specific suggestions for action. In collaboration with partners who specialize in user-centric UI, Fraunhofer IZFP develops solutions that make it possible to demonstrate complex relationships in a comprehensible way and make highly technical systems accessible to people who are not experts in the field.

Multimodal sensor-based assistance systems are not only relevant in the course of digitalization in industry, where the need for such systems is constantly increasing, but also enable the expansion of target groups, for example in agriculture or the expansion of urban infrastructure. Intelligent assistance systems can and will therefore find their place in all sectors of the economy. In addition to industry, a sharp increase in technically sophisticated systems can be observed in agriculture and related sectors as well as in supply infrastructure. The use of sensor-driven operational assistance is an added value everywhere: to avoid operating errors, to increase efficiency and yield and - depending on the environment used - to increase animal welfare, job satisfaction and the stabilization of supply services.