All articles

The future will rely on widespread, massively-connected, highly-intelligent systems. To implement this vision, we need dependable cloud-based cyber-physical systems. Research in this direction encounters as many opportunities as challenges. Such systems exist in changing environments and interact with other systems and humans, which requires intelligence, autonomy, safety and adaptability – in fact, conflicting goals. An example of this is autonomous driving.

Machine learning means a disruptive challenge for safety assurance. Without an acceptable safety assurance concept, many great ideas will not find their way to the market. It’s therefore no surprise that AI safety has gained much more attention over the past few months. A brief series of articles will take a closer look at the importance of safety for AI. The first article focuses on the basic understanding of what safety actually means. It highlights why it is so important to understand that safe AI has less to do with AI itself and much more to do with safety engineering.

Machine learning, and especially deep learning, can be used to enable many highly complex applications, such as autonomous driving. However, there are new challenges that need to be overcome to secure such systems. An overview.

Today, autonomous vehicles function reasonably well in test situations since the conditions are severely restricted and thus easy to manage. A key issue however is how to design autonomous vehicles so that they operate dependably even in complex and previously unknown situations. A solution from Fraunhofer IKS is helping to uncover and predict such difficult situations.

In autonomous systems or medical engineering, uncertainty in machine learning (ML) processes needs to be reliably determined. That is why we are comparing four state-of-the-art methods of quantifying the uncertainty of deep neural networks (DNNs) in the context of safety-critical systems — so that the AI can learn when there is scope for doubt.

Complex machine learning (ML) processes present researchers with a problem: Eliminating errors before autonomous systems are put into operation and reliably identifying errors at runtime are laborious and challenging processes. At present, this severely restricts the use of machine learning in safety-critical systems and necessitates new approaches in order to benefit from the advantages offered by ML in these areas in the future.

In this interview apl. Prof. Dr. habil. Mario Trapp, Director of the Fraunhofer Institute for Cognitive Systems IKS, talks about the competencies of the institute that make cognitive systems safe.

Dr. Sabine Sickinger, Director of Organizational Strategy and Administration at the Fraunhofer Institute for Cognitive Systems IKS, talks in the following interview about how dynamic research work succeeds through agile project organization.