Cognitive systems

The performance of machine learning (ML) models is typically assessed using metrics such as accuracy, precision, recall, F1 score, etc. But how trustworthy are these single point estimates really? Such metrics reflect reality only partially and can provide us with a false sense of safety. Especially in safety-critical areas, this can be dangerous. Fraunhofer IKS develops a formal framework for the quantification of uncertainty which aims to make the residual risk “more computable”.

Safety is a property of a system and its environment. That's why predicting a system's context is so important to safety engineers. However, we must acknowledge that it is impossible to sufficiently predict and specify a system's context when it must operate outside of a controlled environment. In the previous parts of this series, we've seen how we can use self-adaptivity to build systems for uncertain, dynamic contexts. In this fourth and final part, we add the missing ingredient by taking safety engineering to the next level.

Resilience is a key ingredient for lifting cars, industrial manufacturing and medical devices to the next level. It is far more than a fancier synonym of dependability. And the aspect that resilience is standing for is essential for building cognitive systems. That's why resilience is the focus of a new series on the Safe Intelligence Blog. In the first part, we will look behind the term and clarify why resilience is so important and why it is more than just a synonym for dependability.

Software development is a comparatively young, but complex and quickly evolving field. One of the more recent additions to the common portfolio of software architects is cloud technologies. In general terms, cloud technologies decouple software and the resources required for running. This allows services provided by the software to be quickly scaled according to demand. Nevertheless, there are several pitfalls that can be encountered when integrating cloud technologies into your software architecture.

Safe intelligence is the central research topic of the Fraunhofer Institute for Cognitive Systems IKS. This topic is very important from a technology perspective as well as with regard to the economy and society. The two Chairs of the Advisory Board for the IKS development project see it the same way — each from their own perspective.

Fraunhofer IKS has established an advisory board to provide expert support to the IKS development project. The advisory board convened for its first regular meeting on April 14. The two Chairs, Thomas Gallner and Prof. Thomas Seidl, talk about their motivation for getting involved. And about the aspects on which they intend to focus.

There’s nothing Ania Kosmalska likes more than taking a computer program and a few electronic gadgets and experimenting in her daily life. But these days, she devotes a large part of her free time to her young daughter. Part two of our employee profile series.

Science needs to prove that systems based on artificial intelligence (AI) are safe enough. That is what Prof. Simon Burton, who recently became Division Director for Safety at Fraunhofer IKS, is calling for. In this interview he explains the institute’s approaches to ensuring safety.

Reconciling safety, performance and reliability in cloud-based cognitive systems is no easy task. Especially in safety-critical autonomous systems such as driverless cars, expectations rightly assume that all three requirements are implemented at the highest level. However, this comes up against limited resources. The Fraunhofer IKS has developed an approach for optimising safety, performance and reliability depending on the current situation and concrete applications: flexilience.

Deep Neural Networks (DNNs) have proven their potential. This applies above all to their use in the laboratory. Their performance in real-world applications leaves a lot to be desired. One major problem is the estimation of uncertainty. A closer look at methods that meet this challenge.

Flexible service-oriented architectures (SOAs) in safety-critical applications have been virtually out of the question until now. This is because services need to be reliably available and safety requirements need to be met at all times. Fraunhofer IKS has developed resilientsoa, a software framework that meets these needs by ensuring that service orchestration is as safe as necessary while performing as well as possible.

Addressing the wide range of challenges in dependable cloud-based cyber-physical systems of systems (CPSoS) requires new approaches that are automated and efficient enough to shift some of the system’s elements to runtime. To design and evaluate the system architecture, Fraunhofer IKS has developed an iterative and automated process.

The future will rely on widespread, massively-connected, highly-intelligent systems. To implement this vision, we need dependable cloud-based cyber-physical systems. Research in this direction encounters as many opportunities as challenges. Such systems exist in changing environments and interact with other systems and humans, which requires intelligence, autonomy, safety and adaptability – in fact, conflicting goals. An example of this is autonomous driving.

Machine learning means a disruptive challenge for safety assurance. Without an acceptable safety assurance concept, many great ideas will not find their way to the market. It’s therefore no surprise that AI safety has gained much more attention over the past few months. A brief series of articles will take a closer look at the importance of safety for AI. The first article focuses on the basic understanding of what safety actually means. It highlights why it is so important to understand that safe AI has less to do with AI itself and much more to do with safety engineering.

In this interview apl. Prof. Dr. habil. Mario Trapp, Director of the Fraunhofer Institute for Cognitive Systems IKS, talks about the competencies of the institute that make cognitive systems safe.