Artificial intelligence

Modern AI models have demonstrated remarkable capabilities on various computer vision tasks such as image classification. Their decision-making process, though, remains mostly opaque (black box problem). This can be particularly problematic for safety-critical applications like medical imaging or autonomous driving. Fraunhofer IKS explores ways to increase the transparency of this black box.

Achieving safety with AI systems requires a comprehensive and collaborative approach, including technical, ethical, and regulatory considerations. A central aspect is designing AI systems that comply with safety requirements defined in constraints.

What are the basic principles of self-adaptive systems? This is the basic prerequisite for bringing the concept of resilience to life. Part 3 of our series.

Using artificial intelligence (AI) in the rail industry could potentially increase both efficiency and quality. At a workshop hosted by Fraunhofer IKS, experts from Deutsche Bahn, Siemens Mobility and the institute itself gathered to discuss the possible uses and challenges associated with AI in rail applications, as well as strategies for ensuring system safety.

If uncertainties in an autonomous system are not dealt with, they impair the system’s functionality and can lead to safety risks. In part 1 of our series on uncertainty, we presented a system for classifying uncertainties. In part 2, we will look at specific sources of uncertainty that have been noted in the relevant literature.

Resilience is the key ingredient for the next generation of cognitive cyber-physical systems. After all, many visions of the future can only become reality if the systems become aware of themselves, their goals and their context in order to be able to adapt to unknown and changing environments. And in doing so always optimize utility while preserving safety. But resilience can only be bought with increased complexity. In this article, part 2 of our series on resilience, we look at the character of complexity to better understand where to start engineering in order to manage this complexity.

Cognitive systems are often affected by the problem of uncertainty, particularly if those systems use machine learning to perceive their environments. A system’s sensors are not the only source of uncertainty, however. The life cycle of any highly complex system involves a large number of decisions that are vulnerable to the effects of uncertain knowledge. Part 1 of our series on uncertainty focuses on the latest developments in this technology.

While many current CDS expert systems (Clinical Decision Support) in laboratories are being criticized due to difficulty in scaling, constant review from the medical perspective and limited data cases, artificial intelligence (AI) has become a highly plausible alternative. »Safe Intelligence for Laboratories«, a webinar hosted by Fraunhofer IKS, focused on the cases and challenges of AI in laboratories from the perspectives of both industry and patients.

Artificial intelligence (AI) offers a lot of potential for Industrie 4.0, but there are technological barriers that make it more difficult to implement. Researchers at Fraunhofer IKS are developing a framework to support and optimize the data and AI life cycle. This will significantly increase the scope of value creation provided by AI.

How can intelligent systems learn to perform tasks – and do so simply by interacting with their environment? This is the subject of reinforcement learning, a subfield of machine learning. Problems occur primarily when situations arise that the system is not yet familiar with.

Can deep neural networks in image processing reliably recognize new, unknown test patterns? First considerations for different methods — especially for safety-critical applications.

Deep learning has boosted the interest in deploying artificial intelligence (AI) to more systems. Reinforcement learning (RL) has also been through a revival, with Deep RL models designed to play Atari games and even beat grandmasters at the game of Go. However, despite these recently obtained impressive achievements, RL is still struggling to be accepted and successfully deployed in some applications, especially in the case of safety-critical tasks such as robotics, autonomous driving and industrial control.

In the near future, medical engineering will be shaped by artificial intelligence and machine learning. However, the complex processes involved in machine learning create a problem for researchers — it is costly and challenging to eliminate errors before autonomous systems are put into use, and to reliably detect errors during operation.

When ensuring the safety of automated driving systems, it is important to take into account technological as well as societal and legal aspects.

As we move toward the “smart city”, sensors are set to also collect data on air quality, traffic, etc., and — in the long term — provide us with cleaner air and help us get where we want to be faster. The Fraunhofer Institute for Cognitive Systems IKS offers the necessary technologies for the safe operation of artificial intelligence (AI) and provides cities and communities with a framework to help them get started.

A sticker on a give way sign. Branches hanging in front of a stop sign. Graffiti on a speed limit sign. These are all completely normal sights on our roads, aren’t they? But things that wouldn’t generally be a problem for humans can really make life difficult for artificial intelligence.

In modern perception applications, such as in medical engineering, models based on artificial intelligence (AI) are increasingly being used due to their strong performance. However, this increasing performance often comes at the cost of the transparency of results. An online tool developed by the Fraunhofer Institute for Cognitive Systems IKS can help here.

Safe intelligence is the central research topic of the Fraunhofer Institute for Cognitive Systems IKS. This topic is very important from a technology perspective as well as with regard to the economy and society. The two Chairs of the Advisory Board for the IKS development project see it the same way — each from their own perspective.

Fraunhofer IKS has established an advisory board to provide expert support to the IKS development project. The advisory board convened for its first regular meeting on April 14. The two Chairs, Thomas Gallner and Prof. Thomas Seidl, talk about their motivation for getting involved. And about the aspects on which they intend to focus.

Reinforcement learning (RL) is one of three main machine learning (ML) paradigms. Unlike the other two, supervised learning and unsupervised learning, in RL the learning takes place in direct interaction with the surroundings. A brief introduction to RL follows.

Artificial intelligence (AI) has to be able to handle uncertainty before we can trust it to deliver in safety-critical use cases, for example, autonomous cars. The Fraunhofer Institute for Cognitive Systems IKS is investigating ways to help AI reason with uncertainty, one being the operational design domain.

Fraunhofer IKS is carrying out research into safe software applications for quantum computers. The work is part of the new Bavarian initiative Munich Quantum Valley.

Science needs to prove that systems based on artificial intelligence (AI) are safe enough. That is what Prof. Simon Burton, who recently became Division Director for Safety at Fraunhofer IKS, is calling for. In this interview he explains the institute’s approaches to ensuring safety.

When it comes to environmental protection and climate action, most people think of renewable energies, electric cars and sustainable consumer goods. Very few would associate it with artificial intelligence (AI), but in fact AI has a crucial part to play in the fight against pollution and climate change.

Deep Neural Networks (DNNs) have proven their potential. This applies above all to their use in the laboratory. Their performance in real-world applications leaves a lot to be desired. One major problem is the estimation of uncertainty. A closer look at methods that meet this challenge.

On the automotive market, traditional manufacturers are in a head-to-head race with new players from the software world. In the end, the winner will be decided not only on the basis of technological innovation, but also by how secure the technologies are.

Machine learning means a disruptive challenge for safety assurance. Without an acceptable safety assurance concept, many great ideas will not find their way to the market. It’s therefore no surprise that AI safety has gained much more attention over the past few months. A brief series of articles will take a closer look at the importance of safety for AI. The first article focuses on the basic understanding of what safety actually means. It highlights why it is so important to understand that safe AI has less to do with AI itself and much more to do with safety engineering.

Machine learning, and especially deep learning, can be used to enable many highly complex applications, such as autonomous driving. However, there are new challenges that need to be overcome to secure such systems. An overview.

Today, autonomous vehicles function reasonably well in test situations since the conditions are severely restricted and thus easy to manage. A key issue however is how to design autonomous vehicles so that they operate dependably even in complex and previously unknown situations. A solution from Fraunhofer IKS is helping to uncover and predict such difficult situations.

In autonomous systems or medical engineering, uncertainty in machine learning (ML) processes needs to be reliably determined. That is why we are comparing four state-of-the-art methods of quantifying the uncertainty of deep neural networks (DNNs) in the context of safety-critical systems — so that the AI can learn when there is scope for doubt.

Complex machine learning (ML) processes present researchers with a problem: Eliminating errors before autonomous systems are put into operation and reliably identifying errors at runtime are laborious and challenging processes. At present, this severely restricts the use of machine learning in safety-critical systems and necessitates new approaches in order to benefit from the advantages offered by ML in these areas in the future.

In this interview apl. Prof. Dr. habil. Mario Trapp, Director of the Fraunhofer Institute for Cognitive Systems IKS, talks about the competencies of the institute that make cognitive systems safe.