The way forward demands cooperation
Collaborative driving maneuvers as a success factor for autonomous driving

Whether on a bicycle, in public transport or in a car: we are all happy to get from A to B as quickly as possible and preferably without stress. But the reality in many cities and metropolitan areas today looks usually different: The flow of traffic grinds to a halt or even comes to a complete standstill, for example due to an accident or congested roads. In the city of the future, things are supposed to be different: Here, intelligent traffic systems will guide the traffic and optimize the traffic flow. That sounds easier than it is. There are still technological hurdles, that need to be overcome. One of them is the communication amongst traffic participants and with the traffic infrastructure.

To enable us to experience this new, more relaxed traffic world, João-Vitor Zacchi researches at Fraunhofer IKS on collaborative perception of autonomous vehicles, which combines the perspectives of vehicles and traffic infrastructure. This research project is funded by the European Union as part of the Horizon 2020 program.

Different types of perception for decision-making

»There are basically three different types of perception: decentralized, centralized and hybrid perception,« explains João-Vitor Zacchi in our conversation. In decentralized perception, the sensor-based decision-making takes place directly in the individual car. In this case, communication with other vehicles is not necessary because each vehicle has a complete perception chain. With centralized perception, it is the opposite: Here, the traffic infrastructure makes decisions based on a combination of its own sensor information and that of the vehicle. In this case, reliable communication is required for the exchange of information. The vehicle itself does not make an independent decision, but acts based on information from the central system. Therefore, the car also has only parts of the complete perception chain. João-Vitor Zacchi's research focuses on the third type of perception, the hybrid perception. This uses information from the traffic infrastructure to improve the perception of the individual vehicle. Decision-making still lies with the individual autonomous vehicle. With hybrid perception, the vehicle has a complete perception chain so that it can also move safely in areas where no intelligent infrastructure is yet in use.

»The challenge with hybrid perception is a powerful data association, such as the classification and tracking of detected objects. Only then can an autonomous vehicle derive appropriate actions in complex situations, for example at a crowded intersection in a city,« explains João, the underlying problem of his research question. »Due to the required interaction with non-connected road users for example pedestrians, such situations become even more complex. Successful autonomous driving systems must be able to deal with this. In particular, the behavior of humans is often difficult to predict.«

New framework for hybrid perception

To solve these challenges the Fraunhofer IKS researcher has developed a proposal for a framework that will simplify the classification of other traffic participants and thus the tracking of detected objects. »Autonomous vehicles need to correctly recognize different objects on the road and calculate where they will be next. For this, it is important to know how fast an object is moving and in which directions it can move,« summarizes João-Vitor Zacchi. With other connected traffic participants, this is relatively simple: by communicating and exchanging data with each other, an autonomous vehicle knows how fast another autonomous vehicle is traveling. In addition, an autonomous vehicle has only limited directional options due to the given road layout. A car for instance, can only go left, right or straight at an intersection. For pedestrians or cyclists however, the situation is different. Here, the autonomous vehicle can only collect information on speed based on its own sensor measurements. Also, pedestrians and cyclists can continue moving in any direction from their current location. Pedestrians and cyclists can theoretically cross an intersection diagonally or travel on the sidewalk against the actual direction of travel.

The framework developed by Fraunhofer IKS therefore uses a Bayesian Occupancy Filter to simplify object detection and tracking. This basically grids the environment of a vehicle and blocks all positions that could be occupied by another traffic participant.

For fast decision-making, this gridding is deliberately very coarse in the first step. »The use of a standardized Bayesian Occupancy Filter initially leads to comparatively low accuracy, since the prediction assumes that a traffic participant can in principle move in any conceivable direction,« explains Zacchi. »In the next step, the object is then classified more precisely, for example as another vehicle or pedestrian. This allows the system to continuously refine the position of other traffic participants.« Individual positions on the surrounding grid are thus released, giving the autonomous vehicle additional room to maneuver.

Comparison to other peception approaches planned

Fraunhofer IKS's proposed framework for hybrid perception combines different perspectives on a traffic situation to increase the contextual knowledge of individual autonomous vehicles and maintain their independence in terms of decision-making. Fraunhofer IKS researchers have already implemented and tested this individual perception chain on individual vehicles in a simulation environment. »In a next step, we are now introducing computation and fusion using the Bayesian Occupancy Filter to be able to evaluate how our approach compares to individual perception chains and other collaborative approaches.«

So, it will take a while before different traffic participants communicate with each other and we can get from A to B fully autonomously and stress-free. »We will see a gradual development toward full autonomy over the next few years. In the beginning, there may be selected roads or areas designed for fully autonomous driving, especially in cities. However, there is still a long way to go before all national and international road traffic is covered,« concludes João-Vitor Zacchi.

For further information on the proposed framework feel free to connect directly with João-Vitor Zacchi or download his paper presentation from the European Dependable Computing Conference 2021.