AI in Workforce Management
Reinforcement Learning Shift Planning Agent Set to Transform Hospital Staffing

Workforce management is key to making the best use of resources, especially with the exciting developments in artificial intelligence (AI) integration. AI-driven methods can make resource allocation more efficient and reduce manual work by creating customized schedules that follow both internal and external rules. The Fraunhofer Institute of Cognitive Systems IKS teamed up with ATOSS Software to create and test an automated personnel rostering agent using Graph Neural Networks (GNNs) and Deep Reinforcement Learning (RL) algorithms. This project gave us some interesting insights into scalability and performance.

Harnessing AI for Workforce Management

Workforce management is all about making sure employees are productive and in the right roles at the right times. It's also about assigning employees to the right tasks, which can have a big impact on things like how efficiently employees work and how happy they are. Some important things to consider are:

• Skill matching: Correctly assigning employees to tasks within their expertise can lead to better output quality and higher personnel well-being.
• Workload balancing: Equitably distributing tasks between staff members can reduce redundancies, minimize costs, and increase worker satisfaction.
• Flexibility: The ability to adapt to sudden changes, such as unexpected absences or changing business needs, leads to a more agile and stable organization.

Traditionally, domain experts with years-long experience manually create and periodically update schedules, considering external and internal rules and demands to minimize scheduling costs. Yet, the manual creation of individual schedules is time-consuming, challenging due to a high number and complexity of constraints, and often results in non-optimal schedules, e.g., when rest times are not respected. Furthermore, it is hardly feasible to reschedule at short notice in the event of unforeseen absences or extraordinary changes in business operations in compliance with all rules.

AI-based personnel scheduling can offer time savings, increased accuracy, flexibility, and enhanced compliance.

Shift Planning Agent Based on GNNs and Deep RL

In a recent research project, Fraunhofer IKS has joined forces with ATOSS Software to create an AI-based Shift Planning Agent aimed at optimizing hospital staffing through Reinforcement Learning (RL). This innovative tool is designed to generate personnel rosters tailored to hospital requirements, taking into account workforce demand, specific
wards, and prior shift schedules, all while navigating a complex array
of internal and external constraints. Compared to combinatorial optimization approaches such as linear programming and constraint programming, which do not scale well with
higher-dimensional input and often require extensive tuning for each use
case, AI models have the ability to learn from a variety of existing plans and
adapt to shifting business needs and dynamic situations.

The Shift Planning Agent developed by our team employs a Reinforcement Learning (RL) model that integrates Graph Neural Networks (GNNs) and deep RL algorithms to systematically generate personnel rostering schedules. This RL agent examines multiple scheduling configurations, optimizing for rewards that incorporate factors such as resource utilization costs. Through its iterative approach, the model continuously improves, adapting dynamically to evolving constraints and preferences. This pipeline yields a robust scheduling solution that enhances operational efficiency and supports better decision-making in hospital staffing, outperforming traditional shift planning methods.

Three findings from our research project are:

1. Reward scaling is crucial for convergence and RL-model performance.
2. (Real-time) monitoring tools are key in diagnosing the RL model.
3. The architecture of the GNN-based feature map can highly impact the obtained reward, that is, the main performance indicator of the agent.

Growth Directions and Important Considerations

AI-based workforce management presents significant automation potential, with Reinforcement Learning emerging as a promising strategy to prevent violations of shift planning constraints while exploring various scheduling options for cost minimization. In our project, we have examined various methods to address challenges in optimizing shift plan costs, such as managing high reward values and the extensive number of possible actions. Looking ahead, further experiments to enhance the capabilities of the RL Shift Planning Agent are planned.

Beyond our RL project example, we want to highlight two (out of the many) important considerations in using AI for workforce management and similar use cases:

1. Training data quality: AI requires high-dimensional, well-processed training data to generate accurate predictions or schedules. This involves not only the input features, but also the function that determines the relevance of those predictions.
2. Robustness and transparency: We see a continuous need and effort to introduce robustness and transparency into AI-based decision-making processes that allows stakeholders to understand why an AI model suggests a certain decision (e.g., resource allocation) over another. In our current RL Shift Planner Agent example, it is relevant to understand why hospital staff is assigned to one shift or workplace over another.

At the Fraunhofer-Institute for Cognitive Systems IKS, we are therefore developing frameworks and solutions that not only deliver accurate predictions for a particular use case, but also analyze contributing factors and employ methods to enhance AI explainability.