As artificial intelligence penetrates more and more areas of life, the need to understand why decisions are made in a certain way is also growing. The award is all the more valuable because the winner was not decided by a predetermined jury, but by the conference participants themselves - the global community of researchers involved in automated planning and decision-making in artificial intelligence. Among the twelve selected demonstration projects from leading world universities and research institutions, PANSim received the most votes.
“This is an extremely valuable award for us, because ICAPS is the main global conference dedicated to automated planning. Unlike large conferences focused on artificial intelligence, it brings together practically the entire professional community of our field. We appreciate it all the more that its participants recognized our tool as the best demonstration of this year,” says doc. Lukáš Chrpa from the Department of Industrial Informatics of the Czech Institute of Informatics, Robotics and Cybernetics, CTU.
When it's not enough to know that AI made the right decision
Today, artificial intelligence helps control robots, autonomous vehicles or industrial systems. However, it is often much more difficult than verifying the correctness of the result to understand why the algorithm made the decision in this way. Even the developers themselves cannot always read complex decision-making processes from text logs or program statements alone.
PANSim converts artificial intelligence decision-making into an interactive graphical simulation. It focuses on the area of so-called symbolic artificial intelligence, which instead of statistical learning works with explicit modeling of the environment and planning of individual steps. The user can watch step by step how the agent evaluates the situation, reacts to changes in the surrounding environment and finds a safe path to the goal. Thanks to this, it is much easier to detect errors in the algorithm, understand its reasoning and further improve the entire system.
“It’s not just about the graphics themselves. The visual form allows a much better understanding of what is actually happening during the decision-making process. Thanks to this, the developer quickly recognizes why the agent behaved in a certain way and where the algorithm needs to be adjusted,” explains Ing. Jakub Med, supervisor of Erol Medenčević’s bachelor’s thesis and a doctoral student at the Faculty of Engineering and Technology of the Czech Technical University, who also works in the Department of Industrial Informatics at the CIIRC of the Czech Technical University.
How to show the complex decision-making of artificial intelligence in the most understandable way? The authors chose a simple situation. A frog must get across a pond and collect all the coins. However, finding the shortest path is not enough – some water lilies can disappear below the surface at any moment. Therefore, the agent must continuously reassess its options and respond to random changes in the environment. The second scenario simulates autonomous underwater vehicles that must safely respond to the movement of ships above the surface when performing a mission. In addition, the team requires the agent to have a guarantee of success, which distinguishes the techniques developed by the team from traditional ones using machine learning algorithms.
From Bachelor's thesis to further research
The PANSim project is not the first international success of FEE CTU student Erol Medenčević, who continues its development during his subsequent master's studies. Earlier this year, it was presented at the prestigious AAAI conference in Singapore. While AAAI is one of the largest world conferences focused on artificial intelligence with thousands of participants, ICAPS is the main global forum for specialists in automated planning.
"After being accepted to the first world conference, I understood that the project has a real impact. But when my colleagues sent me a photo of the diploma for the best demonstration from Dublin, I was really surprised. It is a great motivation to continue," says Bc. Erol Medenčević.
In the next phase of development, the authors want to make the agents' decision-making even clearer. For example, it will be possible to return the simulation a few steps back, better display the current intentions of the artificial intelligence, or expand the tool with other scenarios. The long-term ambition is to create a universal environment that will allow for the visualization of a wide range of planning tasks.
Although PANSim is primarily for researchers, the principles it uses have applications in areas such as autonomous robot control and cybersecurity, where systems must respond to unpredictable changes in their environment. A better understanding of how artificial intelligence thinks and why it makes certain decisions will become increasingly important as the number of autonomous systems increases.