Publication date: 
2022/11/21
OI graduate Jiří Ulrich is in the final of the prestigious IT SPY 2022 competition with his diploma thesis "Multicamera localization system based on black and white pattern detection". In it, he presents a multicamera localization system that detects black-and-white circular WhyCode reference marks based on an improved algorithm, determines their position and orientation in space, and uniquely identifies them. Built on inexpensive and widely available cameras, the WhyCode system represents an affordable localization system as an alternative to current expensive and closed systems with specialized hardware.

In order to assess the performance, Jiří also tested the original single-camera system used in mobile and swarm robotics. For this purpose, he created a simulated environment that allows for dynamically changing scenarios, and he and his team collected a real dataset capturing the deployment of the system to track the position of a mobile robot during autonomous navigation in outdoor environments. The results of these experiments show improved accuracy and reliability using the presented system compared to its single-camera form.

Variants of the presented system are used to determine the position and orientation of robotic swarms by a number of European universities researching swarm robotics or to track bee queen activities within the European RoboRoyale project.

Available camera system for detecting and locating black and white marks

> What's the purpose of the system you created?

The system is used for precise tracking and positioning of fast-moving objects. Current systems, such as PTI Phoenix or OptiTrack, require special markers that are detected by expensive infrared cameras. These systems are very expensive, require long and complex set-up and maintenance, and have difficulty functioning in direct sunlight.

> So what is the biggest advantage of your system?

It's cheaper and easier because you only need a camera and a printer. No special equipment is needed and if the markers get damaged, you can just print or draw new ones. Our system can be up and running in minutes, works in direct sunlight and has a longer range than similar methods.  The system is also available on the Android platform, so you just need to download a small app to test it. The big advantage is the open source code, so everyone can customize it as they want.

> What can this technology be used for in practice?

The technology can be used in a variety of applications for locating and forming autonomous drone formations, augmented reality, determining the position of drones when interacting with each other or the environment, or simply any time you need to know the position of an object on which a tag is attached. Different variants of the system have been and are being used by a number of universities around the world. University of Pennsylvania uses our tags for research on drones and smart materials, Reykjavik University has created a modified version of the tag to locate maritime drones, University of Lincoln uses tags to locate mobile robots, tracking the movement of social insects is implemented by tags at University of Graz, and University of Manchester uses tags to locate and evaluate the activity of swarm robots.

We have also established contact with several technology companies and especially their development department. Synergy Logistics tested the application of tags in an autonomous logistics warehouse, SAGA used tags in robotic agriculture, Zuri explored the use of tags in a VTOL landing maneuver, and Enchanted Tools is working on precision interaction with humanoid robots. Furthermore, this technology can be found in the European research projects STRANDS, HAZCEPT and in the current RoboRoyale project, which combines microrobotics, biology, ethology and chronorobotics to stabilise and support endangered ecosystems.

> How long have you been working on this topic?

Specifically, the WhyCode system has been a topic of mine for quite some time, as I have already dealt with it in my bachelor thesis. In it, I focused on reducing the tag localization error while increasing the stability of identification when using a system with only one camera.

> And what are your plans for the project in the future?

It's currently part of my PhD and a good jumping off point as there is publishing interest in the topic. The big attraction of WhyCode is its diverse applications and the people who use it. What I also enjoy about the research is that it links robotics and computer vision.

You can vote for Jiří in the IT SPY, People's Choice Award until 22 November!

https://www.itspy.cz/rocnik/2022/