Listen to this article

Space exploration imposes the highest demands on remote-controlled and autonomous systems. In September, university teams from around the world participated in the ERC Space and Robotics Event, Europe’s largest competition of its kind. The ongoing coronavirus pandemic required the event’s organizers to quickly find a way for the teams to participate and be monitored remotely. Freedom Robotics ultimately provided the platform for competitors to teleoperate their robots in simulated Martian terrain.

“The European Rover Challenge is held every year in Poland and is typically more hardware-focused. They build a landing site and mimic tasks,” said Achille Verheye, lead robotics engineer at Freedom Robotics. “This year, COVID-19 made travel impossible, and our solution was introduced to the competition through partners.”

“It wouldn’t have been possible to hold the competition to the same standards as previous years without a tool to remotely control the robots and to develop on them,” said Lukasz Wilczynski, president of the European Space Foundation.

Freedom Robotics enabled plug-and-play competition

“With our system, stock hardware can run their software remotely from all over the world,” Verheye told The Robot Report. “We didn’t have to do any customization. The teams were like our startup customers focused on debugging, and the judges were like our enterprise customers, which are managing fleets.”

“One of the two main tasks in the ERC challenge was using a Mars rover to drive around a field,” he explained. “It was the same for all teams, which had to improve on the software to drive manually or load algorithms and use Freedom to work on those algorithms and check inputs and outputs.”

“Another part of the competition was remote controlling a UR robot arm,” said Verheye. “They could only see what was on the camera on the table close to the robot. Seeing only part of the data was harder. The teams used an Xbox controller to control the end effector to conduct three tasks, including plugging in a power cable or using a panel like on the International Space Station or in building a base on Mars.”

“Freedom‘s support team was indispensable for making the competition run smoothly. Being forced to have the teams remote was a blessing in disguise in some ways,” said Krzysztof Walas, assistant professor at the Poznan University of Technology. “It actually forced us to replicate similar scenarios as driving robots on Mars, where teams can only see the robots from the robot‘s camera perspective.”

With only a 20-day turnaround, how much training on the company‘s platform did the teams need for teleoperation or loading algorithms?

“It was pretty light — the ERC competition had a simulation environment, and we gave a one-hour webinar,” said Nicholas Cortes, head of business intelligence at Freedom Robotics. Cortes studied mechanical engineering at Columbia University and dealt with the competition for the company.

Simulation, streaming, and performance

“We helped with the simulation environment so it would work the same day,” Verheye noted. “The ROS [Robot Operating System] topics were the same as for the Freedom Robotics platform. In the case of real world versus simulation, they had already scanned the field. Even that wasn’t a new use case for us, since we have customers that create simulated environments to test before hardware comes.”

With up to 40,000 live streams of the competition, the ERC organizers needed software that could scale. “Our infrastructure is built to scale with such a load,” said Verheye. “Teams were jumping in and out, so they were not pushing our limits.”

“The time difference did pose a challenge for communications and spectators, with teams in different rooms and time zones, just as it would for space,” he acknowledged. “It was amazing to see our footage in first-person view, as well as judging cameras around the field. The organizers would hide artifacts such as a nozzle in the ground and award teams a bonus if they found them.”

Verheye compared the rover challenge to gamification for robotic delivery or pick-and-place operations. “It’s something we’re actively working toward, like providing statistics on a screen to show how well the operator and robot are performing in one location versus another location,” he said. “We can see stats for operators trying to beat certain delivery times, as well as for fleets.”

“Industrial robots fail quite a lot,” Verheye said. “Lowest-level operators might not notice or just fix it. With Freedom Robotics, both operators and managers can see which robots are causing problems, and they can see huge improvements.”

Working toward a future of full autonomy

Freedom Robotics’ platform can help robots become more autonomous, claimed Verheye. “There’s an analogy with driverless cars,” he said. “While they’re not widely approved until they get it 100% right, Tesla got out on the roads right away to collect data. We’re giving tools to help operators intervene.”

The use of a video game controller for teleoperation was another novelty that has industrial implications, said Verheye. “The teams were the first to test it, and it really opens up a lot of customers,” he said. “For instance, with welding robots, you could previously only monitor an industrial arm and send fixed commands, but now you could just take over the robot, troubleshoot, and keep the production line running.”

Freedom Robotics’ technology worked so well that the European Rover Challenge expects to use it again next year.  “Quarantine or no quarantine, the next edition will have the remote-control aspect to it as well,” said Wilczynski.

“We’re super-excited about next year,” Verheye said. “Even if everything reopens, we will still try to put the teams in different rooms. We enabled competitors and organizers to have visibility, just with smaller companies and our enterprise customers.”