Neil Tardella | Energid, Author at The Robot Report

Telerobotics offers automation at arm’s length for healthcare, manufacturing, and more

Neil Tardella | Energid — Sat, 17 Oct 2020 14:00:30 +0000

Telerobotics — the area of robotics focused on controlling robots from a distance — will play an increasingly important role in society in the wake of the COVID-19 pandemic. Telerobotics lets a remote human operator program, control, and monitor one or more robots. This means a surgeon in New York can perform a robot-assisted surgery on a patient in India, or an engineer in London can program and monitor a robotic workcell in Shenzhen, China.

The ability to decouple a human (the brain) from a robot (the body can physically interact with its environment) can be transformative, especially as we look to maintain productivity while keeping a safe distance from one another and travel is highly restricted.

Telerobotics solutions in healthcare

The healthcare challenges of early 2020 highlighted the need for telerobotics. Many countries were caught unprepared at the peak of the outbreak, with several hospitals being overrun by COVID-19 cases, struggling with shortages of qualified healthcare providers and personal protective equipment (PPE).

Telerobotic solutions, had they been available, could have been a force multiplier. Hospital-based telerobotic systems controlled by remote healthcare workers would take providers off the front line and out of reach of the virus. Such systems could be used for medical material handling and diagnostics, such as taking a patient’s vital signs or administering vitally needed COVID-19 tests. The more we deploy this technology now, the more prepared we’ll be for the next crisis.

Robotic telehealth solutions are valuable for many medical applications off the frontline as well. One example is remote diagnostic ultrasound, where a robotic system holding an ultrasound transducer is remotely controlled by an ultrasound technician or radiologist.

Energid Technologies worked with Dr. Jeff Soble and Sarah Doherty, co-founders of startup Telehealth Robotics, to develop a system for medical kiosks, where patients could go to a local kiosk to get a carotid ultrasound performed by a remote technician. In the video above, the subject was in Chicago, and the operator was in Boston.

Related content: The Robot Report Podcast: Robot-assisted surgery, Amazon’s security drone, and life-size Gundam

Telerobotics in industry

Though telehealth applications are the first to gain popularity, they certainly won’t be the last. Social distancing is forcing us to rethink manufacturing as well. The days where factories are filled with tightly packed workers are likely coming to an end, and industries must determine how to reduce human density without the commensurate reduction in productivity.

Here again, telerobotics can help. Industrial robots that can be remotely controlled, programmed, and monitored allow manufacturers to maintain productivity even with a significant percentage of the workforce offsite. This allows reduced human density at manufacturing facilities with “smart” automation systems that rely on remote workers for guidance — injecting human intelligence into otherwise “dumb” automation systems.

While replacing human workers with automation often means reducing productivity per square meter — since robots take more floor space than humans — advanced software that allows robots to work in close proximity to one another can bring productivity levels back up to human level.

Automation solutions with multiple robots in overlapping workspaces yields significant floorspace savings for the customer, but also increases challenges for robot motion control. Real-time adaptive motion control software lets robots work collaboratively and avoid collisions, even in dynamic conditions.

To accomplish this, all robots in the shared environment must be aware of each other and the trajectory of each predicted in advance. This isn’t knowable at the time of programming for applications where random parts are presented, such as when picking parts from a bin for a machine-tending operation or sorting parts from a conveyor in preparation for downstream kitting. In these cases, the software schedules motion in a collision-free manner by assessing the trajectories of the other robots and planning motions appropriately in real time.

Modes of telerobotic operation

Telerobotics applications typically fall into one of three high-level modes of operation, with many moving between these modes depending on the application.

During programming of the robot task, for instance, a human often works closely with the robot to set and test waypoints and tasks that will ultimately become the robot program. A technician may be co-located with the robot during these high-touch programming components, but that technician doesn’t need specialized knowledge of the robot or application.

Autonomous operation with monitoring

During normal operation, a human is not required to intervene in a robot’s workflow. Monitoring can typically be a hands-off activity, and well-designed systems can allow one remote operator to monitor several automation systems from a distance.

Semi-autonomous control

With semi-autonomous control, a robot performs automatic motions but periodically engages a remote operator for decision making or pattern recognition. An example is an inspection robot that sends low-confidence images back to a human for review.

Manual control

With manual control, a human takes direct control of the robot to perform a task. An example is a robot used for diagnostic ultrasound, where a remote radiologist directly controls a robot equipped with an ultrasound transducer.

Often, all three of these modes are used in coordination. Manual mode may be used to program the robot and to intervene when automated motion gets stuck, for instance. And remote operators may check diagnostics periodically and be alerted when an automated decision-making algorithm (i.e., for inspection) gets confused.

Telerobotics technology enablers

Developing a telerobotic solution requires careful planning and some key technology enablers.

Safe robots

Robots designed for remote operation have less onsite support to fix problems, apply preventive maintenance, and reset controllers. The robots may also have contact with humans who are not familiar with them, such as with patients and technicians in telehealth situations, or workers in a manufacturing facility.

Robots with built-in safety mechanisms protect nearby humans as well as the robot itself and the equipment around it. Starting with a robot that is fundamentally safe — such a collaborative robot or cobot that already has extensive safety features built in –reduces the chance of a catastrophic system failure and makes building the system less complex.

Software can add another layer of safety for remotely operated robots by allowing the operator to create virtual “keep-out” zones. The keep-out zones prevent a remote operator from unintentionally moving the robot into an area containing fragile equipment, for instance. This software can also be used to make sure the robot can’t collide with itself — something that often happens during remote programming and less-structured applications.

Companies in the remote healthcare space are developing safe, intuitive telerobotic applications, using real-time adaptive motion control software that provides mechanisms for creating these virtual keep-out zones, adding a layer of safety for remote robots being guided in manual control mode.

Remote-control infrastructure

Controlling a robot remotely requires an infrastructure that supports two-way communication between the remote operator, the robot, and any tools required for the procedure. Before high-speed networks, connectivity and bandwidth issues made direct remote control of a robot virtually impossible.

Today’s modern infrastructure makes telerobotics much more viable, and upcoming 5G will provide a step function in capability for wireless networks. Wireless telerobotics add a new level of flexibility and portability not achievable with dedicated wired networks. 5G won’t just improve bandwidth but will greatly reduce latency as well.

Acceptable network latency, jitter and bandwidth are highly application-dependent. In some applications, like telesurgery, haptic (or force) feedback is often conveyed back to the remote operator along with 3D video, providing a highly immersive experience to the remote surgeon. Such applications are among the most demanding since they require high bandwidth (for at least two, often HD, synchronized video streams), low latency and jitter to keep the haptic feedback loop from becoming unstable.

Below is a telesurgery architecture developed for a project with the U.S. Army using custom “smart” codec to provide low-latency, low-jitter, and synchronization for video, haptic feedback, and audio channels. The architecture was part of a prototype telesurgery system — intended for use on the battlefield — that used high-intensity focused ultrasound (HIFU) to rapidly cauterize internal wounds.

Telesurgery architecture designed for the U.S. Army. Source: Energid

Applications like this typically require latency below 10 ms and jitter below 2 ms to ensure stability of the haptic feedback loop and a natural immersive sensation for the remote operator. 5G is expected to improve 4G latency by 30 to 50 times, having a latency goal of 1 ms with minimal jitter.

Another equally important component of the infrastructure is the software layer that translates high-level operator commands to low-level robot and tool commands. The software must be flexible enough to support all the modes of operation for a telerobot and should be capable of presenting data to the operator to give sufficient situational awareness of the environment. This typically means streamed high-resolution video, at a minimum. Often multiple cameras from multiple vantage points are used with a telerobotic system so the operator can see the workcell from different vantage points.

Intuitive operator interface

Early telerobot systems required operators to turn a series of knobs on the input device to move each joint in the remote robot. Such systems were difficult to use and didn’t scale well, especially when the robot had more than a few joints. Today, software can translate simple, intuitive, human gestures into natural and meaningful robot motions.

Energid got its start developing software to more intuitively control advanced robots for NASA and the U.S. Department of Defense. In one project, we worked with the U.S. Army to develop a telerobotic solution for the detection of chemical/biological agents, including toxic gases and the chemical components of improvised explosive devices (IEDs). We needed to iterate many times on the control strategies to ensure that the operator had immediate, intuitive control and feedback.

The same is true for surgical robots, where surgeons need the interface to feel as natural and intuitive as performing the surgery without a robot. Ideally, manipulating a tool — even indirectly through a robot — should feel natural.

The operator interface, therefore, can only be as good as the robot control strategies allow it to be. A robot may have many joints or just a few, but in almost every case the kinematics (i.e., the articulated structure) of a robot differs materially from that of a person. This makes mapping a person’s control intentions to the motion of a robot no easy task.

The upside is that when the control mapping is done well, telerobotic systems can have advantages over traditional methods beyond remote operation. Since a robot can be controller precisely and repeatably, it can augment the abilities of the human operator in many cases.

Given an enabling robot control system, attention needs to be paid to developing the actual operator interface. The operator interface is a critical component of a telerobotic system, as this is what the remote operator will interact with day in and day out. The operator presumably knows how to perform the task being teleroboticized, but not much about robotics. Because of this, successful telerobotic interfaces are almost always designed specifically for each unique application.

Successful operator interfaces come together through significant testing and iteration — ideally with the operators that will be using the system (i.e, surgeons in the case of telesurgery systems).

Simulation is an important tool that developers use to create remote interfaces. A comprehensive simulation environment can reproduce what the remote operator will see, how the robot will move, and even the latency inherent in the remote connection. This cuts down significantly on development time and reduces risk in complex telerobotic applications.

The simulation environment offered by advanced real-time adaptive software has been used to simulate many telerobotic systems, including space-based and undersea applications, where latency and automatic collision detection play a critical role.

Adaptive real-time software enables the enablers

In addition to simulating telerobotic systems, real-time adaptive software can also allow a physical remote robotic workcell to be replaced by an interactive graphical simulation, allowing developers to test the operation of their telerobotic application without the need to procure or fabricate hardware.

Advancements with this enabling technology, and an increased knowledge of the capabilities offered, should encourage the implementation of telerobotics across myriad industries as the pandemic continues — and long into the future.

About the Author

Neil Tardella is the CEO of Energid Technologies Corp. He provides leadership and strategic direction for the company. Tardella has spearheaded commercial initiatives in the energy, medical, and collaborative robotics sectors.

Prior to joining Energid, Tardella held key research and development and management roles at Northrop Grumman Norden Systems, Sikorsky Aircraft Corp., and Technology Service Corp.. He has a B.S. in electrical engineering from the University of Hartford and an M.S. in computer science from the Polytechnic Institute of New York University.

Editor’s Note: The opinions expressed in this article are those of the author alone. They are not necessarily shared by The Robot Report. To submit a guest column to The Robot Report, e-mail Senior Editor Eugene Demaitre.

The post Telerobotics offers automation at arm’s length for healthcare, manufacturing, and more appeared first on The Robot Report.

Reasons why high-DOF robots haven’t caught on

Neil Tardella | Energid — Mon, 30 Sep 2019 20:05:30 +0000

Rethink Robotics’ Baxter robot featured two 7-DOF arms. | Credit: Rethink Robotics

High-DOF robots (robots with more than six degrees of freedom, or axes) hold the promise for a more automated future. However, the real value of high-DOF robots will only be realized when integrators move away from the “one task, one robot” mindset.

As robots become more general-purpose, the need for more axes becomes evident. Switching tasks (e.g., going from a pick-and-place task to a screwdriving task, and then to an inspection task) requires a system designed with inherent flexibility – flexibility that can only be achieved with high-DOF robots.

Adoption challenges of high-DOF robots

The majority of deployed industrial robots in the world have six or fewer axes. There have been attempts to popularize high-DOF industrial robots, with more than six axes, but these attempts have not been very successful. Why is that?

In my opinion, there are two inter-related reasons why high-DOF robots have not caught on: cost and programming difficulties.

Every axis costs money for hardware and engineering, as each additional axis adversely affects repeatability and payload. With the “one robot, one task” mindset, integrators have historically chosen a robotic solution with the minimum number of axes needed to solve the one task at hand efficiently. The belief was that using a 6-DOF robot for a 5-DOF problem added unnecessary cost and complexity to the work cell. True enough in traditional automation.

However, what if one robot could be used for more than one task? What would it take for integrators to reconsider the “one robot, one task” paradigm? To consider this, they would need access to high-DOF robot hardware that provides flexibility in movement, and a means to program and control the robot that is no more complex than the programming required for a low-DOF one.

Today, software to control high-DOF robots exists but has been mostly found in academia and R&D institutions like NASA and DARPA. But commercial solutions that can be applied to traditional automation are becoming increasingly available.

Kinematic redundancy and why it matters

Without specialized software, it’s extremely difficult to program high-DOF robots. The more joints a robot has, the more difficult it is to control. The complexity of control grows non-linearly with the number of joints. For lower-axis robots (with 6-DOF or less), it’s usually possible to find a closed-form solution for how the joints must move to place the end effector. Robots with more than six axes are considered kinematically redundant because they can achieve a particular end effector pose from multiple joint states. The video below shows kinematic redundancy with a 7-axis KUKA iiwa robot.

It is incredibly important for high-DOF robots to be able to deal meaningfully with kinematic redundancy. Humans are experts at this. Think of all the different ways a human can reach down and pick up a ball. Our brains automatically determine how to optimally move our “axes” (a human has over 200 joints) to achieve the necessary hand motion — all while avoiding collisions with ourselves and the environment, preventing joints from locking up, and making micro-corrections based on visual and tactile feedback. All of this redundancy makes us humans highly generalized. We can move seamlessly from one task to the next. It’s really quite amazing. Virtually all evolved forms of life have musculoskeletal systems that allow for the exploitation of redundancy.

Most industrial robots, on the other hand, have not been equipped with software that can deal with kinematic redundancy at all, let alone use it to its advantage. The lack of sufficient software is why the world of industrial automation is filled to the brim with simple, low-DOF robots that have been purchased, programmed, and deployed to perform one task, and one task only. If you would like a traditional industrial robot to perform a new task, it’s possible. But only after a very costly integration effort, and then that robot will be able to only perform that new task. It’s completely lacking in flexibility, one of the key growth drivers for robotic systems deployed in industrial automation settings.

Left to Right – ABB YuMi, Motoman SDA-5F, Rethink Baxter. | Credit: Energid

The future of high-DOF robots

With the advent of software that can exploit kinematic redundancy and simplify programming of high-DOF robots, the cost of additional axes can easily be justified by amortizing it over a broader set of tasks the robot can perform.

The real benefit will be seen when high-DOF robots are used to replace a series of steps in a given manufacturing process — being much closer to a direct replacement for a high-DOF human. When this happens, the change will not be incremental but rather a step function.

About the Author

Neil Tardella is the Chief Executive Officer at Energid. He provides leadership and strategic direction for the company. He has spearheaded commercial initiatives in the energy, medical, and collaborative robotics sectors.

Prior to joining Energid, Tardella held key R&D and management roles at Northrop Grumman Norden Systems, Sikorsky Aircraft Corporation, and Technology Service Corporation. He has a B.S. in Electrical Engineering from the University of Hartford and a M.S. in Computer Science from the Polytechnic Institute of New York University.

Editor’s Note: The opinions expressed in this article are those of the author alone. They are not necessarily shared by The Robot Report. To submit a guest column to The Robot Report, email Editor Steve Crowe

The post Reasons why high-DOF robots haven’t caught on appeared first on The Robot Report.