Robotic hand can identify objects with just one grasp

April 14, 2023

Researchers at the Massachusetts Institute of Technology (MIT) have developed a robotic hand with three fingers, each with a rigid 3D-printed endoskeleton encased in a soft outer shell that has multiple high-resolution sensors embedded beneath its transparent silicon skin, enabling it to accurately identify an object with a single grasp.

Unlike other robotic hands that have sensors embedded in the fingertips, requiring multiple grips before an object can be identified, MIT's robotic hand has a pair of GelSight touch sensors embedded in the endoskeleton of each finger at the top and middle of the transparent skin. The sensors provide continuous sensing along the length of the finger, using a camera and LEDs to gather visual information about the object's shape. Each finger captures rich data about many parts of the object simultaneously and sends these images to a machine learning algorithm, which uses them as input to identify the object.

The researchers built two of the fingers of the robotic hand in a Y-pattern, with the third finger as an opposing thumb. Each finger has two cameras, so it can capture a total of six images in a single grip. The hand can obtain rich tactile data from a single grasp by using the tactile sensors that cover all its fingers.

"Having both soft and rigid elements is very important in any hand, but so is being able to do great sensing over a really large area, especially if we want to consider doing very complicated manipulation tasks like what our own hands can do. Our goal with this work was to combine all the things that make our human hands so good into a robotic finger that can do tasks that other robotic fingers can't currently do" - says Sandra Liu.

The challenges faced by the researchers after taking inspiration from the human hand and finger pattern is that the silicon used in covering the robotic hand could peel easily, however, the team has added small curves along the hinge between the joints in the endoskeleton to limit the peeling. They also added creases to the joint so that the silicon would not get squashed as much when the finger bends.