Beyond vision and proprioception

We believe touch is a fundamental yet underutilized source of information for intelligent robots. Our goal is to develop tactile-aware systems that use physical interaction to better understand the world and operate effectively in unstructured, dynamic environments.

It is completely unintuitive to imagine operating in the physical world without touch. Even in the simplest pick-and-place task, we cannot ignore our expectations about what we should feel. Without vision, you are still able to discern between successfully grabbing an item and it falling through your hands.

Touch provides a direct measure of physical interaction that is fast, invariant to most conditions, and easier to parse. A robot can infer whether a grasp is secure, whether an object has slipped, or whether a manipulation attempt has succeeded solely from tactile feedback. By integrating the fast and lower-dimensional tactile data with slower and richer vision, we might complete a stronger representation of the physical world. This hope motivates the exploration of data beyond anything currently explored in vision and robotics scaling.

By integrating touch with vision and action, we aim to build systems that are able to generalize across conditions that vision and current modes of perception fail in. Varied lighting, occlusions, and novel same-class items are all areas where we suspect that tactile information provides a significant gain in performance and reliability.