A robotic system that can pick up items it has never encountered before. The Keypoint Affordance Manipulation (KPAM) builds on an earlier project that enabled robots to manipulate objects using a collection of coordinates (“keypoints”) on an object.