Fuente: WIPO "tomato" A system for human-robot collaboration is provided herein. The system includes a robotic device having at least one mechanism for moving at least a part of the robot, one or more sensors configured to capture environmental data, a processor in communication with the robotic device and the one or more sensors, and a memory in communication with the processor. The memory has instructions that, when executed, cause the processor to capture an image of a scene, detect objects, generate manipulation options, receive a prompt indicating an overall objective, generate a list of task steps, generate a task graph, determine a probability for each task step, determine a predicted next intention step, and execute a manipulation option corresponding to the next intention step via the robotic device.