Human - Robot Teaming

Allocation and Optimization of Robot Plans

In human-collaborative work, humans and robots bring different capabilities to the shared work. For instance, robots are highly capable of performing highly repetitive physical tasks such as pick-and-place, and other behaviors such as screwing or unscrewing can be done more optimally with robotic revolute joints. On the other hand, humans can outperform in cognitive capabilities such as complex component inspection, and offer increased reliability in unstructured environments. Allocation algorithms must take into account these skill sets in order to generate optimal task plans.

In our solution we develop a multi-objective allocation and optimization algorithm that minimizes human ergonomic strain and task completion time, allowing weight selection depending on task. Findings showed differential improvements in outcomes across a variety of manufacturing task contexts, such as quality control, packaging, assembly, and stocking. Specifically, task contexts where agents could split workload in ways that are highly parallelizable and fall within the capabilities of both agents. In sequential tasks or tasks that heavily favor the capabilities of a human such as in metal-cutting or assembly the findings show less benefit in automated allocation relative to baseline.

User Interface for Authoring Robot Collaborative Tasks

We developed an easy-to-use graphical authoring environment for the design of collaborative interactions. Building on the previous work above, users were able to design the tasks using therblig action primitives in a way agnostic to the individual or robot performing the action. Using principles of program verification, synthesis, and optimization, these tasks are compiled into multi-agent programs that minimize total duration and ergonomic strain

Real-Time Collaboration

Real-time collaboration between robots and humans requires more than just a well-reasoned starting plan. Sometimes, unexpected incidents may occur, which requires the attention of the other agent, or other times, seamless handoffs of responsibility require clear communication practices.  

Currently we are developing a user interface to be displayed near a robot performing a collaborative task that indicates the expected time for the robot to complete its portion of the work. This signal provides a means of quickly polling the robot state when the human must handle multiple tasks. Additionally, the user interface will provide the human with an interaction time estimate while working with a particular robot.

The Team

  • Bilge Mutlu

    Sheldon B. and Marianne S. Lubar Professor, Computer Science

  • Andrew Schoen

    PhD Student

  • Curt Henrichs

    Firmware Engineer at IDES


  • National Science Foundation

The Robot

  • UR5
    Learn more »
  • Skills
    Safe | High repeatability | Robust | Payload: 5 kg |



Project Images