Padowan UR

 

Exploring a workflow that leverages human-robot interaction through existing VR technology, this project focused on the idea of bringing humans into the loop of human-machine work, especially in robotic fabrication or performance arts.

The theme of the seminar explored connecting existing discrete software workflows through web sockets.

Even though the robotic motion is precise and accurate, it is challenging to program robotic tool paths to react to real-world scenarios and specific fabrication or artistic decisions. We explore the paradigm of leveraging Human intuition to generate these decisions.

The aim of this project was to develop a human interaction toolkit.

We were explicitly inspired by the work of Daniella Mitterberger, on the topic of Human-robot collaboration. And we chose the controller as a device to extract positions and poses in 3-D.

Software Interface & Setup

The setup consisted of an HTC Vive headset that communicates over a WebSocket to grasshopper and Compass, the toolpath is then generated for direct Robot control on the UR.

Pseudocode – Unity

We explored several input actions in the VR controller to make specific interactions,

The poses of the Htc-Vive were used to generate 3-D points in space, from the lighthouse 1.0 system to an accuracy of ± 2 mm.

Workflow:

  1. Press trigger for saving the poses to project 3D points into the XZ plane to make linear tool paths and generate a line visualization to see actions in 3D Space.
  2. Press the gripper to save the poses as points and Euler rotations in a JSON file and send it over a WebSocket.

  1. Receive the JSON file in Grasshopper through a WebSocket listener and parse them to extract points.

4. Remap points into the required canvas and generate the required toolpath from the command.

5. We have our light painting, robotic performance generated by human motion.

In essence, this pipeline can be used in any 2D canvas, printing extruders, spraying paint or plaster on a wall, or for robotic assembly of structures.

PADOWAN UR  is a project by MRAC 01 students Mit Patel, Libish Murugesan, Jordi Vilanova and Robert Michael Blackburn, during the Hardware III seminar of the third term. Studio was taught by IAAC faculty Starsk Lara & Daniil Koshelyuk.

IaaC, 27.05.2022

Faculty/    Starsk Lara

Assistant/     Daniil Koshelyuk

Students/           Mit Patel

                             Robert Michael Blackburn

                             Jordi Vilanova

                             Libish Murugesan