MASTERCRAFTSMAN
Hannah Arendt defines work as the creation of objects that construct the world in which humans can inhabit. It is, in a sense, culture itself. The local artisanal character of makers and craftsmen throughout history is a manifestation of our cultural identity. They create it through their actions, and their accumulation creates the ability to dwell in humanistic existence. Thus, our project deconstructed the craft concept into three main processes: Sense, Think, and Do.
LOSS OF CRAFTSMANSHIP
The invention of automation and assembly lines, while opening ample workplaces – reduced the worker to specific tasks while disconnecting him from the entirety of the artifact creation. In addition, the specialization of labor, Kark Marx claimed, alienated the worker from the products of their work, alienated them from the work process. For the craftsman, this means both the loss of value (as cost) of his work. And for the fabricator, it’s the disintegration of the holistic process – disconnecting sensing from thinking and doing. The holistic view of the vision, body, tool and artifact is scattered. The body is subordinated to the tool and the machine. The golem has risen on his creator.
In recent years, voices are calling to rethink our relationship with our tools and imagine a different relationship between man and machine. In the potential of collaborating with machines in the creative process and fabrication. Unlike the mute industrial machine or the traditional silent, simple tools, robotics can have an agency of their own. By this, the robot becomes a collaborator and interpreter for the master craftsman. We see this as a call to revisit the underlying processes that make the craft and its mastery and then consider how we can communicate seamlessly? What are the possibilities that can emerge?
RESEARCH MOTIVATION
1.How can we use digital senses to transfer knowledge from the craftsman to the Robotic arm?
2.Can we teach craftsmanship to robotics and make them our collaborators in the creative process?
3.Can we connect the robot intuitively to the craftsman, so it’ll become his instinctive extension?
These three questions are the core agenda of this research and lead us to distinguish and find the proper technologies to respond to our concept. To summarize, our research motivation is: we want to learn…the tacit knowledge from the craftsman body. We believe that recording those nonstructural data via a sense to shift to robotic fabrication can challenge more complicated robotics applications and revise the loss of craftsmanship.
SENSE
Start from sense, The main question as we saw it for the sensing is – How can we analyze his way of sensing and operating on the material. In short, what for us meant is how we can sense the craftsman’s movements. The strategy for this research to respond to this issue is to obtain data on the artisan’s body through sensors and compare their kinematics with the robot motion system. The sensors we use are divided into two categories: the first is motion capture, and the second is sampling the force exerted by the craftsman. We can digitize the skills hidden in the motion system through these methods and connect the actions and materials.
Motion Capture
Nowadays, many Motion Capture Systems are developed with a lot of off-the-shelf products about state of art. Still, basically, they can generally be split into two categories: Optical and Non-Optical Systems. In our exploration, we decided to explore both approaches to motion tracking: image-based (Google Media Pipe ) and sensor-based. (IMU & HTC tracker) .
To analyze the visual input, we chose Media Pipe – a Library developed by Google, an open AI framework used to detect body poses from images. We engaged this tool because it allows a wide range of motion and rich theme libraries. In this research, after comparing hand gestures and pose detection library, we eventually chose the pose detection library for its reliability and stability.
Force Detection
Our second approach is related to the force. We used handheld and wearable devices for measuring the value of muscle activities. Force detection basically can be categorized into EMG-based ( Electromyography ) and Resistive-based and have different strategies for detection. In our experiments, we narrowed it down to choosing the resistive-based sensor: “Force sensor” to attach with the craftsman’s tool. The tool’s design was crucial to the success of the experiments because it needed to feel and fit the way the craftsman used to work. On the latest version, we also added an HTC tracker – so in addition to the rotation of the arms and force, we also gained position and rotation both from the tool. To conclude, we had several experiments in movement and force recording. The difference in reliability, sensitivity, and environmental setup demands pushed us to use mainly the force sensor and the HTC tracker.
THINK
The data gathered by sensing is used to make decisions. For making the connection as seamless as possible, the decision-making mechanism needs to be transparent; it needs to tap into the intuitive way the craftsman thinks. Since the kinematics of humans and robots are completely different, we decided to create a way to remap the motion. Besides Inverse kinematics, we decided to connect to the forward kinematic to make the most direct and holistic connection between both types of kinematics.
Machine Learning
Looking at the state-of-the-art, we found this experiment “Adaptive Robotic Training Methods for Subtractive Manufacturing published by Giulio Brugnaro. The movement of a handheld tool learned via Neural Network to the 3D scannings of the imprinted result, and the training movement then sent the movements to the robot through IK mechanics. Unlike our experiment, the relation of this experiment is made between the design and the robotic movement. This means that the craftsman role comes to fruition only in the training stage.
Supervised Training for FK
In our experiment, we focused on the connection between human kinematics and robotic ones. We theorized that there is a pattern to be learned. We created 2 matrices and recorded both the robot and a human agent in the same relative position on the matrix to validate the data collection. The way the pattern to be learned is one of scaling upward. The neural network is made in a fashion that, once activated – a human agent will be recorded, that data will be sent to the neural network, which will calculate the forward kinematics instruction for the robot. And in the validation process, we recorded a set of lines, polygons, and free form curves to see if the neural network can produce the correct movement patterns.
We repeated the same experiment and invented Toni Cumella – a clay craftsman, to participate in this process. To achieve a unified data collecting system, we used ROS (a robotic operating system). This way, we could, in future progress, extend the sensing operation directly to robotic control.
We used the OWL plugin for Grasshopper, developed by IAAC faculty member Mateusz Zwierzycki. The trained neural network showed promising results. The result was tested in a simulation, and although they were fairly good and prove the validity of our methodology, more research, data gathering, and analysis are needed to reach applicable maturity.
DO
Looking at the state of the art of human-robot interactive fabrication, we see that this field about real-time control in robotics is rich and bearing real potational in the design field. For instance, Jose Luis Garcia del Castillo developed “Machina,” which is a simple and intuitive Grasshopper plug-in for real-time robot programming and control and showcases fruitful application with COBOTS; and “Quipt” developed by Madeline Gannon, is a gesture-based control software that facilitates new, more intuitive ways to communicate with industrial robots via using wearable markers and a motion capture system.
Learning from the Workshop
About connecting to robots, our research got a great boost from the workshop “Drafting Deception: Developing Mixed Reality Architectural Environment” designed by Jeffrey Anderson, Ryan Luke Johns, where a setup connected the UR robot to the VR environment in UNITY and ROS. By taking the basic setup from this workshop and altering it to our needs, we rapidly closed some of our technical gaps and proceeded to material experiments. Due to the research limitation and the feedback from the workshop, at this point, we decided to shift our research methodology. At this stage, we recorded the position, rotation, and force from a handheld tool and sent it through IK as robotic instructions.
Experiments
In order to link to our target: recording knowledge of the craftsman’s body. We recorded Toni Cumella’s creative movement and the force he applied to the clay. In this research, we set up two experiments: Real-time imitating & recording and Ornamental & remapping.
As a first step, the first experiment shows the ability to capture and recreate a human motion during the fabrication process. The recording was then remapped to the robot’s workspace and engraved. Doing so shows the potential in creating a vast database of human gestures for later usages.
In the next experiment, besides imitation and recording, we performed real-time interactions, scaling ( uniformly and non-uniformly ) and adding 2 different kinds of augmentations – dashed lines and dotted lines. So the movement is not only recorded but reparameterized to make rich usages of the robotic capability.
FUTURE DEVELOPMENT
As future projections, we can imagine these kinds of systems utilized as a new type of mass customization – mass craftization. In this manner, we could help introduce new manufactured commodities or construction details by adding the craftsman qualities and the humanistic touch.
Mentioning a key aspect – is the understanding that whenever you introduce an agent between the maker and the artifact, you open up new possibilities. In this case, we can imagine creating an open-source community of makers, each contributing his own physical knowledge. In this case, the knowledge can be used to teach, learn or stylize, mix, etc.’ between the different kinematics and personal styles.
To conclude, we see a body of research being built around the topic of Human-Machine Interface and robotic craftsmanship. We believe that our project can open another perspective on revisiting craftsmanship. What is the physical knowledge we can harvest to create the interaction, and what can be the role of the master craftsman in a robotic ecosystem?
</p>
HOMO ROBO FABER is a project of IAAC, Institute for Advanced Architecture of Catalonia developed
at the Master in Robotics and Advanced Construction in 2020/2021 by:
Students: Shahar Abelson, Charngshin Chen
Faculty: Alexandre Dubor, Aldo Sollazzo
Faculty Assistant: Soroush Garivani, Ashkan Foroughi, Antoine Jaunard
Special Thanks: Toni Cumella (CERAMICA CUMELLA)