Problem
Robots are difficult to calibrate, it is a process that requires time and specific expertise to be achieved with precision. Plus, understanding kinematics and its applicability on different types of machines is a very complicated task. For this reason we decided to work around this problem teaching the machine to self-calibrate and determine its own kinematics.
Q-Learning
The process of Reinforcement Learning involves these simple steps:
- Observation of the environment.
- Deciding how to act using some strategy.
- Acting accordingly.
- Receiving a reward or penalty.
- Learning from the experiences and refining our strategy.
- Iterate until an optimal strategy is found.
Project Features
Setup
This project works as a vertical plotter, and its composed by:
- Two Nema17 stepper motors.
- 3D printed tool-head.
- External camera.
- Custom UI built in Python.
- 12V power source.
Color Detection
- Observation of the environment.
- Separate the image from the background.
- Detect color via (RGB /HSV /HSL).
- If the color has an Area bigger than a certain parameter sketch a circle around its shape.
- Once it finds the Area, detect the center of the geometry and draw a square.
- Connect the two centers with a line.
System Functioning
Subdivide the camera frame into sub-areas (States) in order to evaluate the distance “Agent-Target” based on the pixels feedback of the robot position.
- Direction towards the target +
- Direction no towards the target –
- Robot hits boundary —
- Pick on target +
- Pick not on target –
- Place not on target –
- Place on target +
- Avoid obstacles
- Node 1 Clockwise
- Node 1 Counter-clockwise
- Node 2 Clockwise
- Node 2 Counter-clockwise
The robot gets a reward or a punishment about the right or wrong direction to the target.
- Agent (Tool)
- Target
The robot gets a reward or a punishment about the right or wrong position towards the target.
Advantages
The advantages of this technology are that the Motors can be placed in whatever way. Thanks to the self-configuring algorithm the tooltip will always locate itself in a space that will take it some iterations to understand where the boundaries are. Allowing a simpler use and more direct way to operate the robot.
Conclusions
The System works only as a proof of concept on our system (vertical-plotter), but we believe it could be applied to any type of robot, which would help to make it simpler and more accessible to a bigger range of workers and not only technicians.
Future Steps
- Achieving a more stable control over the computer vision color detection
- Extending the system from a 2D to a 3D environment
- Control an N number of motors at the same time without caring about the kinematics
Students: Riccardo Mura, Luis Arturo Pacheco Alcalà, Stefano Meloni
Faculty: Mateusz Zwierzycki
Tutor: Starsk Jose Lara