IDEA

The MRAC Studio II.Term was about robotic sensing, to integrate collected data in the workflow and get a step closer to to automation and artificial intelligence. Therefore we got a Bebop Parrot II Drone to find a own project in the giving context to play with sensing and working woth the collected information.

Our idea was to project building instructions with a laser projector, mounted on the drone, on a detected surface, for projecting in the right scale with the freedom giving by the drone for avoiding ostacles and correct distortion. We wanted to create an AR-system without the need for expensive glasses to visualize informations for the construction industry.

HARDWARE

To set up our drone, we developed a way to mount a galvanometer on the drone with additional parts with respect to the shape of the drone and the giving wholes and screws.

Furthermore we had to connect the galvanomter with a PCB-board, Arduino-board, the galvo drivers and the power-supply. Therefore we had to connect the Galvo with a couple of cables to a box where all other parts where integrated. The Drone was just to small to get everything mounted on it.

For developing and testing the system to detect edges or an ARuco-marker with a camera, processing the information and sending the right projection information to the Galvanometer, we decided to build a Pan&Tilt-System to be not addicted to the drone and the control of the drone.

SOFTWARE

To get access to the drone to control and receiving the camera informations we learned to use ROS running on Ubuntu 16.04 . We where also able to detect ARuco-Markers and the normals of these markers to align the drone to the marker in a specified distance. For the Pan&Tilt we connected a Action-Camera above the Galvanomter to align both in a vertical direction . In this setup we used OpenCV to detect the ARuco-Markers and to realign . After realigning, the galvanometer received informations to move its two mirrors to project the specific geometry. This geometry is calculated in Processing with a distortion correction according to the normals of the ARuco-Marker, received from a Python-script.

So the Pan&Tilt don’t have to be perfetcly parallel aligned to the surface it should project on to, because the projection is corrected by the sensed information. In case of drone we have much more flexibility. We can detect surfaces with OpenCv using EdgeDetection and use the Galvanometer to laserscan the distances of the detected edges . With this infomration we can calculate the normals of the surface on and avoid using ARuco-markers which are useless in big-scale-projections.

APPLICATION

For the construction industry there are many possible applications for this idea. First of all you can project AR for a bigger group of people on building site to visualize a floorplan, building informations or simple measurements without the need of expensive AR-Glasses for every single persons.

Students: Luigi Pacheco, Subhash Prajapat, Stefano Meloni, Sebastian Voigt
MRAC Faculty: Aldo Sollazzo, Starsk Lara