SOFTWARE II / INTRO TO ROBOTICS WITH ROS – BIM // LINKBOT

MOTIVATION

Worksites tend to be very difficult to keep organized and staying on top of progress and issue tracking can be a big challenge for site managers, architects and all other stakeholders.

  • The real world and the virtual world are difficult to keep aligned.
  • On-site staff need to add to remotely access BIM data.
  • Office-based staff need to communicate important data from their desks

As QR codes are becoming more ubiquitous, we believe they could be used as unique identifiers to label worksite issues. When used alone, they merely profide an improved labeling system. However combining QR codes with mobile robotics can offer a unique tool to keep work progress updated and accessible to all stakeholders during a construction project.

Site managers could now stick a QR code onto an issue, and embed information about the problem from their phone onto the QR code.

Using QR codes with autonomous robots enables a bi-directional link between the BIM model and the building site.

Regular updates by an autonomous rover read the QR codes and coordinate data with physical and digital location

SNAGGING / PUNCH-LIST

Site managers currently need to do daily site visits to inspect, log and document problems or inconsistencies they observe on site. This process is inefficient as it still relies on long incomprehensible excel databases and is prone to errors.

PROJECT GOALS

ROBOT [FOLLOW] | SNAGGING

We propose 2 ways of mapping. The first is a mobile robot which follows the site managers, simultaneously mapping and updating the QR code information

ROBOT [MAP+FIND] | SNAGGING

The second method involves the robot initially mapping the space at the beginning of a worksite, and then using that map along with obstacle avoidance using LIDAR to navigate the space every evening, updating  the database on a regular basis.

ROBOT[MAP+FIND]

IMPLEMENTATION

After we map the space, the turtlebot navigates using AMCL probabilistic localization to move around the space. Using the mounted camera, it continuously searches for QR codes using the VISP Auto tracker package. When it detects a QR code, it drops a marker on the 2D map.

FUTURE SCOPE

IMPLEMENTATION

We first run the turltebot with Gmapping and the teleop keyboard to map the space.

Then using the camera and Visp autotracker, we try to look for QR codes in the space. Simultateously we also record the pointcloud data recorded by the depth camera and the PCL package.

ViSP uses openCV to run its detection. It links to the ‘usb’ camera and retrieves the raw images to detect the QR code.

Our interest was in publishing the topic of its position from ViSP onto the 3D map of the site.

The idea is to create a python script that would  use the ‘/status’ to notify when to publish the ‘/object_position’ coordinates on the mapped space.

This is achieved by subscribing to the status of the QR code, which a returns a value between 0 where it isn’t detecting anything and 5 where the confidence value is max:

  1. Pattern detected.
  2. Model successfully initialized (from wrl & xml files).
  3. Tracking model.
  4. Detecting pattern in a small region around where the pattern was last seen.
  5. Detecting pattern in the whole frame.

Given this it is safe to assume that a value of 3 or greater means we can execute the code to generate our marker.

We first evaluate if the QR code is being seen, then we get the QR codes position, which we are also subscribing to.

This code is subscribing to the robot’s position as a way to test the marker creation

We use the QR code position as the origin for publishing our marker in RViz.

This marker is stored in an array so that we can check the placed markers’ positions to ensure markers don’t get created more than once.

Each time the QR code is detected (which is happening every 0.1 seconds until focus is released) the array is checked against the position of the current QR code being detected. If the distance is greater than a given amount (in the case 0.5m), a new marker is generated and added to the marker array. 

QR_code1 + PointCloud

BIM Linkbot is a project of IAAC, Institute for Advanced Architecture of Catalonia developed as part of the Master in Robotics and Advanced Construction in 2021/2022 by:

Students: Amy Kim, Christopher Booth, Alfred Bowles, Micheal Robert Blackburn, Tomas Quijano

IAAC faculty: Carlos Rizzo, Vincent Huyghe