Worldwide, the construction sector is the number one consumer of global raw materials, only a third of which is reliably recovered. In this context MatterSite focuses on the collation of practical data from demolition sites to facilitate material re-use and reduce waste, using reality capture tools and machine-learning based analysis.

Begun in 2020, this project is ongoing – this post will focus on initial technical developments moving the project beyond the initial proof of concept.

The process takes place in four primary stages – scanning the site, geometric reconstruction, localization and analysis of the recoverable materials therein, and presentation of the data to the end user via a planning interface. Lastly, post-demolition, individual elements are scanned and analyzed to extract additional information relevant to their usage in design and further construction.

Development of these stages takes place across various demonstrators. Currently, the primary development has occurred in the Geometric Reconstruction stage, tested with IaaC’s Digital Urban Orchard Pavilion, and the localization and analysis stage, tested on data captured form IaaC’s Atelier building.

Geometric Reconstruction Development

The IaaC Digital Urban Orchard Pavilion was chosen as a test case for the initial geometry processing as it was due to be disassembled shortly, had already been scanned shortly after construction, and featured over 2000 individual elements. The software stack at this stage employs CloudCompare for heavy point-cloud operations, and Rhino+Grasshopper for specific further analysis.

For display and geometric analysis, the point cloud is transformed back into mesh objects primarily using the RANSAC shape detection system in Cloud Compare, particularly the plane detection feature. This model provides a particular challenge to plane reconstruction systems, so is useful for tuning parameters to obtain the best results. Generally, the results are a balance between amount of planes and accuracy / coherency of the planes. For  various reasons, some areas of the original structure remain underrepresented in the resulting models.

The reconstructed planes are then loaded into Grasshopper, where they must be associated and used to reconstruct the full 3d components. At time of writing, the system could only successfully recreate roughly half of the known elements. Furthermore, many contain confirmed dimensional errors; in the diagram at above right, all elements shown in orange have an inaccurate cross section.

From here further development will focus on continuing to improve the accuracy and completeness of recovered geometry, as well as comparing with more recent scans to track degradation of materials over time.

Localization and Analysis Development

The localization system is focused on identifying the location of relevant materials within 2d imagery. Previous research brought this to a proof-of-concept stage, while current development is focused on improving accuracy, resolution, and generalizability.

The main improvement at this stage is the implementation of a stack classifier system. This classifier uses only the class probabilities output by other classifiers, and combines them into a new input vector. This system is able to overcome and smooth over the weaknesses in individual classifiers in a more holistic way, allowing F-1 scores of about 0.90 on average.

This new clarity is illustrated above. T-SNE charts are a method of illustrating the relative shape of data as a particular model understands it. While exact distances don’t have definite meanings, the relative groupings and local distance illustrate the confusion or separation of different classes to the model (here, red=brick, blue=concrete, green=metal, yellow=wood, black=none). Many of the intermediate models show a degree of confusion between wood/brick and metal/concrete, as well as a strong intersection between the different classes in general. When combined through the final stack classifier however, they show much clearer separation between classes.

MatterSite is a project of IaaC, Institute for Advanced Architecture of Catalonia developed at the MRAC-02 program in 2020/21 by:

Research Student: Matt Gordon and Roberto Vargas Calvo

Tutor: Alexandre Dubor, Aldo Sollazo