MR Assistant // Hardware H.3

Defining Immersive Experience

Sensorama, 1962 by Morton Heilig  /  View-Master’s Disneyland reels, 1963 advertising.

As the title indicates, MR Assistant is a tool that utilises the benefits of mixed reality
technologies, in order to assist architects and constructors during the assembly process
of complex, multi part structures. MR Assistant was developed within the Unity environment for
Hololens. Unity provided enough freedom for us to develop custom features and interface for
the user. The main idea of the tool is to use augmented reality to project a virtual model
into the real world and provide the user with features that modify it and provide information
and multiple different ways to view it.

Digital experience technologies / www.ExtremeTech.com

S.3 Project explanation

The idea of developing this project came from the studio III project (name of the project),
about reusable plastic molds for concrete casting. The structures that are used are complex
arcs and beams that use many repeatable or similar molds. Additionally, each mold consists of
many parts similar to each other. This big amount of molds creates a logistics problem that
slows down the assembly process and increases the possibility of wrong assembly. The question
that rises at this point is how can we solve the logistic difficulties in complex
assemblies?

General form analysis

Fabrication process

Design Model

Sorting pieces

Final users

Architect / Designers                               Construction /multiple users      Robot collaboration

MR Assistant is a tool designed for architects/designers and constructors and requires
basic knowledge of assembly processes. For now it is designed for a single user but there is
a potential for multi-user interface, where multiple constructors will be able to see the
same model and interact with each other’s tool, having common menu, buttons, interactions
and information.

UI / MR Assistant

The User Interface of the tool is designed to be simple, minimal and comfortable for the user.
Understanding the confusion that Augmented Reality can cause to the user we tried to minimise
the amount of information that is projected on the user’s environment. For this reason we
split the features in two menus according to the importance of the features, a static and an
on-screen menu that follows the user when he moves his head.

ON SCREEN MENU: In the on-screen menu we put the main features such as forward, backward and
reset buttons that change the assembly stage. There is also a play button that plays a short
animation explaining the assembly process to the user. We also added an assembly stage slider
that the user can use to manually control the assembly stage and a transparency slider that
allows the user to see hidden parts inside the geometry. Finally there is some information
that changes between general information and specific information for each piece.

Software development_ Unity + C#

STATIC MENU: The static menu is a small geometry that can be placed somewhere is space from
the user and when clicked appears a secondary menu of features. A restart button that restarts
the application, a button to change the projected geometry between molds and concrete parts, a
move geometry button that allows the user to move the projected geometry in space and finally
a toggle that allows turning the on-screen menu on and off.

UI

INTERACTIVE PIECES: In addition all the pieces are interactive which means that by clicking on
them, the user can isolate  them, change between concrete parts and molds, see the molds
explode around the concrete part and see specific information about the piece.

UX_ MR experience_ Viewer vison & Hololens visualization

Futures development

MR Assistant is a tool that has the potential to be used for different assembly processes.
The future goal is to make the tool universal where the user can input a geometry, provide
some information about the materials and the assembly process, assign the pieces in the
correct layers and be able to use the output without having to customise the tool.

Future interface development

Another point of focus will be to make the tool more interactive with the real world by
giving it the ability to read sensor outputs (such as humidity or pressure sensors) and
QR codes on the pieces, that will solve the logistics problem.
Finally, MR Assistant is built for hololens but can be easily modified to work as an
application on smartphones, making it more accessible.

Link to the presentation

MR Assistant // Hardware III is a project of IAAC, Institute for Advanced Architecture of Catalonia
developed at the Master in Robotics and Advanced Construction workshop 3.1 in 2020/2021 by:

Students: Orestis Pavlidis, Alberto Browne, Michael DiCarlo, Hendrik Benz

Faculty: Mery Glez

Faculty Assistant:  Daniil Koshelyuk