INTRASPACE is an augmented reality platform that synchronizes digital layers of interior spaces and enables the user to take personalized and data-driven decisions. On one hand platform allows to visualize data from various sensors in the space itself (temperature, humidity, CO2, etc.). On the other hand, based on this data and personal preferences, platform finds the optimal location for an event.
INTRASPACE
Project aims to rethink the role of technology in our spaces as an essential part of it – not just introducing additional layers of intelligence to our experience, but rather directly influencing the description of the space-time. In the digital era, it is not enough to only define physical boundaries of the space. All the environmental and various socio-psychological layers often play as important role since within the lifespan of the same space it can vary drastically.
INTRASPACE | Behind the Scenes
Even within one particular built space, in winter and in summer micro-conditions would vary a lot and thus affect what we as users find comfortable there.
Platform
Mixed Reality
Platform relies on the Augmented Reality headset (Hololens) to act as interface for the user. The device is in relatively early stages of development but the platform is meant to become a proof-of-concept potentially scalable across platforms.
The terms of mixed, augmented and virtual reality has been highly misused as part of the research a clarification was in order. Classification of such a typology of devices has evolved over last 50 years. First framework by Paul Milgram proposed consideration of physical and virtual as opposites – thus defining a mixed reality spectrum. Next major categorization grew from the concept of mediation by Steve Mann – which aspects are reproduced directly or indirectly. This brought us closer to considering interaction as part of the set up. So the context of the project is extending this model spatially and to inter-connectivity with other parties (devices and users) of the space. For more detailed information in the theoretical aspect of research follow <link to blog post on theory>.
INTRASPACE | Spectrum
Augmented Reality
Augmented Reality in this paradigm has a unique potential for architecture. Since while tapping into dynamic and flexible potential of the virtual you maintain aware relationship with the space. Augmented reality has already entered comercial applications. Airbus stated that use of Hololens 2 in their collaboration with Microsoft it improved design stage up 80% and some complex manufacturing tasks – 30%. In the field of architecture most applications focused on design, fabrication and construction phases. The platform fills the niche and is focused on post production stage – it targets the users of the space. Moreover, most applications still follow egocentric model of a separate experience per user while AR provides opportunity for extended and collaborative space-aware experience. Thus, in contrast, research is a space-centric experience – people in the space share the core, while having unique personalized interaction through their interface device.
Connectivity
The configuration then breaks down into various types of devices: sensor, actuator, interface, interaction device, computation device – all considered within context of space.
INTRASPACE | Actors
With that many different parties platform required introduction of a common protocol of communication. Protocol was implemented over Universal Datagram Protocol and had a header containing information about data type and data ID. The data-flow then was structured in a way that devices would communicate with the “server” and back. This way communication of specific device systems is abstracted away (platform takes care of it not the user) and the system has potentially a synergistic effect. Indeed most of the time is not the data itself but the relation with other data that creates meaningful conclusions and reasonable content.
INTRASPACE | Communication Protocol
The protocol was used for control command, raw data, and also geometry (mesh, curve and point cloud) communication – as a proof-of-concept to be further potentially extended with any type of sequence of commands for the user experience and for any geometry visualization.
Demo 1. Location Request
Having this structure in place, I apply it to a following scenario: in a FabLab there are drastic differences in micro-conditions of environment and variety of functions it hosts. Imagining the space slightly in the future more densely populated with sensors, and with applied actuators – the platform then acts as an augmented reality intermediate for the user.
The space I need for a phone call is different than the place (and time) I need for a presentation – and yet again different for someone else.
Configuration
In this demonstration, I take a simple scenario of having 4 data parameters: temperature, illumination, electrical sockets (I want to be close or far from), and windows (as source of wind). For each of three example functions (desk-crit, presentation and prototyping session) I assign some default target values I ant to achieve and default weights of each parameter. At the stage of request user has opportunity to adjust those according to their personal needs.
For instance, for presentation I want darkest area. For desk-crit I need to be close to a plug. And for prototyping I need to be further away from already occupied locations. In general, I prefer to be closer to window for fresh air, but some people prefer warmer areas and others – colder.
Additionally, I request amount of people for the event – to plan appropriate allocation of space.
INTRASPACE | Workflow
Evaluation
For each of data sources I sample the field. Then collectively I create one metric with the respect to target value and weight of the data type. This gives me possibility to compare locations within the same space according to the actual data and preferences of function and user. These values are not universal so for other person they would be different but this metric is still consistent through space.
INTRASPACE | Metric
Last ingredient baked into the metric is proximity to occupied locations.
One one hand, I want to cluster people to have more efficient space use. Moreover, this way as a manager of the space I can encourage collaborations between people. On the other hand, some people do prefer to work separately according to their personality profile. So I can reward these locations as well. At the same time if two users have same requirements for space I need to have a way of avoiding conflicts.
Even by tweaking one parameter at a time you can see drastic difference in recommended locations.
INTRASPACE | Personal Configuration
Based on this setup I try to suggest three relatively most distant locations with the best values of the metric for the particular user to tweak and to choose from. This way user still has control and participation but is not necessarily envolved with analyzing the data that informed the suggestions.
Automation
While this gives informative analysis tool, this operation is computationally heavy. Especially if you consider scaling the system up with more digital layers and with three-dimensional variations. Thus research explored ways to relate data combined with preferences to the suggested locations. First method directly used data and preferences as inputs to a Dense Neural Network. However, it results in a problem that even adding one sensor would require to retrain the whole network. Second prepared data more explicitly by re-sampling it, downscaling its dimensionality, into an image. And then using a not fully connected Convolutional Neural Network which yielded better accuracy of prediction and potential to scale the system up.
In augmented reality I take the user through a serie of interfaces to collect described configurations. At the same time, I communicate it back to server which sends back the default values and ultimately data visualization geometry and suggestions. So upon configuration I present location suggestions, comparison map and the data behind the allocation (if user chooses to adjust the choice).
INTRASPACE | Interfaces
Demo 2. Data Display
We witness gradual but inevitable spread of Internet of Things. Soon saturation and density of data will reach the scales cities face with Big Data. We see many devices being introduced that can sense, act, mix, optimize.
INTRASPACE | Works Spaces
Thus it is extremely important to be responsible and transparent with the data. Thus it is extremely important to be selective and critical with curation of the data. What kinds of data would be present in space? How could the platform present data to the user in the new framework of Augmented Reality?
Data typology
Data informing the decision making process than can be separated into environmental data – either a continuous field or object proximity.Or psychological and social data centered around user comfort.
INTRASPACE | Data Typology
This can become relevant precisely because of the digital layers of space. If I can allocate and potentially activate to maximize not just one but overall comfort level and quality of life.
INTRASPACE | Comfort
Spatial Data Display
Having a spatial canvas to work with drastically changes the potential to convey information. If previously we had capacity to represent data in 2D. For instance, track one value over time as graph or chart (with data and time as the two dimensions). Or at most – a planar representation animated over time. Now, however, I can directly see how does a value change through the space and overlay values to compare. Three explored methods include symbolic scaled representation, heat map slicing and iso-curve interpolation (more details on technicalities in another blog-post. This list can be massively broadened such that the experience benefits from the potential of the virtual.
INTRASPACE | Spatial Data
Conclusions
Research explored many potentials and limitations of Augmented Reality applied to architecture of interiors in the post-construction stage. The project considers theoretical stand point of the subject: in terms of classifying and characterizing device configurations, data types. Yet the platform is practical, be it digital, system that was already in parts adapted for workshops in MAA (robotic workshop, ar/vr workshop). Platform is not limited to be a data visualization tool. Instead it aims to enable user to have richer and more productive relationship with the space.
INTRASPACE | Results
Project became a proof-of-concept first stage to activating digitally enhanced spaces. If user not only has transparent access to data but has ways of seeing the invisible of the space – sound, air quality, wireless networks etc. And envisioning the near future when even the horizontal layers become adaptable user will require a tool to interact with the digital.
INTRASPACE | Soundscape
INTRASPACE | Volume Activation
Further steps include advancing Artificial Intelligence algorithms used in the decision-making stage on server, optimizing connectivity with more stable connection protocols, expanding and increasing robustness of the interfaces, as well as potentially opening the platform to various more accessible at the moment platforms (like phones), hardware testing with actual sensing constellations and expanding more subtle ways of communicating data with the user.
<video: what’s next>
IAAC // MAA 02 // Thesis Studio – Artificially and Materially Intelligent Architecture
Researchers: Daniil Koshelyuk
Thesis Advisor: Areti Markopoulou
2018 – 2019
