UNREALITY WITH AI

Transcending Dimensionality

As advanced AI technologies are gradually being implemented into our daily routines and increasingly more into our daily interactions and tasks, will there be a time when we will be unable to differentiate what is human or machine derived? Especially in combination with other immersive technologies such as augmented and virtual realities, dimensions and realities become intertwined. Are we moving to a point where we lose our sense of reality and control of it?

In a conversation between two AI avatars having a theoretical discussion about dimensionality and social injustice with Abbot’s “Flatland” novel as a topic, how far is the computer from mimicking a real intellectual conversation between two people? By using the Open AI conversational engine GPT3 and Synthesis ML video algorithm we enabled AIs to talk to each other. When you look at the video (

) you get the feeling that the AIs are actually persons, and they have feelings and opinions. How far are we from being unable to distinguish digital-created reality from our day-to-day narrative?

“It is reasonable to suppose that we can use motion capture as technique to let a real human affect a figurative 3D object in augmented reality. Motion capture typically involves tracking parts of the body by infrared cameras to obtain streamed position data. We can realize real human movement by applying this data to a 3D model.”

Hiromitsu Sato & Michael Cohen

In 2016, the first virtual YouTuber was born in Japan. The official biography says Artificial Intelligence (AI) awakened its ego as “Kizuna AI”. We believed an AI named Kizuna AI is taking, moving, and playing video games!! As you can imagine, she was not an AI tho, though voice actress and a technology called motion capture were behind it to pretend an AI. Now, this inventor’s dream seems to get closer to coming true.

A WORLD AUGMENTED WITH AI

Moving from the flat 2D virtual space of the monitor, the unreal can be transferred into our 3D physical world through augmented reality technology and into further postulated spatial dimensions in the 4D with added data and information.   Enhanced with AI technology not only for motion-tracked or personality simulating features but there is also a potential to create more-than real reality whereby what is human-controlled and what is algorithmically generated becomes intertwined.

Aside from the creation of high dimensionality AI avatars, there are many practical applications for combining these two technologies which allow real-time visualization before our eyes to interact intelligently with the world around us.  The ability the create augmented or meta environments with the speed and intelligence of modern algorithms will compete with the regular reality.  Although how convincing these new realities will become are dependent on the technological development and ability to convince the other senses.

Taking the simple form and function of a chair, DC GANS can take existing ordinary objects and produce new variants based on the samples given.  The ability to train the machine or create and design live within the third dimension allows machines to build a reality within our world.

In the novel ‘Invisible Cities’ by Italo Calvino he describes the boundless formation of cities whereby  “The catalogue of forms is endless: until every shape has found its city, new cities will continue to be born.”1  Can our new reality in the future be changed and manipulated by augmented reality as we move through it?   Through increasing computer processing power and software like Tensorflow-flow, these technologies become lighter and portable to allow live processing in the creation of new worlds and objects.

This process can further expand to encompass the natural environment as we know it.  Machine Learning can be used to let the computer design realistic forests and environments by learning from real-world data such as through forest generation using conditional GANs. 2

The manipulation of natural environments in augmented reality through AI learning however is more difficult than ordinary objects due to the complexity of the theme and intrinsic link to the other senses. In the satirical novel Flatland: A Romance of Many Dimensions, Abbot writes about the two-dimensional world of Flatland and the difficulty for a 2D object to adjust and comprehend the 3D object: “ it appears to you a Plane because you are not accustomed to light and shade and perspective….but in reality, it is a Solid, as you shall learn by the sense of Feeling.” 3  Although a plant may be understood algorithmically, human sense perception needs to adapt and accept it through other feelings which may be technologically developed in the future.  The understanding is essential in the future ability to make machine-generated reality indistinguishable from a real one.

The dimensionality applied to Augmented Reality through Artificial Intelligence comes within our proposal of immersive distortion of the real environment. This distortion can take several inputs such as human motion, and object detection to create a metaverse environment or a real scenario with distorted elements. This proposal emphasizes the use of AI as visual elements to be able to interact with the imaginativeness of the user plus the created and the real environment. This comes in hand with the idea of creating another dimension within the physical existence of 3D virtual and real objects, something that could be connected with human cognition stating the art to create new realities from the perspective of the environment we all inhabit. We would like to open the discussion on how these tools can generate infinite scenarios once connected with Augmented Reality as a tool for people to interact with their ideas and transform them in real-time, also amplifying the possibilities of creating different distorted realities within a predefined dimensionality.

1. Italo Calvino and Weaver, W. (2006). Invisible cities. Orlando, Fl Etc.: Harcourt, pp.139
2. Kapp, K., Gain, J., Guérin, E., Galin, E. and Peytavie, A. (2020). Data-driven authoring of large-scale ecosystems. ACM Transactions on Graphics, 39(6), pp.1–14.
3. Edwin Abbott (2019). Flatland.S.L.: Blurb, pp.84

Unreality with AI is a project of IAAC, Institute for Advanced Architecture of Catalonia developed in the Master of Advanced Computation in Architecture and Design 2021/22 by Students: Michal Gryko, Bruno Martorelli, Pablo Jaramillo Pazmino, Sammar Allam and Takeaki Sakakibara Faculty: Mark Balzar and Zeynep Aksoz