TS.3B – Tools Seminar
DIGITAL TOOLS (AR / VR)_ Natural Realities
Faculty: Jacek Markusiewicz

Synesthesia

According to Lev Manovich‘s fifth principle of new media, every medium can be transformed into another. It is called transcoding and – according to the author – it is the most substantial consequence of the computerization of media. As creative coders, you are not limited to one language in communicating information about your works. What can be represented by an image or digital model, may as well be described in form of text, video or even music.
During this brief seminar, we will be investigating advantages and limitations of Augmented Reality in channeling messages when limited by accessible hardware only to the senses of sight and hearing. Can we transcode taste to image, smell to sound, touch to a digital model?
[It is true that most mobile devices are also equipped with actuators that can address the sense of touch via vibrations, however, it is unable to simulate real-life somatic experiences]

TASK

Students will be working by groups. The task is to design and program an AR project that transmits a message inherent in the sense of your choice. We will combine all four
works into one AR application. The detailed workflow may differ for each project, however traditional steps required to complete the task are the following:

1. Choose a sense or a group of senses that may not be directly addressed via AR.
2. Find a message that can be channeled through this sense (hunger, pain, happiness?)
3. Extract crucial information that can be transcoded to other media.
4. Design an AR interface a potential user can interact with to receive and understand the message. Consider the possibility of multiple users interacting with the same
interface and thus with each other (maybe the message is easier to understand when many users ‘experience’ it, or maybe the contrary).
5. Design the digital content through which the information will be transmitted (a model, an image, sound, multiple…)
6. Design an image tracker that will be augmented with the digital content.
7. Build a scene using Unity 3d to combine all the necessary content with scripts, etc. This scene will be then incorporated into our final AR app.

Project Organization
INTRODUCTION – SUBJECT PRESENTATION – GROUP DIVISION – CONCEPTUAL WORK – DEVELOPMENT – PLANNING PRODUCTION – BUILDING FINAL APP

Jacek Markusiewicz
Jacek Markusiewicz is an architect graduated from Warsaw University of Technology and the Master in Advanced Architecture at IaaC Barcelona. He specializes in fields of parametric architecture, programming and responsive design. He has been working on different projects combining parametric approach and urban scale in international offices in Barcelona, Warsaw and Beirut. He remains in close relation to academia, giving classes at Warsaw University of Technology and IaaC.