Augmented Creativityis a project of IaaC, Institute for Advanced Architecture of Catalonia developed at Master in Advanced Architecture in 2017 of the Academic Program by:
Student: Lalin Keyvan
Faculty: Luis Fraguada.
It is known fact that computers are utilitarianist machines, they understand the world through 1 or 0, yes or no etc. This is the reason why they can work in well structured areas of problem solving or they can be used as tools to aid designers through creative phase. However, for a computer, forming creative work by itself is an enormous challenge because it requires a machine which can learn. Today, computer scientists mainly focus on the usage of artificial intelligence in business, medicine, fabrication rather than architecture, design, art which requires creative notion. Therefore, communication and dialog between architect and computer remains the same, it does not change into a partnership and a collaboration.
In a world that machines are getting smarter and they can learn, how can computers be more involved in the design process? What is the potential for a machine to compose a creative work? Can machines help us conceive design possibilities and solutions beyond human creative capacity and limitations? How will the interaction between designer-user-space change? What will be the role of the designer, architect or artist in the future?
This is where Augmented Creativity presents a reflection on the spatial aspects of sound alongside an examination of responsive and collaborative design. It proposes a framework and an interactive installation for participative design by implementing machine learning techniques to build a memory, a faux consciousness with the use of sound as a spatial event, and an auditive experience.
In this approach, design aspects such as the nature of creativity, the muse in the machines, the theory of memory, sound as a material phenomenon and methods and methodologies in machine learning techniques such as recursive neural networks are discussed to design a space which is not just active but also collaborative. The workflow brings recent developments in computation techniques into the design domain while challenging the traditional methods in creative process.
On this extent, it is highly important to define what is creativity and how is it possible to talk about the muse in machines.
“Is creativity the result of a special mechanism or is it just a part of normal problem solving?”
Creativity is the mental capacity to generate novel and useful ideas. It is not about specifically art or design. Creativity is, at its core, about ideas and how we develop, understand, and communicate them. As a process involves critical thinking as well as imaginative insights and fresh ideas. It is not just about having an “a-ha” moment; it is about setting ourselves up to that moment, then knowing what to do when it happens. Learning creativity, therefore, does not mean starting from scratch, it means enhancing the creative intelligence that already exist within us.
“How can a machine learn the creative behaviour?”
Creativity requires the hidden combination of unconscious ideas. There are four phases of creativity within which conscious and unconscious mental work figure to varying extents; preparation, incubation, illumination and verification. Similarly, the machine learning techniques which is a subset of artificial intelligence where the machine is trained to learn from its collected data base, have five steps: collecting data, preparing the data, training a model, evaluating the model, improving performances. Although, the data preparation stage is highly challenging in terms of their effects on the consequences, selecting and composing the right learning method and algorithm is another delicate stage for implementing creative behaviour.
Before going through the hypothesis, lets talk briefly about neural networks.
“A neural network (also called an ANN or an artificial neural network) is a sort of computer software inspired by the human brain. Neural networks are studied in a field of machine learning named deep learning. The brain solves problems with large clusters of biological neurons connected by axons. Similarly, a neural network is made up of interconnected cells that work together to produce a result. This is one of the possible ways artificial intelligence may work. Most neural networks can still operate if one or more of the processing cells fail. Neural networks learn to do things based on examples. Traditional software can only do what it is specifically written to do.There are three ways a neural network can learn: supervised learning, unsupervised learning and reinforcement learning. These methods all work by either minimizing or maximizing a cost function, but each one is better at certain tasks.”- For More Information
However “A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs. This makes them applicable to tasks such as unsegmented connected handwriting recognition or speech recognition.” – For More Information
Consequently, the hypothesis of this thesis is that computers can learn creative behaviour using RNN(Recurrent Neural Networks) machine learning techniques while building a LSTM(long short term memory). And as an objective, this can be demonstrated with an interactive installation where the sound becomes a spatial event, a material phenomenon, a auditive phenomenon to explore the invisible public space. As a result, the designer, the user and the space can collaborate and compose a participatory design. Despite the fact that collaborative design often involves the creation and observation of novel design processes, intended to move beyond conventional technical roles, participatory design techniques aim to involve the end-users in the development of that technology and design.
The interactivity of this installation works almost the same way of a jazz improvisation “Call and Response”. In music, a call and response is a succession of two distinct phrases usually played by different musicians, where the second phrase is heard as a direct commentary on or response to the first. Here in the beginning, the user activates the space by moving around with a direct sound feedback. After ten seconds, the generated sound caused by the movement of the user will be saved into the data base folder of the recurrent neural network in order the computer can respond with a next generation of the user’s call. After, the user hears the generated chords, he/she may respond to the computer with improvising his/her movements in the space to participate to the collaborative design. In this point, it is also important to state the fact that the designer’s first calibrations such as the visible space design, the sound and space relationships and the decision of the body part as a controller, are extremely important. According to the feedback from the overall sound and the user’s preferences, the designer can make adjustments or additions to the overall interaction.
The workflow brings many different software and hardware together.
1- Max Msp is used to extract location values from the captured skeleton of the user with the help of a Kinect sensor.
2- Wekinator uses a machine learning algorithm to transform x,y,z locations into a sound.
3- Max Msp saves the sound file into the data base.
4- Recurrent Neural Networks generates next sequences.
In conclusion, Augmented Creativity is the story of replicating creative behaviour in computers that can change the interaction between designer and computer, confronts with autonomous orientation in contemporary computation and gains its finality through an interactive installation which demonstrates the collaborative design through sound, movement and space using machine learning techniques.
