Art History for Machines
Objectives >>> A column is an emblematic symbol of strength, order, and reliability. The column capital allows us to track the technological and aesthetical innovations across time and space.
This research aims to investigate the results if we ran images of various ancient columns in a computer vision ML model. How is our aesthetical history perceived by a machine? How can we teach it to understand it better? What kind of images would it give back?
Less is More >>> The initial hypothesis was that if we handpicked less images and devoted more time polishing the dataset, we could generate good visual results with a styleGAN model without a large dataset.
//DATASET
The collection of images we started with were web scraped images of column capitals from several different architectural styles and region.
//Deep Convolutional Generative Adversarial Network (DCGAN)
Two models are trained simultaneously:
- A generator tries to produce real-looking images.
- A discriminator tries to tell real images apart from fakes.
Hypothesis 01
BATCH SIZES
We started training the with a larger batch size number. Using a bigger batch size might hurt the performance because during the initial training the discriminator might get a lot of examples to train on and it might overpower the generator, which would have a negative effect on training.
Hypothesis 02
BALANCING GEN AND DISC LOSS WITH LEARNING RATE
Generator loss decreases during training, but the discriminator loss remains constant. How do we interpret this? It means that the generator is getting stronger faster than the discriminator. To balance things, we tried training the discriminator more than the generator.
Hypothesis 03
INCREASE IN EPOCHS & BATCH SIZES
While training our network, while increasing epochs will yield better resutls most of the time, we found that at some point we did a training run that was too long as too high epoch that it maybe caused overfitting and overtraining. this degraded the quality of the generated samples.
[Dataset Columns]
Initial Dataset
[Types of Columns]
Egyptian, Gothic, Greek, Indian, Islamic
Neoclassical, Persian, Renaissance, Roman
[Generated Columns]
Outcome
[LEARNINGS] It is possible to achieve good visual results with small dataset. Generally, smaller batch size and more epochs gave better results until overfitting with 10k epochs
[NEXT STEPS] Try different GAN models. Explore other hyperparameter tweaks. Resize dataset.
Height Map
As Stanislas suggested in the critique, we attempted to generate a landscape from a heightmap. The results were not the best as it had a lot of noise for low-pixel resolution. This could be improved by generating higher resolution images to begin with before mapping the image with points and remapping the gradient to z vector. Nevertheless, the results still generate a relatively good outline of a column with an abstract stucco-like ornamentation. The objective was to convert machine’s hallucination of columns into tangible, three-dimensional digital model, thus further explore the application of styleGans in architectural design.
‘Art History for Machines’ is a project of IAAC, Institute for Advanced Architecture of Catalonia developed in the Masters of Advanced Computation for Architecture & Design 2020/21 by Students: Marissa Ridzuan & Andrei Okolokoulak and Faculty: Stanislas Chaillou & Oana Taut