STIR.GANS

SATELLITE TILE IMAGE RESTORATION GANS

Deep Learning and Generative Adversarial Networks are an emerging research areas has successfully been used for high-fidelity natural image synthesis, data augmentation tasks, improving image compressions, and many more. STIR.GANS – SATELLITE TILE IMAGE RESTORATION GANS is an exploration on restoring Satellite imagery.

GAN technologies have new application in select architectural design tasks involving the creation and analysis of 2D and 3D designs from specific architectural styles and how GANs can be used analytically to gain insights into different architectural works

Digital Image Restoration is a technique used to recover original image information from degraded images and plays a significant role in improving quality of noisy, blurred and camera-misfocused images obtained from aircrafts, poor-quality videotapes and satellite images

For our project, we will be exploring the use of GANs for image restoration of aerial satellite imagery in regions sub-saharan Africa where satellite imagery are of very low and unusable quality and generate higher resolution images out of it

Why is it relevant?

We hope to improve the quality of open source aerial images and generate large-scaurban satellite image datasets in contexts where open gov datasets are not existent ,and where means of aerial image data collection is out of reach.
These datasets will support a wide range of applications in architectural and urban design research.

Even though random image generation is a hot topic today, GANs are not limited in usage to these generation tasks (Faces, scenery, paintings etc). If we think out of the box, there are many creative applications that GANs are equally effective in! Apart from image quality restoration, another cool example would be image colorization, where we can go through the same process to generate a discoloured dataset and use that to train our GAN.

Urban data have a range of applications from population estimation, urban planning and humanitarian response, to environmental and climate science.
Our project seeks to create a large-scale open dataset using GAN -generated super-resolution satellite imagery to support urban design applications in contexts where datasets are not readily available and data collection is difficult

A. METHODOLOGY

A bit about the architectures, the deep colorisation is a conditional GAN with a generator that takes in the intensity L values of an image and generates the corresponding RGB values, the discriminator is also fed the condition image alongside either the generated or input image and tries to tell if it is real or not. The super resolution Gan is also a form of conditional gan with resnet block generator and upsampling discriminator

For our workflow we will be preprocessing our data and collecting them form satellite images and will be fed into our stacked gans process of deep colorization and super resolution then we will be tiling them back to generate an enhanced version of the satellite imagery

B. PREPROCESSING

First of all, we considered which platforms to download images from. While google maps has slightly better images than bing maps, they have strict copyright restrictions regarding editing or transformation of images, and as such, we went with bing maps.

We sampled images from different locations. Majority from cities that have similar aerial morphology to African cities and clear imagery, and we looked at european cities for clear images used as reference

C. GANS ARCHITECTURE 1 – DEEP COLORISATION

In comparison, images from non-equatorial climates are clearer with distinguishable features and have less colorisation.

Batch normalisation and dropout to the innermost layers

Removed batch normalisation, leaky relu and zero padding

D. GANS ARCHITECTURE 2 – SUPER RESOLUTION

the general architecture of the srGAN,. We take the Original Image and downsample it into a lower resolution image which will then be fed into our generator network which comprises mainly of a series of residual blocks or Resnets and in the end upsampled to generate a Super Resolution image. Meanwhile the Real Image is placed in the discriminator network.

This is the Architecture of the Generator and Discriminator Network of an SR GANS the residual networks are simply a series of convolutions, batch normalizations and P.Relu and passed to an upsampled block. The discriminator network simply has layers that increase the feature maps of the image with alternating strides then flattened into a sigmoid function to test on the Generator

A key thing to note though the image was downsampled using open CV and a mistake for my part was that OpenCV converts the image format into BGR instead of RGB and that’s why we get the slight blue green tint but regardless the output shows the ability of the generated model to improve its resolution

E. TILE MAPPING

CONCLUSIONS

• Continue training the third model till it learns all colour features
• Load pretrained model on aerial images as backbone for generator
• Dataset size is key. Download more datasets to inform training
• Resolve BRG colors from Super Resolution
• For proper alignment of assembled dataset, use mosaicking tool on GIS software

STIR.GAN is a project of IAAC Institute for Advanced Architecture of Catalonia developed at Master in Advanced Computation for Architecture & Design (MaCAD) in 2021-2022 by: Students: Neil John Bersabe and Oluwadamilola Akinniyi; Faculty: Oana Taut, Aleksander Mastalski