Creating NFT ART images using StyleGAN3 AI

Motivation

We are already aware of Generative adversarial networks(GANs) and their uses. One of the popular types of GAN which NVIDIA released is StyleGAN. Recently they have released a new version called StyleGAN3. This article will see how to fine-tune stylegan3 in Google Colab and create new images/NFT’s out of it in art.

Image by Author — Modern Art

What’s new in Stylegan3?

Stylegan is famous for creating hyperrealistic quality images. It is increasingly used to generate art or any image with high resolution. Most of the images are faces or an animal.

According to the paper Alias-Free Generative Adversarial Networks, they mentioned below. Check the bolded words.

We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation

From the evaluations, the team used StyleGAN2 and their alias-free StyleGAN3-T and StyleGAN3-R generators on six datasets (FFHQ-U, FFHQ, METFACES-U, METFACES, AFHQV2, and BEACHES). The results show that both StyleGAN3-T and StyleGAN3-R remain competitive with StyleGAN2 in terms of the Fréchet Inception Distance (FID) image quality metric while also demonstrating a very high level of translation equivariance.

Train in Google Colab

These models are enormous and GPU-hungry. If you are going to pre-train the model, you need to be a Google Colab pro or pro+ member. It provides a higher GPU.

It is nearly impossible to train the StyleGAN from scratch with the GPUs provided in Colab. Also, we should not attempt to train it from scratch. Here, the good news is we can use any pre-trained models used in StyleGAN2. So all the time you have put in to train a styleGAN2 has not gone in vain.

StyleGAN3 is compatible with old network pickles created using stylegan2-ada and stylegan2-ada-pytorch. (Note: running old StyleGAN2 models on StyleGAN3 code will produce the same results as running them on stylegan2-ada/stylegan2-ada-pytorch.

To get the real benefit from the StyleGAN3 architecture, you need to retrain.

So we have the below options.

1. Train a model from scratch
2. Train using the NVIDIA provided a pre-trained model in stylegan2
3. Fetch a pre-trained stylegan2 model and fine-tune on stylegan3

Setup

We will use google colab. We need colab and some additional packages and connect it with google drive because there is a session run time limit.

Pre-processing

Code comes with a dataset processing tool. We need to mention the destination of the transformed zip file and provide the resolution as the parameter. The model supports 512x512 or 1024x1024 image sizes.

Image pre-processing

Model Training

Once you have the input dataset, we can start with the model training process. Let’s look at some of the required training parameters

outdir — Where to save the results
cfg — Base configuration of the model. It can be stylegan3-t, stylegan3-r, stylegan2
data — Training data images
gpus — Number of gpu’s to use
batch — Total batch size
gamma — R1 regularization weight

There are a ton of optional parameters. I will request you to go over the documentation for train.py

Train a model from scratch

This option is straightforward, and we need to run it with parameters without a resume. I would not recommend this approach unless you know what you are doing.

Training the model

Train a model from a pre-trained model

Here we will use any model which was trained before and start from the models NVIDIA. Below are some of the models provided.

Models provided by NVIDIA

To use a pre-trained model, use the below approach. Here the resume_from is the parameter we need to pass. Also, kimg is an important parameter that mentions the total training duration of the GAN.

Training a model

Model training -logs. Source: Author

Generating Images

Once you have trained the model for a sufficient duration, we can generate the images and check.

Generate images

Generated image

With sufficient training and the input dataset, we can create the images according to the needs.

Final Thoughts

In this article, we have seen how to run a stylegan3 using google colab environment. Images generated can be used in various places such as NFT’s, generating different objects, and the use cases are unlimited.

Code

All the code used in this article is available in Google colab.

Subscribe

Please subscribe to my newsletter to get updates on my articles and free code for my articles.

I write on different data science-related products and try out new code with applications. Connect with me on LinkedIn and Twitter.