Real-Time Deepfake Image Generation Based on Stylegan2-ADA

In recent years, the research community has watched and studied the so-called. Deepfake problem with growing concerns and interests. There is broad consensus regarding the nature and disruptive potential of this technology: they synthesis incredibly realistic multimedia content based on a specific application of Generative Adversarial Networks (GAN) [1]. The interesting issue of Generative Adversarial Networks (GAN) applies to both semi-supervised and unsupervised learning. They accomplish this because high-dimensional data distributions are intrinsically modeled. A pair of neural networks striving to outperform one another is how the framework for GANs, which was first suggested in 2014, characterizes the technology. Consider one network as a plagiarist and the other as an authority on the subject when analyzing visual data. The imitator, known as generator G in the literature, creates copies in an effort to create genuine images [2]. The expert, discriminator D, gathers both original and fake samples and attempts to determine which ones are genuine (as presented in Figure 1). Both are simultaneously trained for and engaged in competition [2]. The discriminator is exposed to both samples taken from the read world dataset and synthetic images. Using the ground truth image dataset, which includes both fake and real images, the discriminator's error is calculated. After each epoch, higher-quality images are produced as a result of the generated mistake being back-propagated throughout the network.

If the generator network is thought of as a functional mapping from a seemingly random, high-dimensional space known as a latent space to the space of actual picture data, then mathematically it is denoted by G: G (z).

1.png

Figure 1. General architecture of Generative Adversarial Networks

Generative Adversarial Networks (GANs) have become a very popular image generating paradigm. As an illustration, StyleGAN is presently the preferred technique for producing images that are nearly image realistic for numerous classes (e.g., human faces, cars, and landscapes). However, producing extremely high-quality results becomes more difficult for classes that show intricate variations. Due to the significant degree of variation in human attitude, shape, and appearance, full-body human generation, for instance, is still an unsolved problem [3]. The limitless number of images accessible online has helped Generative Adversarial Networks (GAN) produce better impressive results. It is still difficult to gather a sizable enough collection of images for a particular application that has requirements for topic matter, image quality, location, time, privacy, copyright status, etc. [4]. The majority of research on the fairness and bias of GANs seeks to either discover and explain the biases or to remove the detrimental impact of using unbalanced data on generation outputs. The three primary areas of bias and fairness research are: discovering and explaining biases; debiasing pre-trained GANs; and improving the training and generation performance of GANs using biased datasets [5].  

This paper aims to generate a large number of images of fake individual that will be used as dataset in researches and train images using two methods, stylegan2 and stylegan2-ADA, and compare between them. Generative Adversarial Networks (GANs) have considerably improved face image synthesis in recent years. Modern image creation techniques now produce images with astounding realism thanks to their high levels of visual quality and accuracy. Most significantly, StyleGAN [6, 7] achieves state-of-the-art visual quality on high-resolution images and presents a revolutionary style-based generator architecture. Furthermore, it has been shown to possess a disentangled latent space that allows for control and editing.

Images, music, and text are just a few of the many sorts of data that GANs may produce. Popular GAN examples from the real world include the following:

Creating faces of people. GANs are capable of creating precise images of human faces. For instance, Nvidia's StyleGAN2-ADA can create great, lifelike photos of fictional persons. Many people mistake these images for real persons because they are so convincing [8].

Creating innovative clothing designs. GANs may be used to produce new clothing designs that mimic older styles. For instance, the clothing chain H&M employed GANs to develop fresh clothing designs for its products [9].

Creating animal images that are realistic. GANs are also capable of producing lifelike animal pictures. For instance, BigGAN, a GAN model created by Google researchers, can generate excellent pictures of animals like dogs and birds [10].

Creating characters for video games. New characters for video games may be made using GANs. For the popular video game Final Fantasy XV, Nvidia used GANs to design new characters [11].

Previous research lacks the use of a high-specification Computer, and this makes it difficult to run the code and train the model. In our research, we will use a high-specification Computer to get the best accuracy, as a computer with a high screen card will be used. We will generate more images and high accurate, and the images will also contain labels, gender and age. When Generative Adversarial Networks (GAN) are trained with insufficient data, discriminator overfitting frequently results, causing training to diverge. Therefore, we will use an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

This section outlines the procedures used in this paper, including the details of generating fake images by using real images with saved these images with labels, age, and gender.

3.1 GAN

Numerous factors, including the distribution of training datasets, network architecture, loss design, optimization method, and hyper parameter settings, define each GAN model. The values of model weights are sensitive to their random initializations and do not converge to the same values throughout each training due to the non-convexity of the objective function and the instability of adversarial equilibrium between the generator and discriminator in GANs [26]. The work starts by training the generator and discriminator from scratch. Where the work of the discrimination is by discriminating the image if it is real from the dataset or fake from the generator. The generator's job is to generate very distorted images, while the discriminator's work is to give random commands that don't know if the image is real or fake. Over time, when you provide it with a lot of data, about, say, a million images, it will start to discriminate between fake and real, and the generator will start making images that look like the real ones. In general, the work of the discriminated is to train the generator to generate an image that is closer to the truth. The following law represents the mechanism of action.

$\frac{\partial}{\partial \theta g} \frac{1}{m}[\ln \ln [1-D(G(z))]]$          (1)

The above Generator is modeled by a neural network G(z, 1). Its function is to translate the required data space, x, to the input noise variables, z. On the other hand, a second neural network D(x, 2) simulates the discriminator and generates the likelihood that the data originated from the genuine dataset, in the range (0,1).

The Discriminator is then trained to accurately identify whether the incoming data is authentic or fraudulent. This implies that its weights are adjusted to increase the likelihood that any actual data input x will be identified as being part of the real dataset and to reduce the likelihood that any false image will be identified as being part of the real dataset. In more precise words, the loss/error function maximizes D(x) and reduces D(G(z)) functions.

3.2 Dataset

Previous studies contain many data that have been worked out extensively. In our research, we downloaded the FFHQ dataset of 76,400 images from kaggle, then trained the images and generated fake images after resizing the real images to 32*32 through the data tool. Also, we will use a dataset-tool for the main image folder to change the format of the real images in it to png to fit our proposed integration in order to obtain consistent and standard images for use in generating fake images shown in Figure 2.

Also, we used our real data by inserting personal images that are being projected to generate fake images that are close to the real images, as presented in Figure 3. Also, we used our real data by inserting personal images that are being projected to generate fake images that are close to the real images. In addition, we trained the model in real-time using the laptop's camera, taking live images of two individual, and generating fake images by moving from the first image to the second, see Figure 4.

2.png

Figure 2. Real images dataset

3.png

Figure 3. Personal images

4.png

Figure 4. Live images from Laptop camera

3.3 Random projection

The cutout augmentation may be seen of as projecting a random subset of the dimensions to zero, for instance, the pixel and patch blocking in AmbientGAN [27]. Let $P_1, P_2, \ldots, P_N$ represent a collection of deterministic projection augmentation operators, where $P_j^2=P_j$ serves as their distinguishing characteristic. Each of these operators, for instance, has the ability to set a separate fixed rectangular region to zero. It is obvious that each projection has a null space (unless it is the identity projection) and that it cannot be inverted independently.

Take into account a stochastic augmentation that chooses either the identity or one of these projections at random. The discrete probabilities of selecting the identity operator $I$ for $p_0$ and $P_K$ for the subsequent $p_k$ are denoted by $p_0$, $p_1$,..., and $p_N$. Define the projections' combination as:

$T=p_0 I+\sum_{j=1}^N x p_j P_j$         (2)

T is a collection of operators once more, but unlike the preceding examples, some (but not all) of them are not invertible. T is invertible on the probability distribution $p$, but under what circumstances? Assume that T is not invertible, meaning that a probability distribution with the shape x≠0 exists that makes Tx=0.

$0=T=p_0 I+\sum_{j=1}^N x p_j P_j$          (3)

$\sum_{j=1}^N x p_j P_j x=-p_0 x$          (4)

We can consider the inner product of both sides of this equation with x if we make some technical assumptions (such as the discreteness of the pixel intensity values, which is justified in Theorem 5.4 of Bora et al. [27]):

$\sum_{j=1}^N x p_j\left\langle x, P_j x\right\rangle=-p_0\langle x, x\rangle$          (5)

If the probability $p_0$ of identity is larger than zero, the right side of this equation is absolutely negative since x≠0. Since the inner product of a vector with its projection is non-negative, the left side is a non-negative sum of non-negative components. Therefore, unless p₀=0, the assumption contradicts itself; on the other hand, if there is a non-zero chance that it would yield the identity, random projection augmentation does not leak.

3.4 The proposed method

The development of Deepfake's neural networks, which excel at producing convincing human-sounding images and videos, has significantly enhanced the technology [28]. We presented a system that could identify forgeries image files in order to lessen the negative impacts brought on by these types of images. Recently, several new methods have emerged in this field. Many Deepfake models exist, including CycleGAN, GANimation, StarGAN, and others. GANimation can only change one whereas StyleGAN can edit many characteristics. Since it is robust enough, we create a model to counteract StyleGAN manipulation. The StyleGAN is supplied with real images [29]. The altered photos are produced by StyleGAN.  In this study, we will examine and contrast the StyleGAN 2 and StyleGAN-ADA. Understanding (and managing) the image synthesis process in the convolutional GAN generator is the goal of SyleGAN, See Figure 5.

5.png

Figure 5. Traditionally, a feed forward network's input layer, or the first layer, is where the generator receives the latent code (a). By completely removing the input layer and beginning from a learnt constant instead, we stray from this architecture (b)

The architecture of the generator would be altered, according to the authors. A feed-forward network projects and detangles the input into a middle latent space rather than giving the generator the latent code (Z) directly (W). To directly adjust the adaptive instance normalization (AdaIN) after each convolution, an affine transformation may be generated from W. As a result, W will be encouraged to specialize in various styles by the affine transform settings. Keep in mind that a learning constant tensor serves as the generator's input. Each feature map is then given an additional dose of Gaussian noise to facilitate the creation of stochastic details.

The AdaIN operation is defined as:

$\operatorname{AdaIN}\left(x_i, y\right)=y_{s, i} \frac{x_i-\mu\left(x_i\right)}{\sigma\left(x_i\right)}+y_{b, i}$          (6)

Since the learnt mapping induces its sampling density, the intermediate latent space W does not need to support sampling in accordance with any fixed distribution (whereas Z must). Since disentangled factor representation makes it simpler to produce realistic images, the generator has an incentive to linearize the factor of variation in Z while they are learning. The authors found no advantage to giving the latent code as input to the generator when adding the mapping network and AdaIN. They streamlined the design by requiring an input of a constant (learned) tensor. These modifications have the significant benefit of enabling both high-level and low-level control over the styles, as each style only has a local impact (specific to a convolution operation).

Where the matching scalar components from style y are used to scale and bias each feature map $x_i$ after it has been individually normalized. Since there are twice as many feature maps on that layer, y has two dimensions. Instead of using an example picture in our method as in style transfer, we calculate the spatially invariant style y from vector w. As comparable network topologies are previously utilized for feedforward style transfer [30], unsupervised image-to-image translation [31], and domain mixes [32], we decided to reuse the term "style" for y. AdaIN is especially well suited for our aims because to its effectiveness and concise representation when compared to other generic feature transformations [33, 34].

3.4.1 StyleGan2

StyleGAN-2 outperforms StyleGAN-1 in two different ways. In order to address the "blob" issue, it first applies the style latent vector to alter the weights of the convolution layer. The created image has a "blob" problem since important information is lost when the resulting image is normalized using the style latent vector. As a result, the generator learns to produce a big blob that acts as a "distraction," absorbing the majority of the normalization's effects (somewhat similar to using flares to distract a heat-seeking missile). Two, it makes advantage of residual connections to assist it to avoid the problem where some features become stuck at pixel intervals [7]. There are two elements to the StyleGAN2 generator. An original, normally distributed latent is first converted into an intermediate latent code, $w \sim W$, via a mapping network. Finally, a synthesis network G uses a learnt 4×4×512 constant $Z_0$ to create an output image $Z_N=\mathrm{G}$ by applying a series of N layers made up of convolutions, nonlinearities, upsampling, and per-pixel noise ($Z_0$; w). The modulation of the convolution kernels in G is controlled by the intermediate latent code w. Two layers are run at each resolution according to a strict×2 upsampling schedule, and the number of feature maps is trimmed in half after each upsampling. Skip connections, mixing regularization, and path length regularization are further techniques used by StyleGAN2.

Stylegan2 was updated by the StyleGAN-2-ADA (where "ADA" stands for "adaptive") [4] which employs invertible data augmentation. The term "adaptive" refers to how it adjusts the amount of data augmentation used by beginning at zero and progressively increasing it until a "overfitting heuristic" reaches a goal level.

3.4.2 StyleGan2-ADA

Designing a technique to train GAN with less data is the goal of StyleGAN2-ADA, where ADA stands for Adaptive Discriminator Augmentation, see Figure 6.

6.png

Figure 6. Stochastic Discriminator Augmentation [35]

The letters G and D stand for the discriminator and generator, respectively. The augmentation probability, p, which determines the intensity of the augmentations, lies between 0 and 1. We often add augmentations to the pipeline, so the discriminator D seldom gets a clean picture. The value of p will be set at about 0.8.

Before the discriminator D during training evaluates the generated images, they will be enhanced. The generator G is instructed to only create clean images because the augmentation process is performed after the generation.