Generative Adversarial Network–Based Landscape Design Image Generation and Visual Style Transfer with Feature Decoupling

2.1 Overall framework of the method

The landscape design scheme image generation and visual style transfer method proposed in this paper is based on a two-stage GAN framework, aiming to achieve collaborative improvement of layout accuracy and style delicacy, and is specifically adapted to the professional requirements of landscape design, as shown in Figure 1. The framework adopts a progressive process. The first stage is the layout-guided generation stage, which relies on landscape semantic information extraction and parsing to generate an initial landscape image with complete structure, reasonable layout, and compliance with landscape design rules. The second stage is the fine style transfer stage. Taking the initial image generated in the first stage as the structural basis, it completes precise transfer and fusion of style features, ensuring that style expression is natural and delicate while not damaging the integrity of the core landscape structure. In order to realize cross-stage structural constraints and feature collaboration, the framework introduces a shared mechanism of a landscape semantic encoder. The encoder is responsible for extracting semantic features of core elements such as vegetation, water bodies, and paving in landscape images, and sharing them in two stages, providing unified structural references and constraint baselines for the two-stage generation process, and laying the foundation for collaborative optimization of subsequent core modules. The framework design focuses on technical pain points specific to landscape scenes, omits the conventional basic structure description of GAN, and highlights the core process and key design adapted to landscape design requirements.

Figure 1. Overall framework of landscape design scheme image generation and visual style transfer method based on generative adversarial network (GAN)

2.2 Stage one: Layout-guided landscape scene generation

The core objective of the first stage is to generate an initial landscape scene image with complete structure, reasonable layout, and clear semantics based on input layout information, providing a reliable structural foundation for subsequent fine style transfer. To address the problems that the traditional SPatially Adaptive DEnormalization (SPADE) module cannot effectively distinguish foreground and background in landscape scenes and lacks multi-scale feature fusion, resulting in low foreground accuracy and feature mixing in generated images, this paper designs a multi-scale spatially adaptive normalization module as the core component of the generator in this stage, achieving collaborative control of coarse-grained global composition and fine-grained local texture. Different from the traditional SPADE that injects semantic information only at a single scale, Multi-Scale SPatially Adaptive DEnormalization (MS-SPADE) independently injects layout information at each resolution level of the generator. Through parallel processing of multi-scale feature channels, it captures global spatial layout and local detail features of landscape scenes respectively, enabling the generation process to follow overall layout constraints while accurately restoring local texture characteristics of different landscape elements, fundamentally solving the problem of insufficient multi-scale feature fusion. Figure 2 shows the structural diagram of MS-SPADE.

Figure 2. Structural diagram of Multi-Scale SPatially Adaptive DEnormalization (MS-SPADE)

The core innovation of MS-SPADE lies in the design of a learnable position weight coefficient. This coefficient can adaptively adjust the dependence of different spatial positions on layout information through network training, realizing differentiated processing of foreground and background regions in landscape scenes. For foreground regions with high generation difficulty and high semantic priority, such as solitary trees and landscape ornaments, the position weight coefficient is adaptively increased to strengthen the guiding effect of layout information on feature generation in these regions, ensuring that the shape and position of foreground elements are highly consistent with the input layout. For background regions such as lawns and sky with relatively simple semantics and low generation difficulty, a parameter sharing strategy is adopted to reduce the position weight coefficient, reducing computational overhead while ensuring generation quality, and improving model generation efficiency and overall accuracy. The module is embedded in key levels of the encoder and decoder of the generator, working collaboratively with convolution layers and activation layers. Through a dynamic feature fusion mechanism, layout information and feature maps are adapted pixel-wise, effectively avoiding mixing between features at different scales and improving structural integrity and detail fidelity of generated images.

To further strengthen layout constraints and ensure that the generated image strictly follows the semantic rules of the input layout, this paper designs a layout-aware loss function. Through the collaborative effect of cross-entropy loss and boundary alignment loss, the layout fidelity of the generated image is constrained in a dual manner. Among them, the cross-entropy loss is used to constrain the semantic layout consistency of the generated image. A pre-trained Pyramid Scene Parsing Network (PSPNet) scene parsing network is used to perform semantic segmentation on the generated image to obtain the semantic prediction result of the generated layout. By calculating the cross-entropy between this prediction result and the input layout, the semantic distribution of the generated image is forced to be consistent with the input layout, effectively avoiding structural errors such as water bodies intruding into paving and disordered plant positions. The calculation is shown in Eq. (1):

$L_{c e}=-\frac{1}{H \times W} \sum_{i=1}^H \sum_{j=1}^W \sum_{c=1}^C y_{i, j, c} \log \left(p_{i, j, c}\right)$          (1)

where, H and W represent the height and width of the image, respectively, C is the number of landscape semantic categories, y_i,j,c is the ground-truth label of category c at pixel (i,j) in the input layout, and p_i,j,c is the predicted probability of category c at this pixel in the generated image.

The boundary alignment loss is used to address the problem of blurred edges of landscape semantic regions in traditional methods [16]. By applying a Laplacian gradient penalty to the semantic segmentation result, it encourages clear and sharp edges at the boundaries between different semantic regions. This loss applies a penalty to blurred edge regions by computing the Laplacian gradient of the semantic segmentation map, forcing the model to generate clear semantic boundaries. The calculation is shown in Eq. (2):

$L_{b a}=\frac{1}{H \times W} \sum_{i=1}^H \sum_{j=1}^W\left\|\nabla^2 S(i, j)\right\|_1$        (2)

where, ∇² denotes the Laplacian operator, S(i,j) is the output value of the semantic segmentation result of the generated image at pixel (i,j), and || ||₁ denotes the L1 norm. During gradient computation, the gradient change of edge regions is captured through second-order difference operation on the semantic segmentation map. Larger penalties are imposed on blurred edges with smaller gradient values, thereby improving the clarity and sharpness of edge regions, ensuring boundary distinction between landscape elements, and further improving layout fidelity and visual quality of the generated image.

2.3 Stage two: Instance-aware fine style transfer

2.3.1 Adaptive feature decoupling and reorganization module

To address the problems of local texture distortion in landscape scenes and rigid fusion between structure and style caused by global feature matching in traditional style transfer [17, 18], the adaptive feature decoupling and reorganization module (AFDR) module takes instance awareness as the core. Through frequency-domain feature decoupling and dynamic reorganization mechanisms, it achieves precise separation and natural fusion of landscape structural features and texture features, ensuring that the core landscape structure is not damaged during style transfer. The core innovation of this module lies in the frequency-domain adaptive decoupling logic. Different from traditional methods that perform feature separation in the spatial domain [19, 20], it converts the content image features generated in the first stage into the frequency domain, and decouples the structure stream and texture stream by separating low-frequency components and high-frequency components. The low-frequency components correspond to the structure stream of landscape scenes, carrying spatial distribution and overall shape information of core elements such as vegetation and water bodies, and maintaining their integrity without style transformation. The high-frequency components correspond to the texture stream, carrying surface details and texture features, and style transfer is performed only on this component, fundamentally avoiding mutual interference between structure and texture. Figure 3 shows the AFDR frequency-domain processing flow.

Figure 3. Frequency-domain processing flow of adaptive feature decoupling and reorganization module (AFDR)

To achieve differentiated decoupling of different landscape elements, the module designs a learnable band-stop filter. After converting the feature map into the frequency domain through two-dimensional discrete Fourier transform (2DDFT), the filter is used to dynamically divide the boundary between low-frequency and high-frequency components. The computation of 2DDFT is shown in Eq. (3):

$F(u, v)=\frac{1}{\sqrt{H W}} \sum_{i=0}^{H-1} \sum_{j=0}^{W-1} f(i, j) e^{-j 2 \pi(u i / H+v j / W)}$        (3)

where, f(i,j) is the feature value of the spatial-domain feature map at pixel (i,j), F(u,v) is the transformed frequency-domain feature, H and W are the height and width of the feature map, respectively, and u and v are frequency-domain coordinates. The transfer function of the learnable band-stop filter is shown in Eq. (4):

$H(u, v)=1-\sigma\left(k \cdot\left(\sqrt{u^2+v^2}-\theta\right)\right)$        (4)

where, σ is the Sigmoid activation function, k is the adjustment coefficient, and θ is a learnable frequency boundary threshold. Its parameters are adaptively adjusted according to landscape element categories during training. For example, for water surface elements, the model adaptively reduces θ to retain more high-frequency components to restore ripple details; for grassland elements, θ is appropriately increased to reduce high-frequency noise interference. In the feature reorganization stage, two independent convolution networks are used to enhance the decoupled structure stream and the stylized texture stream respectively, strengthening structural integrity and texture delicacy. Then, channel-wise concatenation is adopted to fuse the two types of features and send them into the decoder to complete final image reconstruction, ensuring natural connection between structure and texture and achieving instance-aware fine style transfer.

2.3.2 Cross-scale attention mechanism

To address the problems of excessive repetition of landscape textures and single style hierarchy caused by single-scale feature patch matching, this paper designs a cross-scale attention mechanism. Through multi-scale pyramid feature matching, collaborative style transfer of global tone and local details is achieved. The decoupled content texture features are constructed into a multi-level feature pyramid, and local patch matching is performed at different scale levels with the corresponding scale features of the reference style image. The low-scale level focuses on global tone distribution and overall style basis of landscape scenes, ensuring overall consistency of style transfer. The high-scale level captures fine brush strokes and texture variations of landscape elements, restoring local detail features such as plant texture and water body texture. Multi-scale hierarchical matching breaks the limitation of single-scale matching and fundamentally alleviates local texture repetition and style flattening.

The cross-scale attention mechanism adopts a soft attention weighted fusion strategy to integrate multi-scale matching results, and realizes smooth transition of stylized texture features through adaptive weight allocation. First, similarity response values of feature patches at each scale are calculated, and then Softmax normalization is applied to obtain attention weights of each scale. The weight calculation is expressed as:

$\alpha_s=\frac{\exp \left(\phi\left(F_s^c, F_s^s\right)\right)}{\sum_{k=1}^K \exp \left(\phi\left(F_k^c, F_k^s\right)\right)}$       (5)

where, ϕ( ) is the cosine similarity function, F_sc and F_ss are the content texture feature and style reference feature at scale s, respectively, and K is the total number of pyramid scales. Based on the obtained weights, multi-scale stylized features are weighted and summed to generate the final stylized texture feature map. This fusion method effectively avoids rigid texture and style shift caused by single-scale dominance, ensuring natural transition of landscape textures and uniformity of style expression.

2.3.3 Boundary consistency loss

During style transfer, unnatural mutation of texture and color often occurs at semantic boundaries, which damages structural integrity and edge sharpness of landscape scenes. Boundary consistency loss takes this as the optimization objective. By constraining edge feature consistency between the generated image and the content image, clear and regular semantic boundaries of landscape elements are ensured. This loss focuses on preserving edge features in boundary regions of landscape elements, avoiding erosion of original layout boundaries during style transfer, and suppressing edge blurring caused by style color bleeding. It forms cross-stage constraints with the layout-aware loss in the first stage, further strengthening structural fidelity of the overall scene.

Boundary consistency loss achieves precise constraints through edge detection and structural similarity measurement. First, the Canny operator is used to extract edge response maps of the content image, style image, and generated image. Then, the structural similarity index between the generated image edge response and the content image edge response is calculated to construct the loss function. The specific form is:

$L_{b c}=1-\operatorname{SSIM}\left(E_c, E_g\right)$        (6)

where, E_c is the Canny edge map of the content image, E_g is the Canny edge map of the generated image, and SSIM is the structural similarity function. By minimizing the difference between edge features of the generated result and the original content image, this loss forces the model to preserve contour and sharpness of semantic boundaries during style transfer, avoiding unreasonable texture jumps and color mutations at boundary regions, so that the final generated image satisfies both style expressiveness and structural integrity.

2.4 Progressive adversarial training and loss function optimization

To achieve precise discrimination of multi-scale features and address the difficulty of traditional discriminators in balancing global layout and local detail realism, this paper designs a progressive multi-scale discriminator. Through hierarchical discrimination and progressive training strategy, multi-scale fidelity and training stability of generated images are improved. The discriminator consists of three independent subnetworks corresponding to original resolution, 1/2 down-sampling, and 1/4 down-sampling scales. Each subnetwork has a clear division of tasks: the low-resolution subnetwork focuses on global layout rationality of landscape scenes and evaluates the matching degree between overall structure and input layout; the high-resolution subnetwork focuses on material details and texture realism, capturing fine features such as plant texture and water ripples; the medium-resolution subnetwork connects global and local features, ensuring consistency of multi-scale features. The training process adopts a progressive strategy. The high-resolution subnetwork is trained first. After convergence, the medium-resolution and low-resolution subnetworks are added sequentially, effectively avoiding training oscillation caused by gradient conflicts of multi-scale features. Meanwhile, spectral normalization is introduced inside each subnetwork to constrain discriminator weights, stabilize adversarial training dynamics, suppress mode collapse, and further improve realism and naturalness of generated images at different scales. Figure 4 shows the progressive multi-scale discriminator network structure.

Figure 4. Progressive multi-scale discriminator network structure

Soft cycle consistency loss addresses the problem that traditional cycle consistency loss leads to insufficient style transfer intensity and weak style expressiveness due to excessive regularization. By splitting training objectives and relaxing constraint conditions, precise balance between style expressiveness and structural fidelity is achieved. This loss decomposes the traditional cycle reconstruction process into two sub-objectives: structure preservation and style recovery. The core logic is to apply strict constraints only to structural feature maps of reconstructed images and original content images, while not imposing rigid constraints on texture feature maps, thereby preserving flexibility of style transfer. Specifically, the L1 distance between reconstructed structural feature map and original structural feature map is constrained to be less than a preset threshold τ. When the distance is less than τ, the loss is set to 0 to avoid style weakening caused by excessive constraints. When the distance is greater than τ, L1 loss is applied to ensure structural integrity. The mathematical expression is:

$L_{s c c}=\max \left(0,\left\|S_g-S_c\right\|_1-\tau\right)$        (7)

where, S_g is the structural feature map of the reconstructed image, S_c is the structural feature map of the original content image, and τ is the preset structural constraint threshold, determined as 0.05 through validation set grid search. This design ensures structural fidelity in the cycle reconstruction process while avoiding suppression of style expressiveness caused by excessive regularization, achieving collaborative optimization of structure and style.

The total loss function in this paper consists of six components: adversarial loss, style loss, content loss, layout-aware loss, boundary consistency loss, and soft cycle consistency loss. These components collaboratively optimize layout, detail, and style quality of generated images. The adversarial loss is output by the progressive multi-scale discriminator and constrains overall realism of generated images. The style loss ensures texture style consistency between generated images and reference style. The content loss preserves core structural features of the original content image. The layout-aware loss and boundary consistency loss strengthen structural fidelity from global layout and local boundary dimensions, respectively. The soft cycle consistency loss balances style strength and structural integrity. The weight coefficients of each loss component are determined through validation set grid search. After optimization, the weights are set as: adversarial loss λ_adv = 1.0, style loss λ_style = 10.0, content loss λ_content = 5.0, layout-aware loss λ_lc = 8.0, boundary consistency loss λ_bc = 3.0, and soft cycle consistency loss λ_scc = 6.0. The total loss function is expressed as:

$\begin{gathered}L_{\text {total }}=\lambda_{\text {adv }} L_{\text {adv }}+\lambda_{\text {style }} L_{\text {style }}+\lambda_{\text {content }} L_{\text {content }}+\lambda_{l c} L_{l c}+\lambda_{b c} L_{b c}+\lambda_{s c c} L_{s c c}\end{gathered}$        (8)

The grid search process of weight coefficients strictly follows the control variable method, ensuring reasonable effects of each loss component. It avoids dominance of a single loss during training and achieves collaborative optimization of multiple objectives, fully reflecting rigor and scientificity of the method design.

The proposed two-stage GAN framework achieves a technical breakthrough in landscape design image generation and visual style transfer tasks. Its core advantages come from essential differences between each module and existing methods, and it is consistent with research hotspots in the image processing field, with significant academic value and general significance. Different from the traditional SPADE module which only injects semantic information at a single scale, MS-SPADE realizes differentiated decoding of landscape foreground and background regions through multi-scale layout information independent injection and learnable position weight coefficient design, fundamentally solving the problems of insufficient foreground generation accuracy and feature mixing in traditional methods. Its core innovation lies in integrating layout-guided fine-grained control into each resolution layer of the generator, rather than simple semantic information addition.

The AFDR module breaks through the limitation of global feature matching in traditional style transfer, and realizes precise separation of structure flow and texture flow through frequency-domain decoupling strategy, and only performs style transformation on texture flow, ensuring coordination of structural integrity and style expressiveness. This design is different from traditional spatial-domain feature separation methods and effectively solves the core bottleneck of structure and style coupling.

The collaborative optimization of loss functions achieves balance between style transfer strength and structural fidelity through complementary effects of soft cycle consistency loss, layout-aware loss, and boundary consistency loss, and solves training instability and style bleeding problems in traditional models. The above design not only specifically solves the exclusive pain points of landscape scenes, but also conforms to current research hotspots such as feature decoupling and multi-scale attention. Its modular design idea can be migrated to other professional image generation and style transfer tasks such as architectural design and urban planning, and has strong generality and academic reference value.

Although the proposed method achieves significant performance improvement, there are still some limitations, which point out clear improvement directions for future research. In high-resolution landscape image generation scenarios, due to the large computational cost of multi-scale feature fusion and frequency-domain decoupling process, the generation speed of the model needs to be improved, and it is difficult to meet large-scale and real-time design requirements. In extreme artistic style transfer tasks, such as abstract style and surrealist style, the frequency boundary division of the AFDR module is still insufficient, and problems such as insufficient style transfer or structural damage may occur. To address the above problems, future work can introduce Transformer architecture to enhance feature extraction ability, use attention mechanism to accurately capture global correlation and local details of landscape elements, and optimize model lightweight design to reduce computational cost. The learnable filter design of AFDR module can be further improved by introducing adaptive style strength adjustment mechanism to dynamically optimize frequency boundaries and feature reorganization strategies according to different style types, improving adaptability of extreme style transfer. In addition, stability optimization strategies of adversarial training can be introduced to further improve generalization ability in complex landscape scenes.

The modular design of the proposed method enables strong scalability, and multiple directions can be extended based on the existing framework to further expand application scenarios and practical value. Regional-aware hybrid style transfer is an important extension direction. Its technical implementation can rely on existing semantic segmentation results to divide landscape images into multiple semantic regions. Through the feature decoupling mechanism of AFDR module, independent style weights and style references are assigned to different regions, achieving precise fusion of multiple styles and meeting practical needs of multi-style combination in landscape design. For example, in the same landscape scene, plant regions can be assigned Chinese style and paving regions can be assigned modern minimalist style. Interactive layout editing can be combined with the shared mechanism of the landscape semantic encoder, allowing designers to manually adjust layout semantic labels. The model can respond in real time and generate corresponding stylized images while retaining the adjusted layout structure, achieving real-time interaction between design concept and style presentation, and significantly improving design efficiency. These extension directions not only reflect the modular advantages and generalization ability of the proposed method, but also closely match the cross-demand of image processing and landscape design, providing richer technical support for digitalization and intelligent transformation of landscape design, and have broad application prospects.