Results in Engineering 20 (2023) 101485
Available online 10 October 2023
2590-1230/© 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by- nc-nd/4.0/).
Heritage applications of landscape design in environmental art based on image style migration
Xuyang Guo
a, Jun Ma
b,*aSchool of Architectural Art and Design, LuXun Academy of Fine Arts, Shenyang, 110004, China
bSchool of Visual Communication Design, LuXun Academy of Fine Arts, Dalian, 116000, China
A R T I C L E I N F O Keywords:
Image style migration Landscape design Environmental art
Generative adversarial networks Transfer learning
A B S T R A C T
To create environmental art on the basis of preserving environmental and ecological resources, landscape design considers the peaceful coexistence of art and ecology. In picture generation and rendering for landscape design, image style migration techniques are frequently employed, however their quality is still lacking at this point. In order to enhance the migration effect, the study builds an image style migration model for landscape design, tries to employ a multi-scale discriminator, and introduces a content feature mapping module and a migration learning approach. According to the test results, adding the content feature mapping module enhanced the feature mapping effect of the photographs. The revised model’s peak signal-to-noise ratio and similarity assessment indexes were 17.42 and 0.91, respectively, and the model’s generated images were less distorted.
While the Frey interval distance was only 16.32 and the generated images were strongly correlated, the model had good generalisation. The addition of migration learning somewhat improves all of the measures. In conclusion, the research findings are beneficial for the expression of environmental art in landscape design, and the model’s picture creation assessment index and stability are both good.
1. Introduction
Incorporating the ecological harmony of the natural environment into the expression of environmental art, landscape design (LD) is a type of man-made ecological design that is significant for preserving the ecological integrity of the environment, fostering the ecosystem’s healthy cycle, and preventing the waste of unneeded resources. There- fore, to the greatest extent possible and in line with the environmental circumstances, LD calls for a combination of natural and man-made landscapes. It also calls for the incorporation of rich cultural meanings and aesthetics, accentuating the expression and replication of art and beauty. In order to maintain a balance between the full expression of the designer’s subjective opinion and ensuring certain boundaries of painting style, excellent LD works frequently need to draw from various painting art styles. Because this requires both art and technology, the traditional hand-drawn approach demonstrates glaring technical limi- tations in the expression and effectiveness of the design [1,2]. Much deep learning research has started as a result of the advancement of artificial intelligence technology [3]. Image style migration (ISM) is one of the most important compositional and painting approaches. While allowing for the alteration of the image style, the photos created using
this technique maintain the image’s primary form. The visual quality of the migration results, the scalability of the style migration model, and the migration efficiency of the model still need to be developed and extended [4,5]. Nevertheless, ISM still confronts some technological obstacles. Based on this, the study built an ISM model that was LD-oriented, incorporating a content feature mapping module and the concept of transfer learning (TL). The study employs the attention mechanism to improve the feature mapping module and uses transfer learning to solve the decoupling of content and style. The landscape design, based on image style migration, reflects innovation and personalization fully. The study proposes a unique concept for landscape design that integrates natural style, further promoting the landscape effect of environmental art. The overall study is divided into four main sections: a research review of domestic ISM technology; a proposal for a style migration algorithm based on the TL content feature mapping module; Test studies to gauge the algorithm’s performance; a descrip- tion of the outcomes of the research experiment.
2. Related works
Artistic style migration and image rendering have a plethora of
* Corresponding author.
E-mail address: [email protected] (J. Ma).
Contents lists available at ScienceDirect
Results in Engineering
journal homepage: www.sciencedirect.com/journal/results-in-engineering
https://doi.org/10.1016/j.rineng.2023.101485
Received 1 August 2023; Received in revised form 14 September 2023; Accepted 2 October 2023
application markets, having been significant research areas in computer vision and image processing for many years. Concerning image style migration algorithms, researchers have conducted a range of targeted studies. To address the issue of semantic mismatch in image style manipulation, Xie Chuan et al. propose a semantic segmentation-based algorithm for image style migration, which extracts the semantic in- formation of the content and style images via a mask R–CNN network.
This information then guides the style transfer process. The dataset re- sults from Celeba and Wikiart highlighted that this method was more effective than existing style migration methods in avoiding image style transfer and rendering issues. Furthermore, it can successfully overcome the problem of semantic mismatch during image retrieval, when compared to previous style migration methods [6]. Kim Minseong et al.
have developed an end-to-end learning approach for improving the image style migration technique using encoder and decoder networks.
Their method reduces the computational complexity of the current models for associative-perceptual feature alignment, and minimizes channel redundancy of the encoded characteristics during network training while being ideal for multi-scale image style transfer [7]. To address the issue of preserving texture in image style transfer models, Ding H et al. devised an image style transfer model that employs wavelet transform and deep neural network techniques for synthesising style and detail restoration. The model can match semantic relations with the assistance of attention mechanism and semantic segmentation while also handling images with minute details. The experimental results validate the superiority of this model in preserving image texture [8]. Li et al. constructed an image segmentation model using the improved semantic segmentation network, DeepLab2. The model was capable of local image style transfer, and the experimental results confirmed its practicality with transfer efficiency [9]. Chen et al. have developed a model for image style transfer that utilises multiple convolutional filters.
Concurrently, there is an autoencoder and style bank learning compo- nent to the model. The filter bank was used for multi-parameter image smoothing and denoising, and allowd the model to produce comparable results to those achieved by single parameter settings [10]. To enhance the efficiency of generating style transfer, Huang L et al. developed an algorithm which relies on semantic segmentation and residual networks, alongside Visual Geometry Group Network for feature extraction. Ac- cording to the experimental outcomes, the model has improved the local image style transfer and the generation efficiency. Further, this tech- nique can be fostered in entertainment, film and television, medical, and industrial fields [11]. To address the issue of challenging semantic image style transfer, Liao Y S et al. have proposed a novel method that is context-aware and semantic. The model includes a global context
network and a local context network and focuses on both stylised image derivation and semantic context style transfer. According to experi- mental results, this method produced stylization outcomes that better align with human semantic perception than existing models [12]. Zhou and colleagues enhanced the conditional generative adversarial network by training the recurrent generative adversarial network on style transfer between original optical coherence tomography scanner im- ages. Following that, they trained the mini-contingent generative adversarial network using real datasets for image noise speckle reduc- tion. Finally, they utilized the recurrent generative adversarial network for scanner image style conversion. Experimental results indicated that the enhanced model was superior to the pre-existing model in elimi- nating image speckle noise, maintaining image structure, and enhancing image contrast [13].
More research has also been conducted into landscape design in conjunction with other urban architectural attractions. Fewella L. N.
proposed several light and shadow methods and designs for night lighting at historic sites to enhance the visitor experience [14]. Zakaria, S.A et al. argued that in design, designers need to balance sustainability by fully expanding the range of visual, historical and functional re- quirements of buildings [15]. The design of urban landscapes between augmented reality, virtual reality, and virtual worlds is discussed by Sweeney S. K. et al. [16]. Holographic technology is used to present images in three dimensions while using a transparent medium to faith- fully replicate the original objects and blend them into the surrounding environment.
In summary, numerous studies have focused on modelling image style migration and designing urban landscapes, resulting in significant research outcomes. However, the future research on the visual quality of image migration outcomes, qualitative and quantitative assessments, scalability and generalisation of the model, and the integration of style migration in landscape design remains novel areas of study in image style migration.
3. Design of LD-oriented unsupervised ISM models
The local and global effects of the conventional style migration procedure are obscure. The paper attempts to construct an LD-oriented ISM model by introducing a feature mapping module to enhance the migration impact and employing a multi-scale discriminator to account for the global and even local image style differentiation.
Fig. 1. Schematic diagram of self attention mechanism model.
3.1. Design of an unsupervised image migration algorithm based on improved attention mechanism
The attention model in the neural network component is based on the human attention process, and as a result, the neural network gives varied amounts of attention to various input data subsets, as indicated by the weights. The attention model was initially heavily utilized in machine translation applications, but it has since evolved into a crucial neural network module and a crucial component of artificial intelli- gence, with numerous applications in speech recognition, statistical learning, natural language processing, and other computer-related fields [17].
The model is more likely to concentrate on the information that is helpful for model learning and discard extraneous information when attentional mechanisms are used. An essential component of the atten- tion system, the self-attention mechanism, is used in the research to build models. The self-attention mechanism further reduces the model’s reliance on external knowledge and enhances its ability to identify relevance within the data, making it more useful for visual tasks such as image recognition, classification and style modification, among others.
Fig. 1 [18,19] depicts a schematic representation of the self-attention mechanism model.
Firstly, the features extracted by the model are input to the convo- lution layer to calculate the attentional feature map, and the calculation procedure is revealed in equation (1). In equation (1), x denotes the feature, x∈RC×N, C denotes the number of channels of the feature, N denotes the number of features; βj,i denotes the degree of influence; j denotes the region and C denotes the location.
⎧
⎪⎪
⎪⎪
⎪⎨
⎪⎪
⎪⎪
⎪⎩
βj,i= exp( si,j
)
∑N
i=1exp( si,j
)=f(xi)Tg( xj
)
Wgx=g(x) f(x) =Wfx
(1)
The calculation of the self-attentive layer is shown in equation (2). In equation (2), oj denotes a certain self-attentive layer and Wh, Wv, Wg and Wf all denote the weight matrix.
oj=Whxi,v(xi) =v (∑N
i=1
βj,ih(xi) )
,h(xi) =Wvxi (2)
Ultimately, the self-attentive layer output is shown in time (3). In equation (3), γ denotes the learning scalar and xi denotes the input feature map.
yi=γoi+xi (3)
By merging the style of the target image with the original image’s content in the same domain space, the research enhances the model. And add a content feature mapping module to realize the process of mapping content features between different domains to enhance the style migration effect. The structure of the content feature mapping module is shown in Fig. 2. The original image PA and the reference image PB go through the content encoder and style encoder to extract content fea- tures hA and hB, and style features sA and sB respectively. PA and PB have different encoders and are trained separately for feature extraction. The role of the content feature mapping module is to map hA into the domain of hB and transfer it to the generator GB in combination with the style feature sB, completing the process of migration of the original image through the style. The overall model consists of the content feature mapping module, the generation module and the discriminator module [20].
Firstly, the content features hA and hB extracted by the encoder are normalised and the process is shown in equation (4). In equation (4), Wf, Wg and Wh denote the weight matrix.
⎧
⎪⎪
⎪⎪
⎪⎪
⎪⎨
⎪⎪
⎪⎪
⎪⎪
⎪⎩
hiA→B= 1 C(F)
∑
∀j
exp (
f( hiA)T
g( hiB)
h(hB) )
C(F)
=∑
∀j
exp (
f( hiA)T
g( hiB))
f(hA) =WfhA,g(hB) =WghB,h(hB) =WhhB
(4) The mapping relationship between hA and hB is calculated by the content feature mapping module after the normalisation process, and Fig. 2. Structure diagram of content feature mapping module.
the process of mapping the two is shown in equation (5). In equation (5), λ and γ denote learning scalars.
{cA→B=hA+λhA→B
cB→A=hB+γhB→A (5)
The generation module comprises a content and style encoder and an image decoder. Notably, a residual module is enhanced with an adaptive instance normalisation layer. The multilayer perceptron generates the parameters for the adaptive instance normalisation layer based on the style features in a dynamic manner, ensuring diverse image styles. The calculation of the adaptive instance normalisation layer is depicted in equation (6). In equation (6), letters AdaIN and z represent for the input characteristics, letters μ and σ for the mean and standard deviation, respectively, and letters γ and γ for the dynamic parameters that were generated.
AdaIN(z,γ,β) =γ (z− μ(z)
σ(z) )
+β (6)
The discriminator module of the research constructed model consists of a multi-scale discriminator composed of three discriminators, each of which contains a convolutional layer and a fully connected layer to discriminate the whole, 12 and 14 dimensions of the image, respectively, to achieve the extraction of overall and local information, and the discriminator structure is shown in Fig. 3 [21].
The study constructs a model containing a wide variety of loss functions. The content loss is calculated in equation (7), where E denotes the feature extraction of the image by the Relu layer, c denotes the content image, s denotes the style image, and s denotes the stylised image.
Lc= ‖E(T(c,s)) − E(c)‖2 (7)
In equation (8), the style loss is calculated, where Ei denotes the features extracted by the encoder Relui layer.
Ls=
∑n
i=1
‖μEi(T(c,s)) − μEi(s)‖2 (8)
Equation (9), which illustrates the calculation, incorporates both content reconstruction loss and style reconstruction loss in the recon- struction loss. f(c)stands for the content feature map, f(s)for the style feature map, g for the decoder, g(f(c))for the reconstructed content image, and g(f(s))for the rebuilt style image in equation (9).
Lrec=Lrec−c+Lrec−s= ‖ − s+g(f(s))‖2+‖ − c+g(f(c))‖2 (9) The total loss function is calculated in equation (10), with α, β and λ denoting the weighting factors respectively.
Ltotal=αLc+βLs+λLrec (10)
3.2. GANISM model design incorporating TL
Deep learning neural network structures called Generative Adver- sarial Networks (GANs) are used to handle speech, pictures, and natural language in generative models. A GAN consists of a generator and a discriminator; the generator creates new samples by learning the training data, while the discriminator’s primary function is to distin- guish between two different sets of data. During training and iteration, the two compete with each other to continuously improve performance, allowing the generator to produce realistic samples and the discrimi- nator to accurately distinguish the difference between the two. Fig. 4 shows a schematic diagram of the structure. GAN can perform unsu- pervised learning, generate new and high quality data, and GAN has significant application advantages in image transformation, generation and style migration [22,23].
Pdata(x)and Pmodel(x(i);θ)denote the probability distribution func- tions of the real data set and the generative model respectively, the GAN training process continuously updates the parameter θ, the ultimate goal of the model training is to make Pdata(x)and Pmodel(x(i);θ)as consistent as possible. Pmodel(x(i);θ)is calculated from the sampled samples of the real data, the cumulative probability of all real samples in the generative model is the probability function. The parameter θ is updated according to the maximum likelihood estimation theorem to find the θ∗ that maximizes the probability, and the θ∗is updated in equation (11).
θ∗=argmax
θ log∏m
i=1
Pmodel
(x(i);θ)
=argmax
θ
∑m
i=1
Pmodel
(x(i);θ)
=argmax
θ
∫
Pdata(x)logPmodel(x;θ) Pdata(x) dx
(11) Fig. 3.Structure diagram of discriminator.
Fig. 4. Schematic diagram of generating adversarial network structure.
Transforming equation (11) into KL scatter form, the final result is given in equation (12). At this point, the larger the disparity in scatter, the more the discriminator tends to be false and the generator is generated to obtain data that makes the scatter value smaller and the adversarial effect is then formed.
θ∗=argmax
θ KL(Pdata(x)‖Pmodel(x;θ)) (12)
The game process of GAN training is shown in equation (13), where z denotes random noise, x denotes real data, D denotes the discriminator and G denotes the generator. The discriminator and the generator are trained separately, and the training process fixes one side while the other side is trained. The training goal of the discriminator is to make the real sample score higher, and the training goal of the generator is to make the discriminator discriminate the generated data with a higher score.
minG max
D V(D,G) =Ex∼Pdata(x)[log(D(x))] +Ez∼Pdata(z)[log(1− D(G(z)))] (13) When GAN is used for ISM tasks, the traditional GAN model still has some shortcomings, mainly in the excessive degrees of freedom during training leading to poor model training stability, so the research uses a derivative model of GAN, Cycle Generative Adversarial Network (CycleGAN). CycleGAN does not require data pairing to transform images.
The content feature mapping module has been effective in improving the quality of style migration, but the model still has some limitations.
The process of decoupling content and style features of images is in the same model, causing the local effect of style migration to be affected.
The TL is a machine learning method in which the knowledge learned for task A is applied to the training process of task B. The TL consists of a source domain Ds, a target domain Dt, a source task Ts and a target task Tt. The design idea of the TL is to improve the new learning task Tt by learning and transferring knowledge through Ds and Ts, the TL structure is shown in Fig. 5. The study uses a relationship-based TL. this learning relationship establishes mapping relationships between different domains and completes knowledge transfer through correla- tion mapping, avoiding the difficulties of finding feature spaces and the drawbacks of other TL methods whose parameters do not easily converge.
The content and style of the model are first kept as separate as possible, and the network knowledge of this separation is transferred to the style algorithm using TL to enhance the ISM effect. Here, the content features are used to underpin the structure and the style features are used to represent it. A pre-training module is added to the unsupervised image migration algorithm based on an improved attention mechanism, as shown in Fig. 6, to fix the network parameters for the content and style features to prevent the training process from corrupting the fea- tures, and the content feature mapping and generation modules are left unchanged.
Some loss functions are added to the pre-training module, including image, style and content reconstruction losses and adversarial losses, and the various types of loss functions are calculated in equation (14). In equation (14), Lx1 denotes image reconstruction loss, Ls1 denotes style Fig. 5. Structure diagram of transfer learning.
Fig. 6. Structure diagram of pre training module.
Table 1
Qualitative evaluation results of production result images of different models.
Content Style Method Stability Artistic level Image smoothness Significant area retention
Square Realistic Non-shared Excellent Good Good Good
Share Poor level Average Average Average
Block Comic Non-shared Excellent Excellent Good Excellent
Share Average Poor level Average Poor level
Park Color Non-shared Excellent Excellent Average Excellent
Share Average Average Poor level Poor level
Green spaces Traditional Non-shared Good Excellent Excellent Good
Share Poor level Average Average Poor level
Residential Black and White Non-shared Good Good Good Excellent
Share Poor level Average Average Average
reconstruction loss, Ldic1 denotes content reconstruction loss, LDadv and LGadv denote adversarial loss, Ec denotes the capacitive feature module and Es denotes the style feature module. Finally, all loss functions are
summed to obtain the total loss function.
⎧⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎨
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎩ Lx1=Ex
[‖G(Ec(x),Es(x)) − x‖1]
Ls1=Ex
[‖Ec(G(Ec(x),s)) − s‖1]
Ldic1 =Ex
[‖Ec(G(Ec(x),s)) − Ec(x)‖1]
LDadv=1 2Ex
[(D(x) − 1)2] +1
2Ex
[(D(Ec(G(Ec(x),s)))2]
LGadv=1 2Ex
[(D(Ec(G(Ec(x),s))) − 12]
(14)
in addition, a content-invariant loss is added to the model, which is calculated as shown in equation (15).
Ldic1A=ExA,xA→B
[⃦⃦EBC(xA→B) − EAC(xA)⃦
⃦
1
] (15)
4. Improving unsupervised ISM model performance testing An experiment is created here to test the performance of the research-constructed model’s style migration in order to confirm its Fig. 7. Comparison of SSIM and PSNR metrics for generated images under style transfer.
Fig. 8.Comparison of generalization indicators for different model styles transfer.
Table 2
Comparison of indicators for different generated images.
Index Algorithm Square Block Park Green
spaces Residential PSNR GAN-SA 16.584 15.315 12.045 15.496 11.256
GAN-SA-
TL 17.241 16.082 13.633 16.717 12.675
SSIM GAN-SA 0.374 0.314 0.541 0.463 0.606
GAN-SA-
TL 0.478 0.465 0.748 0.609 0.825
FID GAN-SA 25.417 26.489 24.287 26.478 29.142 GAN-SA-
TL 23.145 22.106 21.062 22.136 26.243
LPIPS GAN-SA 0.512 0.461 0.306 0.674 0.462
GAN-SA-
TL 0.524 0.503 0.412 0.741 0.521
efficacy. The experimental dataset was split into a training set and a test set, both of which included content images and style images. The dataset was obtained from the design portfolio of the LD course in landscape architecture at a university’s School of Architecture and Engineering, which contained a total of 14592 design images. The content images include the planning and design of city squares, commercial streets, office buildings, residential environments, urban parks and waterfront green spaces, etc. The style images include traditional, realistic, cartoon, coloured pencil and black and white styles.
4.1. Performance testing of unsupervised image migration algorithms combined with improved attention mechanisms
To verify the effectiveness of the content feature mapping module pairs, the study used whether or not to share content coding as a variable indicator, content images were selected from city squares, commercial
streets, city parks, green space plans and residential areas, and style images were selected from realistic style, comic style, coloured pencil style, traditional style and black and white style. Table 1 displays the findings of the initial qualitative assessment of ISM.
In Table 1, the qualitative results for the non-shared content en- coders are excellent or good in the four metrics of stability, artistry and graphical smoothness, and preservation of significant areas, while the shared content encoders mostly perform at fair or poor levels. The qualitative results show that the non-shared content encoder achieves the greatest stylistic approximation to the reference image, while the shared content encoder limits the use of the Content Feature Mapping module. Therefore, the content feature mapping module can improve the mapping of image features between content domains.
The Multimodal Unsupervised Image-to-Image Translation (MUNIT) model and Domain-Specific Mappings for Generative Adversarial Style Transfer (DSMAP) were chosen for comparison with the research-built model GAN-SA in order to further compare the various models’ style transfer abilities. Fig. 7 illustrates the experimental outcomes using the evaluation metrics of Structural Similarity Index Metric (SSIM) and Peak Signal-Noise Ratio (PSNR).
As seen in Fig. 7, the highest PSNR metrics of GAN-SA occupied four out of the five image groups, 9.18, 13.27, 12.61 and 17.42 respectively;
the SSIM metrics were the highest in all five image groups, with the highest value reaching 0.91, indicating a high degree of image similar- ity. The MUNIT model did not establish the relevant feature mapping, and its two metrics performed slightly worse than GAN-SA. For the city square class, the lowest PSNR value was 8.62 and the lowest SSIM value was 0.22, which were 0.56 and 0.16 lower than the corresponding in- dicators of GAN-SA. DSMAP and GAN-SA include content feature map- ping, but they change the structural characteristics of the original image too much, resulting in low SSIM values and PSNR metrics, with SSIM values in the range of 0.04–0.16 and PSNR metrics in the range of 5.4–11.0, with low similarity. For the style migration of different types of images, the image generation effect of GAN-SA is better than other models, and the structural features of the original image are modified while maintaining the similarity between the two.
4.2. Analysis of the effectiveness of unsupervised image migration models for LD applications
Frechet Inception Distance (FID) and Learned Perceptual picture Patch Similarity (LPIPS) were employed as evaluation metrics in ex- periments using various LD graph types and uncommon picture kinds.
Fig. 9. Style transfer quality of different GAN models.
Fig. 10.Model loss function curve.
Table 3
Model image style migration effect.
The results of the generalization performance of the model are shown in Fig. 8. As can be seen from Fig. 8, the image FID of GAN-SA is at the lowest level and the image similarity is high, with a minimum FID value of only 16.32. The LPIPS score of GAN-SA is the highest for all four groups of images, with a maximum value of 0.52. The LPIPS metric reflects the diversity of the generated images, and the GAN-SA model has the highest image diversity, which is beneficial to the presentation of the LD results. However, the LPIPS scores of the other two models also performed relatively well, with values in the range of 0.3–0.4. This shows that the content feature mapping module has little influence on the diversity of the generated images.
Since GAN-SA-TL represents the model with TL introduced, the values of the four evaluation metrics of the two models were compared using this as a variable to confirm the effectiveness of the pre-training module using TL. Table 2 shows that GAN-SA-TL outperformed GAN- SA in terms of performance metrics, with the PSNR and SSIM metrics showing more pronounced growth trends, with the PSNR values increasing by 0.657, 0.767, 1.588, 1.221 and 1.419 and the SSIM metrics increasing by 0.104, 0.151, 0.207, 0.146 and 0.219 respectively. The FID values of GAN-SA-TL decreased by 2.272, 4.383, 3.225, 4.342, and 2.899, respectively, and the degree of correlation between the stylised image and the real image improved. The LPIPS metrics did not differ significantly, with a maximum difference of only 0.106, and the two had no significant effect on the image. The difference between the LPIPS metrics is not significant, with a maximum difference of 0.106, and there is no significant difference between the two for the change in image style diversity.
The experimental results of the style migration effect application comparison between GAN-SA-TL and two traditional GAN models are shown in Fig. 9. As seen in Fig. 9, the PSNR metrics and SSIM values of the GAN-SA-TL model are both stable at the highest level, and the model has the best style migration effect, which is better than the existing GAN and CycleGAN models. After learning all the images in the training set, the PSNR metric of the GAN-SA-TL model finally stabilised at around 20.00 and the SSIM value finally stabilised at around 0.9.
Several picture datasets were used to assess the model’s stability. The experimental results are depicted in Fig. 10, which shows that the loss function drops gradually and with minimal volatility. Given the nar- rower gap between generated and real images and the model’s improved robustness, the GAN-SA-TL model’s improved network stability signifi- cantly reduces the loss function’s curve oscillation in later stages.
Finally, to confirm the model’s practicality, the effect of style migration on landscape design images is discussed. Four different style images are selected to correspond with the two primary landscape design programmes, generating the relevant style migration images. The results of these migrations are displayed in Table 3. As observed in Table 3, the structural characteristics of the initial image alter corre- sponding to the various style images. The style of the produced image closely resembles that of the style images, resulting in an improved overall image quality that is advantageous for the landscape design image’s style rendering.
5. Conclusion
Traditional landscape design is traditionally based on the subjective aesthetics and experience of the designer. However, image style migration technology can provide a new design method for landscape design. Nevertheless, the current image style migration model still en- counters style migration distortion troubles such as blurriness or loss of image texture, edges, and details. In order to address the issue of subpar image generation quality in the current image style transfer models, a landscape design-specific image style transfer model was developed. The qualitative and quantitative evaluations of the model quality are con- ducted respectively. The qualitative evaluation results indicated that the content feature mapping module was constrained by the shared content encoder. In contrast, the non-shared content encoder generated an
image with the highest degree of stylistic alignment to the reference image, and the content feature mapping module enhanced the image’s feature mapping effect. In the quantitative evaluation, the highest value of PSNR metric of GAN-SA reached 17.42 and the highest value of SSIM metric reached 0.91, the lowest PSNR and SSIM metrics of MUNIT were lower than the corresponding metrics values of GAN-SA by 0.56 and 0.16, respectively; the DSMAP had changed the structural characteris- tics of the model too much, and the performance was not good enough.
The metrics of GAN-SA model were at the same level of the comparison models. The values were the highest of the models compared, and the model produced images with less distortion and higher quality. In the application effect analysis, GAN-SA showed better generalization, with the smallest FID value of only 16.32. The FID value was a character- ization of the difference between the shallow feature information and the high-level abstracted features between the images, which indicated that the correlation of the image generation was high. However, the difference in LPIPS scores between different models was small, and the content feature mapping module had little impact on the diversity of image generation. Migration learning further improved the quality of image generation, compared with the two traditional GAN models, the PSNR values of the GAN-SA-TL model were improved by 0.657, 0.767, 1.588, 1.221, and 1.419, respectively; the SSIM metrics were improved by 0.104, 0.151, 0.207, 0.146, and 0.219, respectively; and the FID values were decreased by 2.272, 4.383, 3.225, 4.342, 2.899, and the overall performance was further improved. The migration quality, network stability, loss function curve oscillation, and overall perfor- mance of the research-improved model are all better than those of the conventional GAN model. However, the computational efficiency of the model still needs to be increased, and this could be the subject of the following research phase.
Declaration of competing interest
The authors declare that they have no competing interests.
Data availability
Data will be made available on request.
References
[1] Y.G. Zhang, T.T. Zhang, Design and planning of urban ecological landscape using machine learning, Int. J. Grid Util. Comput. 13 (1) (2022) 3–10.
[2] J. Cortes˜ao, S. Lenzholzer, Research through design in urban and landscape design practice, J. Urban Des. 27 (6) (2022) 617–633.
[3] Y. Yang, X. Song, Research on face intelligent perception technology integrating deep learning under different illumination intensities, J. Comput. Cognit. Eng. 1 (1) (2022) 32–36.
[4] F. Yang, Y.X. Wang, L. Herranz, Y.M. Cheng, M.G. Mozerov, A novel framework for image-to-image translation and image compression, Neurocomputing 508 (1) (2022) 58–70.
[5] C. Liu, B.Q. Xu, A night pavement crack detection method based on image-to-image translation, Comput. Aided Civ. Infrastruct. Eng. 37 (13) (2022) 1737–1753.
[6] C. Xie, Z.Z. Wang, H.B. Chen, X.L. Ma, W. Xing, L. Zhao, W. Song, Z.J. Lin, Image style transfer algorithm based on semantic segmentation, IEEE Access 9 (1) (2021) 274–287.
[7] M. Kim, H.C. Choi, Uncorrelated feature encoding for faster image style transfer, Neural Network. 140 (1) (2021) 148–157.
[8] H. Ding, G. Fu, Q. Yan, C. Jiang, T. Cao, W. Li, S. Hu, C. Xiao, Deep attentive style transfer for images with wavelet decomposition, Inf. Sci.: Int. J. 587 (587) (2022) 63–81.
[9] Y. Li, Y. Wang, H.W. Tseng, H.K. Hang, C. Chen, Optimized and improved methods of image style transfer for local reinforcement, Sensor. Mater.: An Int. J. Sensor Technol. 33 (9) (2021) 3325–3332, pt.4.
[10] D. Chen, L. Yuan, J. Liao, N. Yu, G. Hua, Explicit filterbank learning for neural image style transfer and image processing, IEEE Trans. Pattern Anal. Mach. Intell.
43 (7) (2021) 2373–2387.
[11] L. Huang, P. Wang, C.F. Yang, T. Hsien-Wei, Rapid local image style transfer method based on residual convolutional neural network, Sensor. Mater.: An Int. J.
Sensor Technol. 33 (4) (2021) 1343–1352. Pt.2.
[12] Y.S. Liao, C.R. Huang, Semantic context-aware image style transfer, IEEE Trans.
Image Process. 31 (1) (2022) 1911–1923.
[13] Y. Zhou, K. Yu, M. Wang, Y. Ma, Y. Peng, Z. Chen, W. Zhu, F. Shi, X. Chen, Speckle noise reduction for OCT images based on image style transfer and conditional GAN, IEEE j. biomed. health inf. 26 (1) (2022) 139–150.
[14] C A M Kay, A T Rohnke, H A Sander, T Stankowich, Fidino M Mason, Barriers to building wildlife-inclusive cities: Insights from the deliberations of urban ecologists, urban planners and landscape designers, People Nat. 4 (1) (2022) 62–70.
[15] S.A. Zakaria, A. Bahauddin, Light art for historical buildings: a case study of the heritage buildings in george town penang island, in: Procedia - Social and Behavioral Sciences, vol. 184, Elsevier, 2015, pp. 345–350, 1.
[16] S.K. Sweeney, P. Newbill, T. Ogle, K. Terry, Using augmented reality and virtualenvironments in historic places to scaffold historical empathy, TechTrends, TechTrends 62 (1) (2018) 114–118.
[17] C. Che, H. Wang, X. Ni, Q. Fu, Performance degradation prediction of aeroengine based on attention model and support vector regression, Proc. Inst. Mech. Eng. G J.
Aerosp. Eng. 236 (2) (2022) 410–416.
[18] Y. Chen, G.L. Wei, J.X. Liu, Y.W. Chen, Q.H. Zheng, F. Tian, H.P. Zhu, Q.Y. Wang, Y.Q. Wu, A prediction model of student performance based on self-attention mechanism, Knowl. Inf. Syst. 65 (2) (2022) 733–758.
[19] H. Yang, L.Y. Wang, Y.T. Xu, X.H. Liu, CovidViT: a novel neural network with self- attention mechanism to detect Covid-19 through X-ray images, Int. J. Mach. Learn.
Cybern. 14 (3) (2022) 973–987.
[20] T. Zhang, Z.J. Zhang, W.J. Jia, X.J. He, J. Yang, Generating cartoon images from face photos with cycle-consistent adversarial networks, Comput. Mater. Continua (CMC) 69 (2) (2021) 2733–2747.
[21] Z. Wang, N. Xu, B.H. Wang, Y.H. Liu, S.W. Zhang, Urban building extraction from high-resolution remote sensing imagery based on multi-scale recurrent conditional generative adversarial network, GIScience Remote Sens. 59 (1) (2022) 861–884.
[22] Z.J. Yang, Z. Qiu, An image style diversified synthesis method based on generative adversarial networks, Electronics 11 (14) (2022) 75–97.
[23] X.H. Li, Z.S. Du, Y.Y. Huang, Z.Y. Tan, A deep translation (GAN) based change detection network for optical and SAR remote sensing images, ISPRS J.
Photogrammetry Remote Sens. 179 (1) (2021) 14–34.