Intelligent Modeling of University Cultural Content via Cross-Modal Image Processing

Within this module, the input image is partitioned into uniformly sized image patches, from which foundational image embedding features are generated through linear projection and positional encoding, thereby completing the structured representation of low-level visual information. To guide the model in prioritizing target regions that carry core cultural connotations, a detection model fine-tuned on the campus cultural dataset is employed to generate a global saliency heatmap, through which the cultural semantic density of each image region is quantified. The heatmap is then downsampled to match the image patch granularity, from which normalized saliency scores are obtained. These scores are incorporated as bias terms into the self-attention computation process, and the optimized attention computation formula can be expressed as:

$\operatorname{Attn}_{i, j}=\operatorname{softmax}\left(\frac{\left(W_Q e_i\right)\left(W_K e_j\right)^T}{\sqrt{d_k}}+\lambda \cdot\left(s_i+s_j\right)\right)$    (1)

where, $W_Q$ and $W_K$ denote the query and key projection matrices, respectively; $d_k$ is the attention head dimension; $s_i$ and $s_j$ are the saliency scores of the corresponding image patches; and $\lambda$ is used to regulate the constraint strength of the saliency prior. Through this mechanism, the feature responses of cultural entity regions are enhanced, while ineffective background interference is suppressed, on the basis of original semantic relevance modeling. Ultimately, a multi-scale Transformer feature set is produced, encompassing hierarchical semantic information ranging from local textural details to global cultural scenes, thereby providing comprehensive preliminary representational support for the subsequent feature decoupling task.

To address the challenge of high coupling between low-level visual morphology and high-level cultural semantics in university cultural images, an orthogonal constraint-based cultural feature decoupling network is constructed in this study, through which the explicit separation and independent representation of these two heterogeneous information types are achieved. Two parameter-independent linear projection branches are established within the network, and the multi-layer visual features output by the encoder are mapped into subspaces via these branches, from which feature matrices specialized for visual morphological representation and cultural semantic representation are respectively generated. The mapping process can be formulated as follows:

$Z_{\text {shape }}^{(l)}=\sigma\left(W_{\text {shape }} F_v^{(l)}+b_{\text {shape }}\right)$    (2)

$Z_{\text {cult }}^{(l)}=\sigma\left(W_{\text {cult }} F_v^{(l)}+b_{\text {cult}}\right)$     (3)

where, $W_{\text {shape }}, W_{\text {cult }}, b_{\text {shape }}$, and $b_{\text {cult }}$ denote the weights and biases of the two projection branches, respectively; and $\sigma$ represents the rectified linear unit activation function. The morphological subspace features preserve physical visual information, including color, texture, and spatial composition, while the cultural subspace features are dedicated to storing humanistic semantic information, including historical connotations and symbolic meanings. To prevent re-entanglement of the two feature types, an orthogonality loss based on the Frobenius norm is introduced to constrain subspace independence:

$L_{\text {orth }}=\sum_l\left\|\left(Z_{\text {shape }}^{(l)}\right)^T Z_{\text {cult }}^{(l)}\right\|_F^2$    (4)

Simultaneously, a morphological reconstruction loss and a cultural classification loss are introduced to constrain the representational effectiveness of the two subspaces, respectively, ensuring that the morphological features can fully reconstruct the visual content of the image and that the cultural features possess precise semantic discriminative capability. The overall loss function of the module is optimized through the weighted integration of the three constraints:

$L_{\text {disentangle }}=L_{\text {orth }}+\mu_1 L_{\text {recon }}+\mu_2 L_{\text {cult-cls }}$   (5)

where, $\mu_1$ and $\mu_2$ are hyperparameters used to balance the optimization weights of the different loss terms, ensuring that the decoupling process accommodates both information completeness and semantic specificity.

Following feature decoupling, an adaptive gating feature interaction unit is designed in this study, through which the dynamic fusion of visual morphological features and cultural semantic features is achieved, accommodating the representational requirements of diverse campus cultural scenes. Within this unit, the two-layer decoupled features are first concatenated along the channel dimension, and a single-layer perceptron with a Sigmoid activation function is employed to generate pixel-wise adaptive gating coefficients, through which the dynamic assessment of scene feature weights is accomplished:

$g^{(l)}=\operatorname{sigmoid}\left(W_g\left[Z_{\text {shape }}^{(l)} ; Z_{\text {cult }}^{(l)}\right]+b_g\right)$   (6)

where, $W_g$ and $b_g$ denote the learnable parameters of the gating unit, and the gating coefficients take values in the range of $[0,1]$. Based on the obtained weights, the weighted fusion of the dual features is performed, and the optimized multiscale features are finally output as follows:

$F_{\text {fuse }}^{(l)}=g^{(l)} \odot Z_{\text {shape }}^{(l)}+\left(1-g^{(l)}\right) \odot Z_{\text {cult }}^{(l)}$   (7)

where, ⊙ denotes element-wise multiplication. Through this adaptive mechanism, the feature proportions can be dynamically adjusted according to the content characteristics of the image: for texture-rich modern campus scenes, the preservation of visual morphological features is prioritized, whereas for culturally rich scenes such as historical architecture and cultural relic imagery, the weighting of semantic features is enhanced. This enables precise matching between feature representations and scene attributes.

The complete encoding and decoupling representation system optimizes the representation quality of cultural images from the feature extraction source. Through the saliency attention mechanism, key cultural features are purified; through orthogonal constraints, the precise decoupling of heterogeneous information is achieved; and through the gated fusion mechanism, adaptive feature optimization is accomplished. The final output fused features possess both complete visual structural information and highly discriminative cultural semantic information, fundamentally resolving the issues of semantically mixed representations and weakened cultural features inherent in conventional visual representations. A high-quality visual representation foundation is thereby provided for subsequent cross-modal semantic alignment and culturally faithful image generation tasks.

2.4 Knowledge graph-guided hierarchical contrastive alignment mechanism

To overcome the limitations of conventional image-text contrastive learning, which relies solely on global sample-pair supervision and cannot adequately accommodate the hierarchical semantic system of university culture, a structured university cultural knowledge graph is introduced in this study, through which a multi-granularity semantic constraint system is constructed, and fine-grained alignment of cross-modal features is achieved via progressive hierarchical contrastive learning. The schematic diagram of the knowledge graph-guided hierarchical cross-modal contrastive alignment mechanism is illustrated in Figure 3. The university cultural knowledge graph is defined as a structured set of triples $G=(\mathrm{E}, \mathrm{R}, T)$, where E, R, and $T$ denote the set of cultural entities, the set of semantic relations, and the set of knowledge triples, respectively. The RotatE algorithm is employed to accomplish the vectorized representation of the knowledge graph, through which discrete cultural entities and semantic relations are mapped into a continuous vector space of dimension $d_k$, and entity embedding features (e) and relation embedding features (r) are generated as output. The embedding dimension is maintained in strict consistency with the representation dimensions of the visual and textual modalities, thereby enabling seamless integration across multimodal spaces. The knowledge embedding process is trained via a standard triple ranking loss and is jointly optimized with the cross-modal contrastive task. Through entity-level feature mapping, deep coupling between the knowledge semantic space and the visual-textual space is achieved, providing stable domain prior support for multi-level fine-grained semantic alignment.

Figure 3. Schematic diagram of the knowledge graph-guided hierarchical cross-modal contrastive alignment mechanism

A three-level progressive contrastive learning paradigm—encompassing sample, entity, and relational levels—is constructed in this study, and a unified optimization objective is formulated through the weighted integration of multi-granularity constraint losses, through which cross-modal semantic features are progressively converged from coarse to fine granularity. The overall loss function is expressed as:

$L_{C L}=\alpha L_{\text {ins }}+\beta L_{\text {entity }}+\gamma L_{\text {rel }}$     (8)

where, $\alpha, \beta$, and $\gamma$ denote the weight hyperparameters of the three loss terms, used to balance the optimization strengths of global sample matching, entity-level semantic alignment, and topological relationship constraints. Sample-level contrastive learning serves as the fundamental alignment unit, through which the matching optimization of global image and text features is accomplished via the information noise-contrastive estimation loss:

$L_{i n s}=-\frac{1}{B} \sum_{i=1}^B \log \frac{\exp \left(\operatorname{sim}\left(f_v\left(I_i\right), f_t\left(T_i\right)\right) / \tau\right)}{\sum_{j=1}^B \exp \left(\operatorname{sim}\left(f_v\left(I_i\right), f_t\left(T_j\right)\right) / \tau\right)}$   (9)

where, $B$ denotes the training batch size; $f_v$ and $f_t$ are the feature mapping functions of the visual encoder and the text encoder, respectively; sim denotes the cosine similarity between vectors; and $\tau$ is the temperature coefficient used to regulate the dispersion of the feature distribution. On this basis, an entity-level contrastive constraint is further introduced, through which the conventional global coarsegrained matching paradigm is abandoned. Standardized cultural entities from the knowledge graph are adopted as semantic anchors, and a fine-grained contrastive loss is constructed via a same-category heterogeneous negative sample sampling strategy:

$L_{\text {entity }}=-\frac{1}{B} \sum_{i=1}^B \log \frac{\exp \left(\operatorname{sim}\left(f_v\left(I_i\right), e_i\right) / \tau\right)}{\sum_{e^{\prime} \in N\left(e_i\right)} \exp \left(\operatorname{sim}\left(f_v\left(I_i\right), e^{\prime}\right) / \tau\right)}$ (10)

Through this strategy, negative sample sets are constructed from same-source broad-category cultural entities, through which the model's discriminative capability for similar cultural scenes and same-category cultural symbols is effectively enhanced, and visual and textual features are precisely mapped to culture-specific entity semantics, thereby circumventing the matching ambiguity issues caused by superficial textual descriptions.

To further preserve the structured associative characteristics of the cultural system, a relationship-level contrastive loss is introduced to constrain the topological consistency of the embedding space, addressing the deficiency that isolated entity alignment cannot represent semantic associations. This constraint is based on the established entity association rules of the knowledge graph, through which the model is forced to learn the fixed semantic offset regularities among entities. The loss function is defined as:

$L_{r e l}=\sum_{\left(e_i, r, e_j\right) \in T}\left\|\left(f_v\left(I_i\right)-f_v\left(I_j\right)\right)-r\right\|_2^2$       (11)

This formulation constrains the visual feature difference between any two associated entities to remain consistent with the predefined relation embedding vector from the knowledge graph, thereby enabling the cross-modal semantic space to fully inherit the structured logic of the knowledge graph, and achieving an upgrade from isolated feature matching to systematic semantic modeling. The three-tier contrastive constraints form a progressively layered optimization system: the basic sample-level loss ensures the accuracy of global image-text matching, the entity-level loss strengthens the discriminative capability of fine-grained cultural semantics, and the relationship-level loss solidifies the topological structure of cultural knowledge. Ultimately, a cross-modal semantic space possessing both global matching precision and local structural integrity is constructed, providing a highly consistent representational foundation for subsequent culturally faithful semantic preservation generation tasks.

2.5 Cultural semantic-preserving diffusion-based enhancement and generation model

To address the issue of cultural information drift in conventional diffusion models during cultural image restoration and generation—caused by excessive optimization of visual textures without constraints on semantic consistency—a cultural semantic-preserving diffusion-based enhancement and generation model is constructed in this study. The structured cultural semantic features output by the preceding modules are incorporated as prior conditions into the diffusion inference process, through which the collaborative optimization of visual quality enhancement and cultural semantic fidelity is achieved. The schematic diagram of the cultural semantic-preserving diffusion-based enhancement and generation model is illustrated in Figure 4. The forward diffusion process of the model follows the standard Gaussian noise iterative superposition paradigm, in which progressive noise perturbations are applied to the original clear image, and a continuous noise degradation mapping relationship is established. This process can be formulated as:

$I_t=\sqrt{\bar{\alpha}_t} I_0+\sqrt{1-\bar{\alpha}_t} \epsilon, \epsilon \sim N(0, I)$    (12)

where, $I_0$ denotes the original input image; $I_t$ denotes the noisy image at step $t ; \alpha_t$ is the predefined noise scheduling coefficient; $\bar{\alpha}_t$ is the cumulative product of the coefficients from the first $t$ steps; and $\epsilon$ is a Gaussian noise vector following a standard normal distribution. The core improvement of this study is concentrated in the diffusion reverse denoising stage, where cultural semantic conditional guidance is introduced on the basis of the original unconstrained denoising network, and a conditional probability generation model is constructed:

$p_\theta\left(I_{t-1} \mid I_t, c\right)=N\left(I_{t-1} ; \mu_\theta\left(I_t, t, c\right), \Sigma_\theta\left(I_t, t, c\right)\right)$    (13)

where, $\mu_\theta$ and $\Sigma_\theta$ denote the learnable mean and variance prediction networks, respectively; and $c$ is the standardized 

cultural semantic condition vector, which is obtained through the fusion of knowledge graph entity embeddings and the aligned textual semantic features. This condition vector is embedded into the multi-layer feature layers of the UNet network via a cross-attention mechanism, through which global semantic constraints are imposed on each denoising iteration step, ensuring that the denoising optimization process of the model remains consistently aligned with the inherent semantic logic of university culture.

For the restoration and super-resolution enhancement tasks of low-quality historical campus images, a cultural semantic preservation constraint is incorporated into the standard diffusion noise prediction loss, and a multi-objective joint optimization paradigm is constructed, through which cultural feature distortion during the restoration process is fundamentally prevented at the semantic level. The model takes the low-quality image $I_{\text {high}}$ as input and the corresponding high-quality original image $I_{\text {low}}$ as the visual supervisory benchmark, through which the image quality restoration mapping is accomplished. To ensure that the cultural connotations of the enhanced image remain completely consistent with those of the original image, the cultural feature encoder trained in Section 2.3 is employed to extract high-level semantic features, with the parameters of this encoder frozen during the diffusion model training process, thereby ensuring the stability and discriminability of the cultural representations. The corresponding cultural semantic preservation loss is defined as:

$L_{\text {cult-preserve }}=\|$ Enc$_{\text {cult }}(\hat{I})-$ Enc$_{\text {cult}}\left(I_{\text {high}}\right) \|_2^2$    (14)

where, $\hat{I}$ denotes the enhanced image output by the model, and Enc$_{\text {cult}}$ represents the cultural semantic encoding function. Through the minimization of the cultural feature distance between the enhanced image and the ground-truth high-quality image, the model is forced to preserve the complete cultural symbol morphology, historical scene style, and deep humanistic semantics, while simultaneously optimizing visual metrics such as textural details and sharpness. The deficiencies of conventional image enhancement methods—including cultural element deformation and semantic loss—are thereby effectively addressed.

Leveraging the unified diffusion model backbone architecture, knowledge-guided controllable cultural image generation is further implemented, through which the precise generation and style replication of arbitrary campus cultural scenes is accomplished. To integrate structured knowledge information with fine-grained textual attribute information, the cultural entity embeddings and scene attribute text features are concatenated along the channel dimension, and a multi-layer perceptron is employed to accomplish feature dimension fusion and semantic calibration, from which a unified condition vector adapted for diffusion model input is generated:

$c=M L P([e ; d])$  (15)

where, e denotes the knowledge graph cultural entity embedding, and d denotes the scene attribute text features. Using the fused semantic vector as the constraint condition, iterative denoising sampling is performed on the random Gaussian noise variable, and a generated image conforming to cultural semantic specifications is finally output. The generation process can be expressed as:

$I_{\text {gen }}=D D P M_\theta(z, c), z \sim N(0, I)$ (16)

To further enhance the historical authenticity and stylistic consistency of the generated content, a cultural style consistency loss is introduced. Through the statistical analysis of the color distributions, textural features, and period-specific style regularities of real historical campus images, the visual style distribution of the generated images is constrained to remain consistent with that of authentic cultural scenes, thereby effectively avoiding issues such as anachronistic temporal features and culturally incongruent elements in the generation results.

This module fully inherits the outputs of the preceding feature decoupling and hierarchical semantic alignment stages, with the purified high-precision structured cultural semantics incorporated as constraint priors throughout the entire training and inference process of the diffusion model. In contrast to the optimization logic of generic diffusion models, which focus exclusively on visual pixel fitting, this paradigm achieves the bidirectional constraint of visual appearance optimization and cultural semantic fidelity, and establishes an image enhancement and generation mechanism adapted to university cultural scenes. Both the visual quality and detailed fidelity of the output images are thereby ensured, while the complete preservation of cultural symbols, historical styles, and humanistic connotations is simultaneously achieved, providing reliable technical support for the digital regeneration of university culture.

2.6 Unified cross-modal semantic space construction and inference

To eliminate the modal heterogeneity biases among visual, textual, knowledge graph, and diffusion-generated features, and to achieve the fusion, reuse, and general-purpose inference of multi-source cultural information, a cross-modal semantic bridging layer is constructed in this study, through which the four types of heterogeneous features are uniformly mapped into a shared semantic space S of dimension d. The semantic bridging layer is equipped with independent linear projection matrices and layer normalization units for each modality, through which feature dimension unification and distribution standardization are accomplished while preserving the inherent semantic characteristics of each modality, and the issues of spatial misalignment and scale inconsistency across different representation systems are fundamentally resolved at the structural level. To achieve deep fusion and alignment of multimodal features, a joint optimization loss function is constructed from two dimensions: global distribution matching and local semantic constraints. The overall unification loss is expressed as:

$L_{u n i f y}=\sum_{m, n \in\{v, t, k, d\}} K L\left(p_m \| p_n\right)+\sum_i\left\|s_i^{(v)}-s_i^{(t)}\right\|_2^2$  (17)

where, $v, t, k$, and $d$ denote the visual, textual, knowledge graph, and diffusion-generated modalities, respectively; $p_m$ and $p_n$ represent the feature probability distributions of different modalities; and the Kullback-Leibler divergence is employed to minimize the overall distribution discrepancies across modalities, thereby achieving global spatial distribution alignment. $s_i^{(v)}$ and $s_i^{(t)}$ denote the sample-paired unified semantic vectors of the visual and textual modalities, and the cross-modal representation distances of same-source samples are minimized through L2 distance constraints, thereby ensuring the precision of fine-grained semantic matching.

Based on the optimized unified semantic space, all modal inputs can be transformed into standardized, same-dimensional cultural semantic vectors, through which a general-purpose representation system adapted to university cultural scenes is constructed. This space fully inherits the output advantages of the preceding feature decoupling, hierarchical semantic alignment, and semantically constrained generation modules, achieving deep fusion of the representational outputs from multiple modules. Without requiring parameter fine-tuning for individual downstream tasks, stable support is provided for various intelligent inference tasks, including cross-modal retrieval, image caption generation, visual question answering, and controllable cultural image generation. Through this design, the task generalization capability and scene adaptability of the overall framework are effectively enhanced, and the unified and integrated modeling of cultural image understanding, semantic matching, and visual generation tasks is realized.

Within the complete framework, the feature decoupling, hierarchical semantic alignment, and semantically constrained generation modules form a complete closed-loop optimization chain, with each unit achieving gradient intercommunication and bidirectional empowerment through the unified cross-modal semantic space, without isolated independent operation. The feature decoupling network accomplishes the subspace separation of visual morphological information and cultural semantic information, eliminating the interference of irrelevant textures and background noise on cultural representations, providing high-purity semantic features for subsequent hierarchical contrastive learning, and reducing the confounding errors of fine-grained cross-modal matching at the source. Through the three-tier contrastive constraints constructed on the basis of the knowledge graph, discrete visual representations are mapped into a unified semantic space endowed with structured logic, and standardized cultural vectors are output as generation conditions for the diffusion model, constraining the content direction of image enhancement and generation processes from the semantic level. Concurrently, the semantic preservation loss embedded within the image generation and restoration branches enables the backpropagation of optimization gradients, continuously refining the front-end feature extraction and semantic alignment processes. The semantic fidelity effects of the generation tasks are used to verify the optimization degree of the representation and alignment stages, thereby forming a complete technical chain in which feature purification, semantic anchoring, and visual regeneration mutually support one another.

The performance of the proposed method is constrained by the completeness of annotation resources and the visualizability of cultural content, with clearly defined application boundaries. For niche campus cultural categories with scarce sample quantities and insufficient annotation materials, the model cannot adequately learn the representation regularities of culture-specific entities, resulting in significant performance degradation in retrieval and generation tasks. For abstract spiritual and cultural connotations, the model cannot achieve complete modeling based solely on image visual information, and still relies on accompanying textual descriptions and knowledge graph triples to provide semantic supplementation. In the absence of structured annotations, semantic alignment accuracy declines markedly. The framework possesses fundamental transferability to other cultural heritage scenarios. For vertical cultural domains such as ancient architecture and museum collections, adaptation can be accomplished by simply replacing the domain-specific knowledge graph and performing minor fine-tuning of the visual encoding layer to match the visual characteristics of the target scenario. Currently, the model covers only three modalities—image, text, and knowledge graph—and has not yet integrated multi-source cultural data such as audio and three-dimensional point clouds, leaving room for expansion in multimodal coverage.

From the perspective of digital humanities and the practical implementation of university cultural digitization projects, the integrated cross-modal modeling system constructed in this study fills the specialized technical gap in intelligent cultural image processing and possesses multi-level engineering application value. Through the capabilities of the framework, tasks such as batch restoration of historical campus photographic archives, intelligent retrieval of campus cultural resources, and automatic generation of customized campus cultural scenes can be accomplished, supporting the development of digital products including digital university history museums, online cultural exhibition halls, and general education cultural courseware. Traditional university cultural digitization efforts have largely remained at the level of material storage and simple image beautification, without achieving deep semantic parsing or controllable regeneration of cultural content. The proposed method bridges the integration pathway between visual image processing and structured cultural knowledge, completely preserving historical symbols, architectural forms, and humanistic connotations while ensuring visual output quality, providing a standardized intelligent solution for the long-term digital preservation and lightweight online dissemination of university cultural resources, while also offering a reusable technical paradigm for vertical-scenario modeling at the intersection of digital humanities and computer vision.

Furthermore, the proposed framework provides an extensible pathway for the institution-specific development of distinctive university cultural resources. For application-oriented higher education institutions with distinctive regional cultural characteristics, industrial backgrounds, or educational traditions, visual resources such as local historical architecture, industrial memory, regional cultural landscapes, and university-industry collaboration scenes can be incorporated into the unified cross-modal semantic space. Through image enhancement, semantic annotation, knowledge association, and content generation, scattered local cultural materials can be transformed into teaching cases, digital exhibition resources, virtual simulation materials, and campus cultural dissemination content. This process enables bidirectional transformation among local resources, university culture, and digital teaching resources, enhancing the scene adaptability, institutional distinctiveness, and application promotion value of university cultural content modeling.