Radiomics-Enhanced 3D ResNet for Multi-Feature Fusion in Adrenal Tumor Diagnosis

2.1 Model architecture

This study proposes an integrated framework combining radiomics and deep learning to enhance diagnostic accuracy for adrenal gland patients [17]. The schematic diagram of the proposed framework is illustrated in Figure 1. The overall architecture consists of three key modules: (1) Radiomics-based low-level feature extraction, (2) 3D ResNet-based high-level feature extraction, and (3) LASSO-based feature selection. Traditional methods often rely solely on deep learning, which tends to prioritize high-level semantic representations while overlooking vital low-level structural details [18]. Our multi-dimensional framework addresses this by capturing tumor characteristics across distinct feature dimensions. In the radiomics phase, CT images are processed using the pyradiomics library, where various filters are applied to enhance diverse image attributes. In the deep learning phase, a 3D ResNet network is employed to exploit the volumetric spatial information inherent in CT scans [19]. Patient slices are preprocessed via localization to standardize sequence lengths. Finally, to mitigate redundancy and eliminate irrelevant information within the high-dimensional feature space, LASSO regression is applied to shrink non-informative coefficients to zero. The resulting subset of filtered features is then fed into a classifier for precise lesion diagnosis.

Figure 1. The overall model of multi-dimensional feature extraction

 2.2 Radiomics feature extraction

Radiomics enables the high-throughput extraction of quantitative features from medical images, converting visual data into mineable datasets [20, 21]. We utilized the pyradiomics library to extract features from the Region of Interest (ROI) in CT images. To fully extract the features of the patient's lesions, we set up a variety of filters. The filters are mainly used to enhance specific features of the image to help extract richer texture features. Different filters have different feature extraction effects. The specific screening process is shown in Figure 2. We applied nine types of filters: Original, Wavelet (for local texture), LoG (Laplacian of Gaussian, for edge enhancement), Square, SquareRoot, Logarithm, Exponential, Gradient (for structural changes), and LBP (Local Binary Pattern) [17, 21]. These filters generated seven categories of radiomics features, totaling 1,345 features (298 first-order, 17 shape, and 1,030 texture features). The texture features include Gray Level Co-occurrence Matrix (GLCM), Gray Level Size Zone Matrix (GLSZM), Gray Level Run Length Matrix (GLRLM), Neighboring Gray Tone Difference Matrix (NGTDM), and Gray Level Dependence Matrix (GLDM) [8, 16].

First-order features describe the voxel intensity distribution. For instance, Entropy is defined as:

$entropy=-\sum_{i=1}^{N_g} p(i) \log _2(p(i)+\epsilon)$          (1)

where, p(i) denotes the normalized first-order histogram, N_g represents the number of voxels, and $\epsilon \approx 2.2 \times 10^{-16}$. The entropy metric primarily measures average information, reflecting the uncertainty and randomness of image values. Shape features describe the 3D geometry of the ROI, independent of gray intensity [16]. The surface area is calculated using a triangular mesh representation, where the total surface area A is the sum of the areas of all triangles in the mesh, indicating the model's complexity.

$A_i=\frac{1}{2}\left|a_i b_i \times a_i c_i\right|$              (2)

$A=\sum_{i=1}^{N_f} A_i$            (3)

In a triangular mesh, $a_i b_i$ and $a_i c_i$ represent the two edges of the $i$ triangle, with its vertices defined by $a_i, b_i$, and $c_i$. The total number of triangles in the mesh, $N_f$, indicates the mesh's complexity. A denser mesh corresponds to a more detailed 3D model, and the total surface area $A$ is calculated by summing the areas of all triangles.

Finally, we extract 1345 features including 298 first-order features, 17 shape features and 1030 texture features, which are used as the low-level features of the model [17].

 Figure 2. Low-dimensional feature extraction process

 2.3 3D ResNet combined with spatially displaced MLP for high-dimensional feature extraction 

3D ResNet is a powerful feature extraction neural network that enhances traditional ResNet by incorporating temporal dimensions to better capture spatial information in 3D data [22]. To ensure consistency, we preprocessed CT images of adrenal patients by precisely locating lesion regions in all cases. Using the lesion center slice as a reference point, we expanded 20 slices vertically. This preprocessing standardized the final CT sequence length to 20 slices per patient. Our study innovates in feature extraction by integrating deep learning with radiomics techniques, enabling multi-dimensional feature extraction to allow the model to learn richer feature representations [17, 21]. Adopting a preprocessing strategy similar to the radiomics pipeline, we converted the 3D CT sequences $V \in \mathbb{R}^{D \times W \times H \times C}$ (where $D$ denotes image depth, $W$ width, $H$ length, and $C$ channels) into uniformly sized voxel blocks $P$ dimensions $d \times w \times h \times c$, which were input into the network for feature extraction. The voxel block segmentation formula is shown in Eq. (4):

$P=\left\{P_{i j k}\right\}_{i=1, j=1, k=1}^{d, w, h}$               (4)

The range of i is from 1 to d, j from 1 to w, and k from 1 to h.

To overcome the limited tissue, contrast inherent in CT imaging, we incorporate a Spatial-Shift MLP ($S^2-M L P$) module for spatial channel mixing. $S^2-M L P$ primarily adopts a spatial feature reorganization strategy, designing channels into four groups including the image space directions of up, down, left, and right. We expand this to six groups by adding two sagittal plane directions to the original four. Specifically, first, the extracted 3D feature tensor is arranged into four groups along the channel dimension, performing right and left shifts in the coronal plane, as well as upward and downward shifts in the sagittal plane, forming feature offsets across anatomical levels to achieve spatial dimensionality reorganization of feature vectors [23]. Subsequently, a Multi-Layer Perceptron (MLP) with quadruple expanded hidden layers is employed to construct high-order feature interactions, expanding into vectors and projecting to vectors e_i , as shown in formulas (5) and (6):

$p_{i j k}=\operatorname{Flatten}\left(P_{i j k}\right) \in \mathbb{R}^{c \cdot t p^2}$             (5)

$e_{i j k}=L N\left(W_0 p_{i j k}+b_0\right) \in \mathbb{R}^c$         (6)

In this context, $\mathbb{R}^{c \cdot t p^2}$ denotes the dimensionality of the deep learning block, where $t$ represents the block size and $p$ indicates the spatial direction block size. The learnable projection matrix $W_0$ and the bias vector $b_0$ are defined accordingly. The specific $L N$-layer normalization formula is given in Eq. (7):

$\widehat{e}_{\imath}=\gamma_i \frac{e_i-\mu}{\sigma}+\beta_i, \forall i \epsilon[1, c]$                 (7)

Here, $\mu=\frac{1}{c} \sum_{i=1}^c e_i, \sigma=\sqrt{\frac{1}{c} \sum_{i=1}^c\left(e_i-\mu\right)^2+\varepsilon}$, where $\varepsilon$ is a constant, and $\gamma$ and $\beta$ are learnable parameters.

The architecture of the high-dimensional feature extraction module is illustrated in Figure 3. We first apply conventional 3D convolution in the first two residual stages of the 3D ResNet network, while replacing some convolution blocks with $S^2-M L P$ blocks in the third residual stage. This spatial communication mechanism enhances diagnostic accuracy. The computational formula for the 3D convolution kernel is given in Eq. (8):

$O(x, y, z)=\sum_{i=0}^{k-1} \sum_{j=0}^{k-1} \sum_{m=0}^{k-1} I_{\text {padded }}(x+i, y+i, z+m) \cdot K(i, j, m)$            (8)

The variables $O(x, y, z)$ represent the output feature map, $I_{{padded }}(x+i, y+i, z+m)$ denote the padded input data, $K(i, j, m)$ denote the 3D convolution kernels, and $k$ represents the kernel size. The 3D residual block primarily consists of 3D convolution, BN, ReLU, and skip connections [22]. Finally, the feature map undergoes global pooling (GAP) to compress the 2048 $\times D \times H \times W$ feature map into a fixed-length 2048dimensional feature vector, which can be further utilized for classification.

Figure 3. Network diagram of the high-dimensional feature extraction module

In stark contrast to radiomics features, deep learning features undergo nonlinear mapping transformations throughout the entire process of connecting neural network input and output spaces. Given that radiomics extracts low-level features with a dimensionality of 1345, we implemented a full connection layer (FC) to reduce the dimensionality of deep learning features extracted by 3D ResNet to 1024 through linear transformations. Finally, we concatenated the radiomics features and deep learning features obtained for each patient, resulting in a combined feature dimensionality of 2369 per patient for subsequent feature screening [17].

2.4 LASSO feature selection strategy

Through a strategy combining high-level and low-level features, we extracted multidimensional features including 1346-dimensional radiomics features and 1024-dimensional deep learning features [17]. Given potential redundancy or interference among features, we implemented feature screening to retain those critical for lesion diagnosis. The process utilized LASSO regression, a linear regression method with L1 norm regularization that introduces penalty terms in the loss function to shrink regression coefficients to zero [11, 21]. Since LASSO demonstrates limited effectiveness in identifying highly repetitive features, we combined it with T-test statistical methods to compare differences between experimental and control groups, ensuring retention of statistically significant variations [21, 24].

Specifically, we first apply Z-score normalization to the extracted multi-dimensional features f as part of data preprocessing. This transforms features from different dimensions into a unified standardized scale, setting the mean to 0 and the standard deviation to 1. The formula for Z-score normalization is given in Eq. (9):

$f^{\prime}=\frac{f-\mu}{\sigma}$              (9)

Here, $f$ represents the original multi-dimensional feature, $\mu$ denotes its mean, $\sigma$ is the standard deviation, and $f^{\prime}$ stands for the standardized feature value. We then perform a t-test on the standardized feature $f^{\prime}$. The null hypothesis $H_0$ states that there is no significant difference in the mean of this feature between the experimental and control groups, i.e., $\mu_1=\mu_2$. The alternative hypothesis $H_1$ asserts that the means of the two groups differ significantly, i.e., $\mu_1 \neq \mu_2$. The t-test calculation formula is provided in Eq. (10):

$t=\frac{\overline{f_1^{\prime}}-\overline{f_2^{\prime}}}{s_p \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$             (10)

Here, $n_1$ and $n_2$ denote the sample sizes of the experimental and control groups respectively, while $f_1^{\prime}$ and $f_2^{\prime}$ represent their standardized features. The pooled standard deviation $S_p$ is calculated as shown in Eq. (11):

$S_p=\sqrt{\frac{\left(n_1-1\right) s_1^2+\left(n_2-1\right) s_2^2}{n_1+n_2-2}}$                (11)

where, $s_1$ and $s_2$ denote the standard deviations of the experimental and control groups, respectively. After obtaining the T-statistic, we directly compute the p-value, retaining features with p-values below the predefined significance level of 0.05. Through these screening methods, we ultimately identified 76 independent features of significant importance for classification, including 40 radiomics features and 36 deep learning features, which serve as input features for the subsequent classifier [17].

The comparative and ablation experiments were conducted exclusively on the dataset from Participating Center 1, whereas the external validation required data from two collaborating hospitals. Specifically, Participating Center 1 served as the internal validation dataset, and Participating Center 2 was designated as the external validation dataset.

4.1 Comparison of experimental results and analysis

To validate the effectiveness of the multi-dimensional feature extraction method proposed in this study, we conducted comparative experiments between the classification model developed in this chapter and multiple deep learning and machine learning classification algorithms. The deep learning algorithms included 3D ResNet34, 3D ResNet50, 3D DenseNet121, 3D VGG-11, 3D VGG-16, C3D, and 3D EfficientNet [25, 26], while the machine learning algorithms comprised SVM, MLP, LR, and RF [20]. All comparison algorithms were trained on the dataset constructed in this study, with hyperparameters such as training epochs and learning rate kept consistent to ensure the validity and fairness of the experiments.

In Chapter 2, we developed classification evaluation metrics to comprehensively assess the performance of multi-dimensional feature extraction methods against classical classification models. The evaluation covered five key metrics: AUC, accuracy, sensitivity, precision, and specificity [27]. Optimal values are highlighted in bold, with comparative results presented in Table 2.

As shown in Table 2, the multi-dimensional feature extraction network proposed in this study achieves optimal performance across nearly all metrics. First, in terms of AUC, 3D ResNet34 achieves the highest AUC of 92.0 among classical classification networks, while our network improves it by 4.9%. Regarding accuracy, 3D ResNet34 also reaches the highest accuracy of 88.8%, but our network enhances it by 2.6%. For sensitivity, Random Forest RF achieves the highest value of 96.0% among classical models, significantly outperforming our network. This is primarily due to Random Forest RF's overemphasis on positive sample features, resulting in strong recognition capability for positive samples but only 59.2% specificity for negative samples, leading to unstable performance [25]. In precision, 3D ResNet34 maintains stable classification performance with a peak of 88.4%, while our network improves it by 4.4%. Finally, in specificity, our network achieves a peak value of 98.4% across all experiments. Across all evaluated metrics, our network demonstrates superior classification performance, with stable recognition capabilities for both positive and negative samples, and effectively balances feature learning between them.

Table 2. Comparison of different network experiment results

In conclusion, the comparative experiment results show the effectiveness of our proposed method. Compared with other classification networks, our method can learn on more feature dimensions to enrich the diversity of features, including high-level and low-level features, and effectively make the model learn more discriminative features [21].

To visually assess the model's performance, we plotted the ROC curves of both our proposed multi-dimensional feature extraction network and the best-performing 3D ResNet34 in the classical network. The ROC curve, a receiver operating characteristic curve, is primarily used to calculate the area under the curve (AUC). As shown in Figure 4, our model demonstrates a higher AUC compared to 3D ResNet34, indicating that the multi-dimensional feature extraction network outperforms 3D ResNet34 in distinguishing positive and negative samples across different thresholds [27].

Furthermore, we generated confusion matrix diagrams for all comparison networks. These matrices visually demonstrate the model's prediction outcomes across different categories, including true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). As shown in Figure 5, our proposed multi-dimensional feature extraction network achieves higher recall rates for negative samples, enabling more comprehensive category identification while maintaining balanced recognition between positive and negative samples. Specifically, this network demonstrates superior accuracy in distinguishing positive and negative samples, with reduced error diagnosis probabilities and lower missed detection rates, thereby validating the effectiveness of our proposed methodology.

Figure 4. ROC curve of multi-dimensional feature extraction network and 3D ResNet34

Figure 5. Confusion matrix diagram of network comparison

4.2 Results and analysis of the ablation experiment

To evaluate the effectiveness of combining high-level and low-level feature extraction modules in multi-dimensional feature extraction networks, we conducted an ablation experiment on adrenal adenoma versus adrenal metastasis classification [3, 20]. Table 3 presents comparative results of three variants to explore the synergistic effects of radiomics feature extraction and deep learning feature extraction. Specifically: (1) The first row shows models using only radiomics features combined with machine learning classification algorithms. We compared four machine learning models (SVM, MLP, LR, and RF from Table 2) while employing radiomics features. The SVM-based approach demonstrated optimal performance, which we adopted as the experimental benchmark [6, 7]. (2) The second row utilizes 3D ResNet exclusively as the high-level feature extraction network without incorporating low-level features [1]. (3) The third row represents the complete multi-dimensional feature extraction network that integrates low-dimensional radiomics features with high-dimensional 3D ResNet features. Results clearly demonstrate the critical role of both low-level and high-level feature extraction modules. Specifically, using only radiomics for low-level feature extraction achieved an AUC of 88.7%, while employing 3D ResNet for high-level feature extraction alone yielded an AUC of 92.0%. However, directly combining high-dimensional and low-dimensional features resulted in reduced model performance due to excessive overall feature dimensions and information redundancy, achieving only an 89.7% AUC. By integrating both modules with feature selection, the model attained peak performance across all classification metrics [14]. Although the fused model's sensitivity marginally underperformed high-level feature extraction, its overall performance reached optimal levels. We hypothesize this stems from 3D ResNet's overemphasis on positive sample features, while its negative sample recognition specificity remained at 85.0%. This discrepancy explains why the high-level feature extraction module demonstrated slightly higher sensitivity compared to multi-level feature extraction.

Table 3. Results of the ablation experiment analysis

4.3 External validation results and analysis

To evaluate the generalization capability of the multidimensional feature extraction network proposed in this study and verify its independence from the distribution of single-center medical data, we conducted external validation at Collaborative Hospital 2 [11, 28]. Using Collaborative Hospital 1's dataset as the training set and Hospital 2's dataset as an independent test set, we performed external validation using the same network from the previous comparative experiment. The results were analyzed to compare the performance of different classification networks on the external validation set. The experimental results are presented in Table 4.

Table 4. Comparison of different network external validation experiments

Table 4 experimental results demonstrate that our proposed multi-dimensional feature extraction network achieved optimal performance across nearly all classification metrics. The network attained peak values of 69.5% AUC, 74.6% accuracy, and 77.6% specificity. While the network exhibited a reduction in sensitivity and precision, its overall performance across key classification metrics confirmed superior generalization capabilities compared to baseline models when applied to multi-center datasets. Notably, all classification networks exhibited mild to severe performance degradation during external validation, indicating overfitting to Participating Center 1 data [1, 21]. When encountering Participating Center 2 data with different distribution patterns, the model failed to acquire stable generalizable features, resulting in significant performance deterioration. Consequently, future research will prioritize enhancing the model's generalization capacity to mitigate performance degradation caused by cross-center data variations [22].