You Only Look Once version 8-Based Audit Document Detection and Classification with Interpretable Design for Auditing Education

2.1 Improved YOLOv8 rotation-sensitive detection network

Geometric distortion phenomena such as curved table lines, folded stretching regions, and tilted stamps are common in audit documents. The fixed receptive field design of the original YOLOv8 backbone network is difficult to adapt to such scenarios [16, 17], resulting in prominent problems of missed detection of rotated targets and bounding box drift, which seriously affect detection accuracy. To address this problem, this paper proposes an improved YOLOv8 rotation-sensitive detection network. The core innovations focus on a DCNv4-enhanced backbone network and a GWD loss-guided rotated bounding box regression head. Through the dual improvement of geometric adaptation capability and rotation localization accuracy, precise detection in complex audit document scenarios is achieved. The network framework is shown in Figure 1.

2.1.1 Backbone network optimization based on deformable convolutional network version 4

The original YOLOv8 uses Cross Stage Partial Darknet (CSPDarknet) as the backbone network, and the final stage adopts standard 3 × 3 convolution for feature extraction [18]. The fixed receptive field cannot dynamically adjust according to local geometric distortions of audit documents, which easily leads to insufficient feature extraction for targets such as curved table lines, folded edges, and tilted stamps, thereby causing missed detection. To solve this problem, in the last two stages of CSPDarknet, this paper replaces all standard 3 × 3 convolutions with DCNv4. By dynamically modulating spatial aggregation weights, adaptive adjustment of the receptive field is achieved, constructing a backbone network adapted to geometric distortions of audit documents.

The core innovation of DCNv4 lies in the dynamic modulation mechanism of spatial aggregation weights. Through the collaborative effect of sampling point offsets and local content adaptive weight adjustment, the convolution kernel can flexibly adjust the receptive field range according to the local feature distribution of audit document images, breaking the limitation of fixed receptive fields. Compared with the early deformable convolution DCN, DCNv4 abandons the fixed sampling weight mode and further improves feature fitting ability in distortion scenarios. Specifically, DCNv4 first predicts the offset of each sampling point, allowing the sampling points to accurately cover distortion regions such as curved table lines and folded edges, avoiding feature omission caused by fixed sampling positions. At the same time, based on the grayscale distribution and texture features of the local region, aggregation weights of each sampling point are adaptively assigned, strengthening feature responses in distortion regions and suppressing background interference. The core mathematical expression of weight modulation is as follows:

$W(x, y)=\sum_{k=1}^K \omega \cdot \operatorname{softmax}_k\left(\frac{F\left(x+\Delta x_k, y+\Delta y_k\right)}{\tau}\right)$                       (1)

where, W(x,y) is the aggregation weight of the current pixel (x,y), K is the number of sampling points, ω_k is the initial weight of the sampling point, Δx_k and Δy_k are the predicted offsets of sampling points, F( ) is the local feature response value, and τ is the temperature coefficient used to adjust the concentration of weight distribution. Compared with the original DCN, DCNv4 introduces a local content adaptive weight adjustment mechanism, avoiding the problem of weakened distortion region features caused by fixed sampling weights. It can more accurately capture features of distorted targets such as curvature and tilt in audit documents, effectively reducing the missed detection rate caused by fixed receptive fields and improving the adaptability of the backbone network to geometric distortions of audit documents.

Figure 1. Improved YOLOv8 rotation-sensitive detection network architecture

2.1.2 Rotated bounding box regression head and gaussian Wasserstein distance loss design

The original YOLOv8 adopts a horizontal bounding box regression head, which can only localize horizontally distributed targets and cannot accurately adapt to non-horizontal targets such as rotated attachment marks and oblique signature fields in audit documents, resulting in bounding box drift and missed detection. At the same time, the traditional IoU loss has problems of angle periodicity and boundary discontinuity when handling rotated bounding boxes, and the penalty for small angle deviations is insufficient, making the localization accuracy of rotated boxes difficult to meet the detection requirements of audit documents [19, 20]. To address the above problems, this paper extends the horizontal bounding box regression head to a rotated bounding box regression head and designs a loss function based on GWD to achieve precise localization of rotated targets.

The rotated bounding box regression head adopts a five-parameter representation. The rotated box is uniquely determined by the center coordinates (x_c,y_c), width w, height h, and rotation angle θ, which can accurately adapt to morphological characteristics of various non-horizontal targets in audit documents and solve the problem that horizontal boxes cannot cover rotated targets. To solve the problems of angle periodicity and boundary discontinuity, the GWD loss models rotated bounding boxes using Gaussian distributions. The position, size, and angle information of rotated boxes are transformed into parameters of a two-dimensional Gaussian distribution. By calculating the Wasserstein distance between the Gaussian distributions corresponding to the predicted box and the ground-truth box, accurate computation of rotated box regression loss is achieved, strengthening the penalty for small angle deviations. The core computation logic is as follows. First, the rotated bounding boxes are modeled as two-dimensional Gaussian distributions N(μ_p,Σ_p) and N(μ_g,Σ_g), corresponding to the predicted box and the ground-truth box, respectively, where μ is the mean of the Gaussian distribution corresponding to the center coordinates of the rotated box, and Σ is the covariance matrix determined by the width, height, and rotation angle of the rotated box. The expression is:

$\Sigma=\left(\begin{array}{cc}\frac{w^2}{12} & 0 \\ 0 & \frac{h^2}{12}\end{array}\right) \cdot R(\theta)^T$                    (2)

where, R(θ) is the rotation matrix. Then, the GWD loss is obtained by calculating the Wasserstein distance between the two Gaussian distributions, and the specific expression is:

$L_{G W D}=\left\|\mu_p-\mu_g\right\|_2^2+\operatorname{tr}\left(\Sigma_p+\Sigma_g-2\left(\Sigma_p^{1 / 2} \Sigma_g \Sigma_p^{1 / 2}\right)^{1 / 2}\right)$             (3)

Compared with the traditional IoU loss, the GWD loss effectively solves the problems of angle periodicity and boundary discontinuity through Gaussian distribution modeling, and has a stronger penalty mechanism for small angle deviations, which can significantly improve localization accuracy of rotated boxes. In audit document scenarios, this loss function can accurately capture angle deviations of non-horizontal targets such as rotated attachment marks and oblique signature fields, effectively reducing bounding box drift. Combined with the design of the rotated bounding box regression head, precise detection of rotated targets in audit documents is achieved.

2.2 Attention-guided multi-scale feature fusion mechanism

There are a large number of small targets and category imbalance problems in audit documents. Traditional feature fusion mechanisms cannot effectively enhance small-object features and suppress background interference, resulting in limited detection accuracy. This paper designs an attention-guided multi-scale feature fusion mechanism. Through an improved attention module and dynamic fusion strategy, precise selection and efficient fusion of features are achieved, breaking the technical bottleneck of small-object detection and category imbalance. The core innovation lies in the targeted improvement of the attention module and dynamic regulation of fusion weights. The schematic diagram is shown in Figure 2.

Figure 2. Schematic diagram of Attention-Guided Multi-Scale Feature Fusion Mechanism (AFPN)

2.2.1 Attention feature pyramid network structure design

Traditional feature pyramid networks lack attention guidance and cannot effectively distinguish target features and background interference according to the feature characteristics of audit documents, resulting in excessive redundant information in fused features and weakened small-object features. This paper designs an attention-guided feature pyramid network. In the top-down and bottom-up feature propagation paths, an improved channel–spatial dual attention module is embedded to achieve precise calibration and enhancement of features. At the same time, a dynamic weighted fusion strategy is designed to improve multi-scale feature fusion performance.

The improved attention module is optimized based on Convolutional Block Attention Module (CBAM). The core innovation lies in combining the image characteristics of audit documents to realize targeted design of channel and spatial attention. At the channel attention level, the single global average pooling method is abandoned. Considering the grayscale discrimination between the red channel of stamps and the grayscale background channel in audit documents, a semantic weight calibration logic is designed. By calculating category response values of different channels, the weights of channels where key targets such as stamps and tampering traces are located are enhanced, and interference from redundant background channels is suppressed. The channel attention weight calculation expression is:

$W_c=\operatorname{sigmoid}\left(M L P\binom{\operatorname{MaxPool}(F)+\operatorname{MeanPool}(F)}{+\operatorname{GrayDiff}(F)}\right)$                         (4)

2.3 Teaching-oriented interpretability and lightweight design

The core requirements of university auditing teaching for detection models focus on interpretability and real-time interaction. The former requires students to intuitively understand the image feature basis of model decision-making, and the latter requires the model to adapt to common teaching equipment. For the special requirements of teaching scenarios, this paper designs a teaching-oriented interpretability and lightweight integrated scheme. The core innovation lies in an improved Grad-CAM++ heatmap generation method and a targeted structured channel pruning strategy. On the premise of ensuring detection accuracy, dual objectives of model decision visualization and lightweight deployment in teaching scenarios are achieved. The process diagram is shown in Figure 3.

Figure 3. Schematic diagram of improved Grad-CAM++ heatmap generation process

2.3.1 Improved Grad-CAM++ heatmap generation

The original Grad-CAM has problems of gradient diffusion and insufficient localization accuracy in audit document scenarios with dense small targets. It is difficult to accurately present the model decision basis for small targets and cannot meet teaching interpretability requirements. This paper performs targeted optimization on Grad-CAM++. The core innovation is the introduction of a pixel-level weighted averaging method of gradients under multiple confidence thresholds, which strengthens gradient responses in small target regions and improves heatmap localization accuracy. At the same time, the complete generation process is organized to ensure that visualization results meet teaching requirements.

The core technical point of the improved Grad-CAM++ is the pixel-level weighted averaging mechanism of gradients. By setting multiple confidence thresholds, gradients in different confidence intervals are differentially weighted to avoid small target gradients being submerged by a single threshold. Let the model output confidence thresholds be divided into three intervals [0.5,0.7), [0.7,0.9), and [0.9,1.0], with corresponding weight coefficients ω₁ = 0.3, ω₂ = 0.5, and ω₃ = 0.2. The pixel-level gradient weighted averaging formula is:

$\nabla_F=\sum_{k=1}^3 \omega_k \cdot \nabla_{F, k} \cdot I\left(c_k \in T_k\right)$                        (5)

where, ∇_F is the final weighted gradient, ∇_F,_k is the feature map gradient corresponding to the k-th confidence interval, c_k is the model output confidence, T_k is the corresponding confidence interval, and I( ) is the indicator function. Compared with the original Grad-CAM, this improvement effectively strengthens gradient responses in small target regions through multi-threshold gradient weighting, solving the problem of blurred heatmap localization in dense small target scenarios. The complete process of heatmap generation is as follows: first extract the last-layer convolution feature map of the backbone network, then perform gradient backpropagation on model prediction results, calculate gradient weights of each channel, perform pixel-level calibration of gradients using the above weighting formula, conduct weighted fusion of calibrated gradients and feature maps, and finally upsample to the original image size through bilinear interpolation to obtain a heatmap with the same size as the input document image.

The visualization adaptation design closely matches auditing teaching requirements. The heatmap is overlaid with the original audit document image, and the transparency of the heatmap is adjusted to ensure that both original document content and highlighted regions are clearly visible. The confidence threshold for red highlighted regions is set to 0.7. This threshold is verified through multiple teaching experiments, which can effectively highlight core feature regions of model decisions while avoiding excessive redundant information interfering with student observation. In the overlaid visualization results, red highlighted regions precisely correspond to audit targets detected by the model, intuitively presenting the image basis of model decisions and helping students understand how the model performs object detection through image features, fully meeting the interpretability requirements of auditing teaching.

2.3.2 Model lightweighting and real-time optimization

The original improved YOLOv8 model has a large number of parameters, and the inference speed cannot meet the real-time interaction requirements of common teaching computers, making it unsuitable for teaching devices without independent GPUs. This paper adopts structured channel pruning technology and designs a lightweight scheme for teaching scenario requirements. The core innovation lies in a sparse training process based on batch normalization layer scaling factors and a low L1-norm convolution kernel removal criterion. Under strict control of accuracy loss, optimization of model parameters and inference speed is achieved, ensuring deployment adaptation to teaching scenarios.

The core technical process of structured channel pruning is divided into two stages: sparse training and channel removal. In the sparse training stage, L1 regularization constraints are applied to the scaling factors of batch normalization layers during model training, guiding the model to automatically learn redundant channels and achieve sparsification of channel weights. Let the scaling factor of the batch normalization layer be γ, and the loss function of sparse training is:

$L_{\text {total }}=L_{\text {detect }}+\lambda \cdot \sum_{i=1}^N\left|\gamma_i\right|_I$                                 (6)

where, L_detect is the model detection loss, λ is the sparsity regularization coefficient, set to 0.001 to balance detection accuracy and sparsification effect, and N is the total number of batch normalization scaling factors. Through this loss function, the scaling factors of redundant channels gradually approach zero. In the channel removal stage, a removal criterion for low L1-norm convolution kernels is set: when the L1 norm of the batch normalization scaling factor is smaller than threshold (θ) (θ = 0.01), the corresponding channel and associated convolution kernel are removed, and the channel number of subsequent network layers is adjusted to ensure integrity of the network structure.

The comparison of model parameters before and after pruning clearly presents the lightweighting effect. The model parameter size before pruning is 28.6M, and after pruning it is reduced to 10.2M, with a parameter reduction of 64.3%. In terms of inference speed, the inference speed on a common teaching computer before pruning is 6.5 FPS, and after pruning it increases to 15.9 Frames Per Second (FPS), achieving a 2.3× speed improvement. The balance strategy between accuracy and speed is achieved through fine-tuning after pruning. After channel removal is completed, the model is fine-tuned for 10 epochs with a learning rate of 0.0001, ensuring that the decrease of mean Average Precision (mAP)@0.5:0.95 does not exceed 1.5%. The final pruned model only decreases by 1.2% in mAP, maintaining excellent detection accuracy.

The teaching adaptation design fully considers hardware conditions of common teaching computers. The pruned lightweight model does not require independent GPU support and can perform efficient inference based on CPU, with inference speed stable above 15 FPS, fully meeting real-time interaction requirements of auditing teaching. In teaching scenarios, students can upload audit document images through common teaching computers, and the model can quickly output detection results and heatmap visualization without obvious lag. At the same time, deployment of the lightweight model does not require complex hardware configuration and environment setup, reducing deployment cost of teaching platforms and facilitating wide application in university auditing teaching, achieving deep integration of intelligent detection technology and auditing teaching.

2.4 Audit document-specific data augmentation and training strategy

Audit document data have problems such as single scenario, high annotation cost, and complex real-scene interference. General data augmentation and training strategies are difficult to adapt to their image characteristics, which easily leads to insufficient model generalization ability and slow training convergence. For the characteristics of audit documents, this paper constructs a dedicated dataset annotation specification and designs a dedicated data augmentation pipeline and a two-stage training strategy. The core innovation lies in scenario-oriented augmentation operations, synchronization between annotation and augmentation, and personalized design of training parameters, ensuring that the model can accurately learn audit document target features while improving training efficiency and robustness, and adapting to deployment requirements in teaching scenarios.

2.4.1 Dataset construction and annotation specification

The quality of the dataset directly determines model training performance. Considering the scarcity and sensitivity of audit document data, this paper constructs a dedicated dataset containing 12,000 audit document images, covering multiple types such as vouchers, invoices, and audit reports. Samples with different scanning quality and different distortion degrees are included to ensure diversity and representativeness of the dataset. Data are collected from university auditing training samples and enterprise desensitized audit documents. After collection, a hierarchical desensitization processing workflow is adopted. Sensitive information is replaced through pixel-level blurring, removing privacy content such as organization names, amounts, and signatures in documents while retaining core features of audit targets, ensuring data compliance and usability and avoiding leakage of sensitive information.

To ensure accuracy and consistency of annotation, strict annotation specifications are formulated for nine categories of core targets in audit documents. Rotated rectangular boxes are used for target annotation. Annotation parameters include center coordinates, box length, width, and rotation angle, fully adapting to morphological characteristics of rotated targets in audit documents. The annotation tool uses Label Studio, and a three-level annotation process of “double annotation + cross validation + expert review” is adopted. First, two annotators complete annotations independently. Then, consistency validation is performed by calculating the intersection-over-union of annotation boxes. Samples with IoU lower than 0.8 are jointly checked and corrected by the two annotators. Finally, auditing domain experts conduct final review of annotation results, ensuring that annotation deviation does not exceed 2 pixels and rotation angle annotation precision retains one decimal place. At the same time, an annotation document is established to clarify annotation standards and boundary definitions for each category, avoiding annotation ambiguity and providing high-quality annotated data support for model training.

2.4.2 Dedicated data augmentation pipeline

Traditional data augmentation methods lack specificity for audit document scenarios and cannot simulate special situations such as scanning interference and geometric distortions of real audit documents, making it difficult to effectively improve model robustness. This paper designs an audit document-specific data augmentation pipeline, including four targeted augmentation operations. Real scenario interference is simulated through precise parameter settings. At the same time, a synchronized transformation method for augmentation operations and rotated box annotations is designed to avoid annotation misalignment and ensure effectiveness of augmented data.

Elastic deformation is used to simulate geometric distortions such as folding and stretching during audit document scanning. The core is the design of a random displacement field. The displacement field follows a two-dimensional Gaussian distribution, and elastic deformation is achieved by dynamically adjusting local pixel displacement. The displacement field calculation expression is as follows:

$\Delta(x, y)=\sigma \cdot N(0,1)$                     （7）

2.5 Integrated application process for university auditing teaching

To achieve deep integration of the improved YOLOv8 detection method and university auditing teaching, an audit document intelligent analysis teaching platform is constructed. A three-layer architecture design is adopted and full-process integration is completed, considering both technical feasibility and teaching practicality. The image preprocessing layer serves as the platform foundation. A skew correction algorithm based on Hough transform detects document edge lines and calculates skew angles, achieving precise correction of arbitrary angle distortions. Gamma correction is used to complete brightness normalization, and brightness differences of documents are adaptively balanced by adjusting the gamma coefficient, eliminating the influence of uneven scanning illumination. Based on the bilinear interpolation algorithm, document images are uniformly resized to 1024 × 768 resolution to ensure consistency and stability of subsequent detection inference. The detection and interpretation module serves as the core functional layer. Real-time inference optimization is performed based on the lightweight pruned model, and inference delay is reduced through model quantization. Combined with the improved Grad-CAM++ algorithm described above, real-time heatmap generation is achieved, and detection results and feature-highlight visualization are synchronously output.

Figure 4. Three-layer architecture and business interaction flow of the university auditing teaching platform