New Enhancement Techniques for Optimizing Multimedia Visual Representations in Music Pedagogy

In the prevailing information era, the rapid advancement of technology and digital methodologies has seen multimedia technologies increasingly permeating various educational domains [1-8]. Notably, within the sphere of music pedagogy, visual expressions afforded by multimedia have been identified as a novel and highly compelling pedagogical tool. Such tools have facilitated students' understanding of music's essence and structure through vivid and intuitive imagery and audio [9, 10]. Nevertheless, the challenge faced by educators and technologists alike lies in the amalgamation of cutting-edge techniques with traditional music pedagogy to craft more efficient and profound teaching methodologies [11-13].

The application of multimedia image enhancement in music teaching is not sought for visual spectacle; instead, it aims for a more accurate conveyance of musical emotions, styles, and structures [14-19]. Through optimization of image luminosity, colour, and contrast, a deeper immersion into the world of music can be enabled, thereby augmenting students' perception and understanding [20, 21]. Consequently, the research and exploration into these techniques hold not only a profound significance in elevating the efficacy of music pedagogy but also actively propel the innovation and development in music education.

Despite the array of image enhancement methods currently available, many tend to concentrate solely on specific image attributes, often overlooking the holistic perception and authenticity of images [22-24]. For instance, some conventional enhancement techniques might excessively emphasize image contrast, leading to potential loss of image details or colour distortion [25, 26]. Moreover, in the face of the complex and ever-changing multimedia scenarios in music pedagogy, traditional methods often fall short, failing to meet the escalating pedagogical demands [27, 28].

Addressing these issues, innovative image enhancement techniques are introduced in this study. Firstly, an exploration into the adaptive Gamma correction method for luminance adjustment and a saturation correction method grounded on luminance components has been undertaken, aiming to elevate image luminance balance and colour authenticity. Secondly, a deep dive into the multimedia image enhancement technique rooted in an improved CLAHE algorithm was conducted, aiming to accentuate image details and augment its contrast. The deployment of these techniques not only substantially enhances the visual effects of multimedia in music teaching but also offers a more authentic and vivid learning environment, further advancing the modernization process of music education.

Multimedia visual representations in music pedagogy play a pivotal role in contemporary education. Through an array of forms such as images, videos, and animations, visualization technologies can present musical knowledge and skills in a more intuitive and vivid manner, significantly enhancing students' learning interest and efficacy. To ensure the optimal presentation of such multimedia content, image enhancement techniques have become paramount.

The quality and detail of images in multimedia visual representations for music education are held to particularly high standards. When it comes to intricate musical scores, detailed displays of instruments, or lively demonstrations of musical scenarios, contrast and detail reproduction of images are crucial. Against this backdrop, the introduction of the multimedia image enhancement method based on the improved CLAHE algorithm holds pronounced significance and advantages.

3.png

Figure 3. Bilinear interpolation

Figure 3 illustrates the principle of bilinear interpolation in the traditional CLAHE algorithm. The conventional CLAHE algorithm enhances contrast by dividing the image into multiple small rectangular areas and equalizing the histogram within each small area. However, such a method might lead to excessive contrast in certain regions, resulting in an over-enhanced effect on the image. To address this issue, a minimum and maximum threshold level is proposed to be set on the grayscale level of each small rectangular area, defining a dynamic range for the grayscale level. Consequently, pixels exceeding this range can be clipped and redistributed, ensuring the enhancement effect appears more natural and uniform. Figure 4 presents the flowchart of the image enhancement algorithm proposed in this study.

4.png

Figure 4. Flowchart of the image enhancement algorithm

Initially, probability density functions of each small rectangular area of the image are calculated. Based on this probability density function, the mean and variance within the area are ascertained. These two statistical measures, reflecting the overall luminance and contrast of the image, provide crucial groundwork for subsequent processing. Assuming the expected value of the probability density function o(z) is represented by R(z) and defined as ω=R(z); and the variance of the probability density function o(z) is denoted by F(z) with the standard deviation represented by δ, the calculation formulas are:

$R(z)=\sum_{z=0}^{M-1} z o(z)$               (8)

$F(z)=R\left(z^2\right)-[R(z)]^2$               (9)

$\omega=R(z), \delta=\sqrt{F(z)}$               (10)

Utilizing the mean and variance obtained from the previous step, the minimum threshold of each small rectangular area is computed. This threshold helps define a dynamic range for the grayscale level, ensuring that the enhancement is neither excessive nor insufficient. Supposing the minimum threshold is Y_MIN and the maximum threshold is Y_MAX, the formulas are:

$Y_{M I N}=\sum_{z=0}^{\omega+1.65 \delta} o(z)$               (11)

$Y_{M A X}=\sum_{z=0}^{\omega+1.96 \delta} o(z)$               (12)

Based on the data from the preceding two steps, the grayscale levels for each small rectangular area are determined. This grayscale level is defined within a dynamic range; pixels beyond this range will be clipped, ensuring a more uniform contrast enhancement of the image. Assuming the dynamic range of the grayscale level is denoted by f, with its minimum and maximum values represented by A_MIN and A_MAX respectively, the calculation formulas are:

$A_{M I N}=\left\{j\left|\sum_{z=0}^j o(z)=Y_{M I N}\right|\right\}$               (13)

$A_{M A X}=\left\{j\left|\sum_{z=0}^j o(z)=Y_{M A X}\right|\right\}$               (14)

$f=A_{M A X}-A_{M N N}$               (15)

Pixels exceeding the dynamic range defined in the previous step undergo a clipping process. These clipped pixels are redistributed within the dynamic range, allowing for a natural enhancement of contrast and ensuring the retention of finer image details. Presuming the preset clipping threshold is CL, pixel value is G(z), and the grayscale level is M, the calculation formulas are:

$T O=\sum_{z=0}^{M-1}(M A X(G(z)-C L I T), 0)$               (16)

$G_p(z)=\left\{\begin{array}{l}G(z),\left(z>A_{M A X}, z<A_{M I N}\right) \\ G(z)+R_e,\left(A_{M N} \leq z \leq A_{M A X}\right)\end{array}\right.$               (17)

High clarity and intricate texture details in images are essential for multimedia visual representations in music education. This is particularly true since music teaching often involves elements like musical scores, instrument details, and demonstrations of playing techniques, where even minute details can profoundly influence students' learning experience and outcomes. For instance, a minute notation mark on a musical score might dictate the method of performance, and subtle structural changes in an instrument or its method of being played could influence the tonality. Hence, ensuring these details are vividly represented in multimedia materials is paramount. The focus of this study revolves around optimizing multimedia visual representations through image processing techniques. Given that traditional image enhancement methods might fail to retain or highlight some minute details fully, this work introduces a multi-scale detail enhancement algorithm to bolster the texture and detail information of the image.

Three distinct scales (small, medium, and large) are first chosen based on specific requirements of music education. The original image undergoes filtering operations using Gaussian filters, producing three Gaussian blurred images representing smoother versions of the image at different scales. Assuming the images for fine detail, intermediate detail, and coarse detail are represented by F₁, F₂, and F₃ respectively, the formulas are:

$S_1=H_1 * U, S_2=H_2 * U, S_3=H_3 * U$               (18)

Differential operations involve simple subtraction of the corresponding Gaussian blurred image from the original, highlighting the discrepancies between the original and its smoothed version, thereby extracting details at that scale.

$F_1=U-S_1, F_2=S_1-S_2, F_3=S_2-S_3$               (19)

To obtain an image that retains macro-structural information while showcasing micro-texture and details, the detail images from the three scales need to be fused. Fusion can be achieved through a weighted average method, where weights can be adjusted based on specific needs in music education to ensure the emphasis on details at a particular scale. Assuming the fusion weights for different detail images are represented by μ₁, μ₂, and μ₃ and SGN as the sign function, then the formula is:

$F^*=\left(1-\mu_1 \times S G N\left(F_1\right)\right) \times F_1+\mu_2 \times F_2+\mu_3 \times F_3$               (20)

Lastly, the fused detail enhancement image is merged with the original image to produce the final multi-scale detail-enhanced image.