A Deep Learning-Based Grouped Teaching Strategy for Experimental Training

Experimental teaching underpins skill training courses. The teaching for experimental training of skill training courses is an essential link of improving students’ skills, abilities, and key competencies [1-6]. It directly relies on group teaching of the students. Grouped teaching for experimental training facilitates students to understand the concepts and teaching rules of skill training courses [7-14]. The diagnosis of students’ pre-class hands-on operation capacity helps to group the students scientifically to facilitate the teaching for experimental training of skill training courses, and benefits the capacity complementation and team spirit cultivation [15-22]. The main difficulty of the diagnosis is how to accurately judge the key competencies of different types of students in the experimental training of skill training courses, and to divide these students into reasonable groups [23, 24].

The traditional experimental teaching of management emphasizes teaching over learning, and stresses knowledge over practice. It no longer meets the development needs of innovative society in the context of new media. In the new media environment, the experimental courses of management are badly in need of collaborative innovation of the teaching system. Drawing on the new media environment, Xia et al. [25] carried out collaborative innovative design of the teaching system for the experimental courses of management, from the aspects of overall design, teaching method, and teaching evaluation. Through practice and application, the experience and shortcomings were summarized, and the improvement measures were presented. Wen et al. [26] put forward a lightweight collaborative experimental teaching method, including computer simulation, teaching guidance, and class reflection. Thirty-three grade 8 students were recruited to explore how the teaching method affects the exploratory scores and scientific knowledge of students. However, further research is needed to verify whether a series of lightweight experiments can improve student performance. Girouard and Kang [27] created an innovative usability training called collaborative learning of usability experiences program for students majoring in human-computer interaction (HCI) of different grades, and detailed the components of the unique experiential training, namely, internship, short-term courses, symposium, and knowledge transfer. In addition, the employment and academic performance of graduates were evaluated, and the five key skills of interns were checked. Kerpen et al. [28] proposed a problem-based lab-centered college course. The course teaching has a high demand for virtual collaboration. The students used various tools to draw a preliminary design plan, which could be realized in future physical prototypes.

The existing literature shows that grouped learning and pre-class hands-on operation capacity are necessary and important to the teaching for experimental training of skill training courses. Practically speaking, the relevant results have rarely been implemented in the practical teaching of skill training courses. No literature has analyzed the correlation between grouped teaching for experimental training, and the key competences of students for skill training courses. To make up for the gap, this paper constructs a grouped teaching strategy for experimental training based on deep learning. The main contents are as follows: Section 2 evaluates the key competences of students for skill training courses from four aspects, namely, general training objective, thinking training, practical training, and scientific literacy training. The evaluation results can be used to diagnose the students’ pre-class hands-on operation capacity, and reasonably group the students participating in experimental training. Section 3 calls the deep learning model to group students of different majors for experimental training courses, combined meta learning with transfer learning, and introduced the attention mechanism to prevent over fitting, when there are a few samples for capacity diagnosis. The proposed model was proved effective through experiments.

Based on deep learning model, this paper groups students in different majors for experimental training courses. The students in different majors need to complete different skill training courses, and acquire different key competencies. As a result, there are a few samples for the diagnosis of students’ pre-class hands-on operation capacity.

The small sample set can be improved by data enhancement, transfer learning, or meta learning. This paper combines meta learning with transfer learning, and introduces the attention mechanism to prevent over fitting, when there are a few samples for capacity diagnosis.

As shown in Figure 1, the overall framework of our model consists of a dense residual module, an attention module in global channel and local space, and an attention module between spatial channels. Firstly, the proposed model is trained by the set of original capacity diagnosis samples. The weights of the trained model are finetuned to adapt to the grouping of students for experimental training courses, in the presence of a small sample set.

1.png

Figure 1. Learning structure of our model

To overcome vanishing or exploding gradients, this paper introduces a skip connection to the ResNet in the dense residual module of our model. Then, the nonlinear operation can be skipped, once the optimal solution is obtained. Let F_k(a_k-1) be the residual obtained through nonlinear operation; a_k-1 be the optimal solution before executing the nonlinear operation. Then, we have:

${{a}_{k}}={{F}_{k}}\left( {{a}_{k-1}} \right)+{{a}_{k-1}}$     (1)

The DenseNet in the dense residual module adopts a new connection mode to promote the gradient transmission between layers, i.e., the output of each layer is directly transmitted to all subsequent layers. That is, the input of the k-th layer contains the feature maps from the 0-th layer to the k-1-th layer. Let [a₀,a₁,...,a_k-1] be the splicing of the eigenvectors from the 0-th layer to the k-1-th layer. Then, we have:

${{a}_{k}}={{g}_{k}}\left( \left[ {{a}_{0}},{{a}_{1}},\cdots ,{{a}_{k-1}} \right] \right)$    (2)

To enhance the feature transfer of capacity diagnosis samples in network training, this paper embeds the dense connection mode of DenseNet to ResNet. Let a_k-10, and a_k-11 be the medians of each dense residual block. Taking the residual block structure of ResNet as the basis, the dense residual blocks can be calculated with skip connection as network mapping:

$\begin{align}  & {{a}_{k}}=G\left( {{a}_{k-1}} \right)+{{a}_{k-1}} \\ & G\left( {{a}_{k-1}} \right)=G\left( \left[ {{a}_{k-1}},{{a}_{k-10}},{{a}_{k-11}} \right] \right) \\\end{align}$      (3)

The purpose of the attention module in global channel and local space, and the attention module between spatial channels is to improve the quality of feature representation, i.e., highlight the features of important capacity diagnosis samples, and weaken the features of unimportant samples. Figure 2 illustrates the structure of an attention module.

Let $A \in R^{F^{*} Q^{*} Z}$ be the eigenvector outputted by the convolutional layer after the input capacity diagnosis sample passes through that layer; [a₁,a₂,...,a_Z] be channels a₁-a_Z; $A_{i} \in R^{F^{*} Q^{*} 1}$ be the feature information in the i-th channel; U=[a₁,a₂,...,a_Z] be the filter group for learning; u_i be the i-th filter; F_k be the function of the deep convolutional layer; u_i be the two-dimensional (2D) spatial convolutional filter. The spatial statistics in a channel can be generated by F_k, and used to generate the output eigenvector B_k:B_k=[b_k,1,b_k,2,...,b_k,z]:

${{b}_{k,i}}={{F}_{k}}\left( {{a}_{i}},{{u}_{i}} \right)={{a}_{i}}*{{u}_{i}}$       (4)

Let F_h be the modeling of the correlation between global channel features; ξ be the rectified linear unit (ReLU). Suppose parameters T₁ and T₂ satisfy $T_{1} \in R^{1^{*} 1^{*} Z^{*} Z / 4}$, and $T_{2} \in R^{1^{*} 1^{*} Z / 4^{*} Z}$, respectively. Then, the output eigenvector $B_{h} \in R^{F^{*} Q^{*} Z}$ can be obtained by:

${{B}_{h}}={{F}_{h}}\left( A,Q \right)={{T}_{1}}*\xi \left( {{T}_{2}}*A \right)$     (5)

The compression mechanism and the relationship between global channel indices can be obtained through the dimensionality reduction of channels with T₁. Let F_hk be element-by-element multiplication. By sigmoid function, B_k and B_h derive the activation value for re-calibration or activating the feature information:

${{B}_{hk}}={{F}_{hk}}\left( A,\varepsilon \left( {{B}_{k}} \right),\varepsilon \left( {{B}_{h}} \right) \right)=A\bullet \varepsilon \left( {{B}_{k}} \right)\bullet \varepsilon \left( {{B}_{h}} \right)$      (6)

Let $b \in R^{Z}$ be the feature of each capacity diagnosis sample passing through the global average pooling. The channel attention mechanism can be given by:

$\theta =\varepsilon \left( Q*b \right)$      (7)

Let Q be the Z*Z parameter matrix; Q_l be the band matrix. The band matrix provides the basis for the learning of channel information attention. The matrix can be expressed as:

$\left[ \begin{matrix}   {{q}^{1,1}} & \cdots  & {{q}^{1,l}} & 0 & 0 & \cdots  & \cdots  & 0  \\   0 & {{q}^{1,2}} & \cdots  & {{q}^{2,l+1}} & 0 & \cdots  & \cdots  & 0  \\   \vdots  & \vdots  & \vdots  & \vdots  & \ddots  & \vdots  & \vdots  & \vdots   \\   0 & \cdots  & 0 & 0 & \cdots  & {{q}^{Z,Z-l+1}} & \cdots  & {{q}^{Z,Z}}  \\\end{matrix} \right]$      (8)

Formula (25) shows that Q_l contains l*Z parameters. It is only necessary to consider the interaction between b_i and its l adjacent channels. Let Ψ^l_i be the set of b_i and its l adjacent channels. Then, the weight of b_i can be calculated by:

${{\theta }_{i}}=\varepsilon \left( \sum\limits_{j=1}^{l}{q_{i}^{j}b_{i}^{j}} \right),b_{i}^{j}\in \Psi _{i}^{l}$     (9)

To improve the effectiveness of model learning, the learning parameters of all channels should be unified. Thus, formula (26) can be updated as:

${{\theta }_{i}}=\varepsilon \left( \sum\limits_{j=1}^{l}{{{q}_{i}}b_{i}^{j}} \right),b_{i}^{j}\in \Psi _{i}^{l}$      (10)

To keep the learning parameters of all channels consistent, a fast one-dimensional (1D) convolution CON_FO-l can be implemented with a kernel size of l:

$\theta =\varepsilon \left( CO{{N}_{FO-l}}\left( b \right) \right)$     (11)

The above formula is the way to compute the attention between channels, and to effectively capture the local information exchange between channels. The computation is of a low complexity, involving only l parameters.

To realize the attention mechanism between channels, it is necessary to determine the kernel size l, and obtain a relatively suitable coverage of adjacent channels. The possible mappings Φ between l and Z can be nonlinearized:

$Z=\Phi \left( l \right)={{2}^{\alpha *l-y}}$     (12)

Let |x|_odd be the odd number the closest to x. If the channel dimension Z is given, then l can be adaptively calculated by:

$l=\Omega \left( Z \right)={{\left| \frac{lo{{g}_{2}}\left( Z \right)}{\alpha }+\frac{y}{\alpha } \right|}_{odd}}$     (13)

To prevent our model from over fitting, meta learning was integrated with transfer learning, which contains lightweight operations like scaling and translation, to initialize neural network parameters on deep layers. The integration enables meta learning to converge quickly facing a few capacity diagnosis samples, thereby reducing the computing load. Figure 3 shows the flow of the proposed meta-transfer learning model.

During model learning, the first step is to train the capacity diagnosis sample set on multiple task levels. Let TT be the training task; δ₁, δ₂ and δ_{1,2} be the scaling operation, translation operation, and the combination between the two operations, respectively. Facing a small set of diagnosis samples, the student grouping classifier can be optimized by:

$\omega '=\omega -\gamma {{\nabla }_{\omega }}{{K}_{TT}}\left( \left[ \Pi ,\omega  \right],{{\delta }_{\left\{ 1,2 \right\}}} \right)$      (14)

δ₁ and δ₂ can be iteratively updated by:

${{\delta }_{i}}={{\delta }_{i}}-\alpha {{\nabla }_{{{\delta }_{i}}}}{{K}_{TT}}\left( \left[ \Pi ,\omega  \right],{{\delta }_{\left\{ 1,2 \right\}}} \right)\left( i=1,2 \right)$      (15)

Let Π be the trained feature extractor of capacity diagnosis samples. Suppose the network layer of Π contains n pairs of weights and thresholds {(Q_i,n,y_i,n)}. Let $\oplus$ be the point multiplication operation. Then, the combined operation of scaling and translation can be expressed as:

$RR\left( A,Q,y;{{\delta }_{\left\{ 1,2 \right\}}} \right)=\left( Q\oplus {{\delta }_{1}} \right)A+\left( y+{{\delta }_{2}} \right)$      (16)

2.png

Figure 2. Structure of an attention module

3.png

Figure 3. Flow of meta-transfer learning model