Few-Shot Speaker Recognition Using Self-Supervised Speech Representations and a Transformer Encoder

Speaker recognition [1-3] is an important research area in speech signal processing and biometric identification. It aims to determine or verify a person's identity by analyzing the acoustic characteristics of speech signals. Compared with other biometric identification methods, such as fingerprint recognition and facial recognition, speaker recognition can be performed without requiring physical contact or specialized image acquisition equipment. Speech signals can also be collected through commonly available microphones and communication devices, making voice-based identity recognition suitable for a range of practical applications. Typical examples include voiceprint authentication for personal smart devices, secure access control in telephone banking, speaker identification in forensic investigations, and caller identity verification in customer service systems. As voice-controlled devices and remote communication services become increasingly common, reliable speaker recognition has become an important component of intelligent human–computer interaction and information security systems.

From the perspective of identity recognition objectives, speaker recognition can be divided into two main categories: speaker identification [4, 5] and speaker verification [6, 7]. Speaker identification aims to determine the identity of an unknown speaker from a predefined set of enrolled speakers. The system analyzes an input speech utterance and assigns it to one of the registered speaker identities according to the extracted acoustic characteristics. In contrast, speaker verification determines whether a speech utterance belongs to a claimed speaker. Rather than selecting an identity from multiple candidates, the verification system evaluates the similarity between the input speech and the reference information associated with the claimed identity. Although these two tasks share several feature extraction and representation learning techniques, their decision procedures and evaluation criteria are different. Speaker identification is commonly evaluated using classification-based measures, whereas speaker verification generally relies on similarity scores and verification error measures.

Speaker recognition systems can also be categorized according to the types of information used for identification. Speech-based recognition methods rely exclusively on acoustic information [4, 7-9], while multimedia-based methods [10] combine speech signals with additional information, such as facial images or video sequences. Multimedia speaker recognition may use visual information to estimate the number of speakers appearing in a recording. Another approach constructs a joint audio–visual representation by combining speaker embeddings extracted from speech with facial embeddings obtained from images. For example, an integrated multimedia vector, referred to as an M-vector, can be constructed by concatenating face embeddings and speech-based x-vectors with an appropriate weighting factor. Such methods make use of complementary information from different modalities. Nevertheless, the availability and quality of visual information may vary considerably across applications. In telephone conversations, audio recordings, and many voice-controlled systems, only speech signals are available. Speech-based speaker recognition therefore remains particularly relevant for applications in which visual data cannot be obtained.

Despite considerable progress in speaker recognition, the performance of existing systems may deteriorate when speech recordings are affected by environmental noise, reverberation, transmission distortion, or differences in recording devices [11-13]. The acoustic characteristics extracted from a speaker's voice may change under different recording conditions, even when the speech is produced by the same person. Such variations can make it difficult to distinguish speaker-specific information from factors unrelated to identity. In addition, speaker recognition systems can be affected by spoofing attacks involving replayed, synthesized, or converted speech [14, 15]. Conventional systems trained mainly on genuine speech samples may not adequately distinguish authentic speech from deliberately manipulated signals. Although noise robustness and spoofing detection are separate research problems, they indicate the broader difficulty of developing speaker representations that remain reliable under varying input conditions.

The performance of a speech-based speaker recognition system depends substantially on the quality of the extracted speech features and the ability of the classification model to distinguish among different speakers. Traditional approaches commonly employ acoustic features such as Mel-frequency cepstral coefficients, filter-bank energies, and Mel-spectrograms. These features describe different aspects of the spectral characteristics of speech and have been widely adopted in speech processing applications. However, conventional acoustic representations are primarily designed to characterize signal properties rather than to isolate speaker identity. They may retain information associated with phonetic content, speaking style, and recording conditions, which can interfere with speaker classification. Consequently, the selection of input features and the construction of effective classification models have long been important research directions in speaker recognition.

With the development of deep learning, speaker recognition research has increasingly focused on learning acoustic representations directly from speech data. Neural network architectures can extract complex characteristics from speech signals and construct representations that are more suitable for downstream recognition tasks. Nevertheless, the training of these models often requires a substantial amount of labeled speech data covering multiple speakers and recording conditions. In practical applications, collecting and annotating large speech datasets can be expensive and time-consuming. The difficulty becomes more pronounced when only a small number of speech samples can be collected for each target speaker. Under such conditions, the classification model may learn characteristics associated with individual training utterances rather than stable speaker-specific patterns, resulting in poor generalization to previously unseen speech samples.

Speaker recognition under limited labeled-data conditions is commonly investigated within the framework of few-shot speaker recognition [16, 17]. In this setting, the recognition system is expected to identify speakers using only a small number of labeled reference utterances. Few-shot speaker recognition [4] is particularly relevant to applications in which it is impractical to collect extensive enrollment recordings from each user. For instance, a newly registered user of a voice-controlled device may provide only a few short speech samples during enrollment. Similarly, forensic recordings and historical audio materials may contain limited speech segments from a speaker of interest. A recognition system operating in these circumstances must extract sufficient speaker-discriminative information from the available samples while reducing its dependence on large labeled datasets.

Few-shot speaker recognition presents several difficulties that are less pronounced in conventional speaker recognition tasks. First, a limited number of reference utterances may not adequately represent the variability of a speaker's voice across different phonetic contexts and speaking conditions. Second, differences in recording environments or speech duration can produce substantial variation between the reference samples and the speech to be classified. Third, when the training samples are insufficient, a classification model with a large number of parameters may overfit the available data. Finally, the acoustic differences between certain speakers can be relatively small, making it difficult to construct well-separated speaker representations. These issues suggest that few-shot recognition requires not only an appropriate classification strategy but also input features that preserve meaningful speaker information when labeled training data are scarce.

One direction for addressing the dependence on labeled speech data involves learning general-purpose speech representations from large quantities of unlabeled recordings. Self-supervised learning (SSL) allows a neural network to acquire useful information from speech signals without requiring explicit speaker identity annotations for every training sample. Instead of relying entirely on manually labeled datasets, self-supervised models learn representations through training objectives derived from the structure of the input data. Such representations may contain information about phonetic characteristics, temporal relationships, speaker identity, and other acoustic properties. The resulting features can subsequently be transferred to downstream speech processing tasks using a relatively limited amount of task-specific labeled data.

Self-supervised speech representations (S3Rs) have already been investigated for conventional speaker recognition tasks [18, 19]. These representations offer an alternative to conventional acoustic features by using information learned during pretraining on unlabeled speech signals. Their potential applicability to speaker identification is particularly relevant when the amount of labeled training data is limited. Nevertheless, the suitability of a pretrained representation for a particular recognition task depends on the information retained during pretraining and the ability of the downstream classifier to distinguish among speaker identities.

Recent studies have also explored different neural network architectures and representation learning strategies for few-shot speaker identification. For example, a channel-attention convolutional recurrent neural network using three-dimensional log-Mel spectrogram features was developed to address feature extraction and overfitting problems in few-shot speaker identification [20]. The method combines convolutional and recurrent processing with an attention mechanism to learn speaker-related information from limited speech samples. A subsequent study investigated a low-complexity speaker embedding module based on feature segmentation, transformation, and reconstruction [21]. This approach considered the computational requirements of few-shot speaker identification and demonstrated the importance of designing speaker representations that are suitable for resource-constrained applications. These investigations illustrate the continuing interest in improving few-shot speaker recognition through the design of specialized feature extraction and classification architectures.

Related research has examined different strategies for learning speaker embeddings without relying on extensive identity annotations. Tao et al. [22] investigated self-supervised training of speaker encoders using multimodal diverse positive pairs, demonstrating an alternative approach to constructing speaker representations without explicit speaker labels. Ge et al. [23] proposed an isomorphic graph attention network for pooling S3Rs, focusing on the aggregation of speaker-related information. Han et al. [24] investigated a cluster-aware self-distillation framework for self-supervised speaker verification, incorporating strategies to address unreliable pseudo-labels. These studies suggest that self-supervised representation learning offers a practical direction for reducing the dependence of speaker recognition systems on manually annotated speech data.

Other investigations have concentrated on the quality of pseudo-labels and the reliability of speaker embeddings learned without direct supervision. Fathan and Alam [25] analyzed the influence of clustering methods on self-supervised speaker verification and reported that the quality of generated pseudo-labels can substantially affect downstream recognition performance. Gao et al. [26] further investigated pseudo-label correction and clustering-based learning for speaker verification. Liu et al. [27] examined the selection of reliable pseudo-labels through confidence distribution modeling in semi-supervised and self-supervised speaker recognition. A recent study and review of SSL for speaker recognition [28] also discussed the characteristics and limitations of different representation learning frameworks. Collectively, these investigations indicate that the quality of learned speech representations and their adaptation to a particular recognition task remain important research considerations.

Among the available self-supervised speech models, Mockingjay and WavLM provide two possible sources of pretrained speech representations. Mockingjay adopts a masked acoustic modeling strategy to learn contextual information from speech sequences, while WavLM combines masked speech prediction with denoising-oriented pretraining to capture information relevant to different speech processing tasks [18, 19]. Although their training strategies differ, both models can generate contextual speech representations that may contain information useful for distinguishing speakers. Unlike conventional Mel-spectrogram features, these representations are learned from speech data through pretraining objectives rather than being determined solely by predefined signal transformations. However, the information contained in a pretrained representation is not necessarily optimized for few-shot speaker identification. Its suitability depends on whether the downstream recognition model can extract and use speaker-discriminative characteristics from a small set of labeled utterances.

Yi et al. [29] further investigated the application of S3Rs to speaker segmentation using a WavLM-based approach. This investigation provides another example of the use of pretrained speech representations in speaker-related processing tasks, although speaker segmentation and speaker identification involve different recognition objectives.

In addition to the input representation, the architecture of the classifier plays an important role in speaker recognition. Speech signals contain temporal variations arising from changes in phonetic content, articulation, and speaking style. A classification model must therefore process sequences of acoustic features and identify information that is relevant to speaker identity. Transformer encoders provide a mechanism for modeling relationships among different positions in an input sequence through self-attention. Unlike architectures that process temporal information primarily through local operations or sequential recurrent updates, self-attention allows the model to establish relationships between feature representations at different temporal positions. This property makes Transformer encoders suitable candidates for processing contextual speech representations. Nevertheless, their actual performance in few-shot speaker identification must be established experimentally, particularly because a limited number of labeled samples may restrict the effective training of the classifier.

The combination of S3Rs and Transformer-based classification offers a potential approach to the limited-data problem. Pretrained representations can provide acoustic information learned from speech data before the downstream recognition stage, while a Transformer encoder can process relationships within the resulting feature sequences. In principle, these two components serve complementary purposes: the pretrained model provides the input representation, and the classifier learns to distinguish the speaker identities represented in the labeled training samples. The recognition performance of this combination, however, cannot be inferred directly from the success of self-supervised models in conventional speaker recognition. Few-shot identification involves different constraints on the number of labeled samples and may require the classifier to operate with substantially less speaker-specific information.

Although self-supervised representations have been studied in speaker recognition, the comparative behavior of different pretrained speech representations when coupled with a Transformer encoder for few-shot speaker identification still warrants further investigation. In particular, it is useful to determine whether such representations provide more informative input features than conventional Mel-spectrograms when the number of labeled speaker samples is limited. It is also necessary to examine whether the choice of pretrained representation affects the recognition performance of the same classification architecture. Addressing these questions can provide a clearer understanding of the relationship between representation learning and Transformer-based classification in few-shot speaker identification.

Motivated by these considerations, this study proposes a few-shot speaker recognition framework that combines S3Rs with a Transformer encoder. Two pretrained representations, Mockingjay and WavLM, are considered separately as input features, and a Transformer encoder is employed as the principal component of the speaker classification model. The study evaluates the recognition performance obtained using these representations and compares it with the performance achieved using conventional Mel-spectrogram features. The objective is to investigate the applicability of pretrained speech representations to few-shot speaker identification and to examine their interaction with a Transformer-based classifier. Since the task addressed in this paper is speaker identification, the term speaker recognition in the following sections refers specifically to speaker identification unless otherwise stated.

The main contributions of this study are summarized as follows:

(1) A few-shot speaker identification framework is developed by combining pretrained S3Rs with a Transformer encoder. The framework investigates the use of speech features learned through self-supervised pretraining for classification under limited labeled-data conditions.

(2) Two S3Rs, Mockingjay and WavLM, are examined within the same Transformer-based classification framework. Their respective recognition results are analyzed to investigate the influence of the input representation on few-shot speaker identification performance.

(3) The proposed framework is evaluated against a conventional Mel-spectrogram-based approach. The experimental comparison provides evidence regarding the suitability of S3Rs for few-shot speaker recognition and identifies differences in recognition performance between the selected representations.

The remainder of this paper is organized as follows. Section 2 reviews related studies on speaker recognition, few-shot learning, and self-supervised speech representation learning. Section 3 describes the proposed framework, including the extraction of S3Rs and the Transformer-based classification model. Section 4 presents the experimental setup, recognition results, and corresponding analysis. Finally, Section 5 summarizes the findings and discusses possible directions for future research.

This section describes the proposed few-shot speaker recognition framework, which combines S3Rs with a Transformer encoder-based (TE-based) classifier. The framework is designed to investigate whether pretrained speech representations can provide effective input features for speaker identification when only a limited number of labeled speech samples are available. Rather than relying exclusively on conventional acoustic features and a classifier trained from labeled speech data, the proposed method uses representations obtained from pretrained self-supervised speech models. These representations are subsequently processed by a Transformer-based classifier to distinguish among the target speakers.

2.png

Figure 2. Framework of the proposed speaker recognition method based on self-supervised speech representations and a Transformer encoder

As illustrated in Figure 2, the proposed framework consists of two principal parts: an S3R extractor and a TE-based classifier. The S3R extractor converts an input speech utterance into a sequence of feature vectors, while the TE-based classifier processes these vectors and produces classification probabilities corresponding to the enrolled speakers. The two parts are connected sequentially, with the output of the representation extractor serving as the input to the classification network.

The main functions of the two parts are described below.

(1) S3R extractor: This part extracts contextual speech representations from the input utterance using a pretrained self-supervised speech model. Two alternative extractors are considered in this study, namely Mockingjay and WavLM. Each extractor produces a sequence of acoustic representations that can be supplied to the subsequent classification network.

(2) TE-based classifier: This part receives the extracted speech representations and learns to distinguish the identities of the speakers included in the classification task. It consists of a feature projection layer, a Transformer encoder, a temporal averaging operation, a second fully connected layer, and a softmax operation. These components perform feature transformation, contextual feature learning, sequence-level aggregation, and speaker classification.

The proposed method separates the extraction of pretrained speech representations from the learning of speaker-specific classification information. This arrangement makes it possible to examine different S3R models within a common downstream classification framework. In particular, Mockingjay and WavLM representations are evaluated separately, allowing their recognition performance to be compared using the same general classification architecture.

The overall processing procedure begins with an input speech utterance. The S3R extractor first converts the utterance into a sequence of feature vectors. These vectors are then projected into the feature space required by the TE-based classifier through the first fully connected layer. The Transformer encoder processes the projected sequence and produces contextualized representations. Subsequently, temporal mean pooling aggregates the sequence-level information into a fixed-length utterance-level feature vector. This vector is passed through the second fully connected layer to produce speaker-specific classification scores, which are converted into probabilities by the softmax operation.

The two principal parts of the framework perform different but connected functions. The S3R extractor provides pretrained speech representations, while the TE-based classifier learns to associate the extracted features with the target speaker identities. The detailed implementation and processing procedures of these two parts are described in Sections 3.1 and 3.2, respectively.

3.1 S3R extractor part

The S3R extractor is responsible for converting an input speech utterance into a sequence of learned acoustic representations. In contrast to conventional feature extraction methods, which obtain acoustic characteristics through predefined signal processing operations, self-supervised representation learning uses a pretrained neural network to transform speech signals into contextual feature vectors. These representations are learned from speech data through self-supervised pretraining objectives and can subsequently be used as inputs to different downstream recognition models.

The motivation for using an S3R extractor in the proposed framework is closely related to the limited availability of labeled speech samples in few-shot speaker recognition. When only a small number of utterances are available for each speaker, training a feature extraction network entirely from the available labeled data may be difficult. A pretrained self-supervised model offers an alternative source of acoustic representations learned before the speaker classification stage. The downstream classifier can then operate on these representations rather than relying exclusively on feature patterns learned from a limited speaker-labeled dataset.

In this study, the S3R extractor can be implemented using either the Mockingjay model or the WavLM model. The two models are considered separately because they differ in their input processing procedures and pretraining strategies. The choice of extractor determines the form of the speech representation supplied to the TE-based classifier, while the subsequent classification procedure follows the same general framework.

For the Mockingjay-based implementation, the input speech is first represented using Mel-spectrogram features. These features are processed by the pretrained Mockingjay encoder to obtain contextualized speech representations. The encoder uses information from different positions within the input sequence to construct a feature vector for each time step. Unlike the masked input sequences employed during self-supervised pretraining, speech representations for downstream recognition are extracted without applying the pretraining masking procedure.

The resulting Mockingjay representations contain information derived from the spectral characteristics of the input utterance and the contextual relationships learned by the pretrained encoder. Although the original Mel-spectrogram describes acoustic information at individual time steps, the extracted representation also incorporates information from the surrounding speech sequence. This contextual processing provides an alternative representation of the utterance for the downstream speaker identification task.

For the WavLM-based implementation, the speech waveform is processed by the pretrained WavLM model. WavLM uses a convolutional feature encoder to generate latent acoustic representations, followed by Transformer-based contextual processing. The representations obtained from the pretrained model are then used as input to the TE-based classifier. Since WavLM is pretrained using masked prediction and denoising-oriented learning objectives, its extracted features may contain information related to different acoustic characteristics and temporal relationships within the speech signal.

The use of WavLM differs from that of Mockingjay primarily in the representation extraction stage. Mockingjay operates on Mel-spectrogram inputs, whereas WavLM processes speech waveforms through its own feature encoding architecture. Nevertheless, the output of either extractor can be regarded as a sequence of feature vectors associated with the input utterance. This common representation format allows both models to be incorporated into the proposed speaker classification framework.

For an input utterance, the S3R extractor produces a feature sequence containing representations at successive temporal positions. The number of extracted feature vectors depends on the input utterance and the temporal resolution of the selected representation model. Similarly, the dimensionality of each vector is determined by the output configuration of the pretrained extractor. Therefore, Mockingjay and WavLM do not necessarily produce feature sequences with identical temporal lengths or feature dimensions.

The TE-based classifier is designed to process these representations through an initial feature transformation stage. Consequently, the representations extracted from different self-supervised models can be supplied to the same general downstream architecture, provided that their input dimensions are appropriately accommodated by the first fully connected layer.

An important distinction should be made between representation extraction and speaker classification. The S3R extractor provides acoustic feature vectors, but these vectors do not directly represent the final predicted speaker identities. Speaker recognition requires an additional decision process that maps the extracted features to the speaker classes defined in the recognition task. This function is performed by the TE-based classifier described in Section 3.2.

The two S3R extractors are evaluated independently in the proposed framework. This arrangement allows the study to investigate how representations obtained from different self-supervised pretraining procedures affect few-shot speaker identification. The extracted features are passed to the subsequent classification stages, where they undergo dimensionality transformation, contextual processing, temporal aggregation, and speaker probability estimation.

3.2 TE-based classifier part

The TE-based classifier is the second major component of the proposed few-shot speaker recognition framework. Its primary function is to transform the representations generated by the S3R extractor into classification results corresponding to the target speakers. Since the extractor produces a sequence of feature vectors rather than a single speaker identity label, the classifier must process the temporal information contained in the sequence and obtain a fixed-length representation suitable for classification.

As illustrated in Figure 2, the TE-based classifier consists of five modules: the first fully connected layer (FC1), the Transformer encoder, the mean operation, the second fully connected layer (FC2), and the softmax operation. The modules are arranged sequentially, and each performs a different function in the classification process.

The FC1 module performs the initial feature transformation. It receives the sequence of representations generated by the S3R extractor and converts their feature dimensions to the dimension used by the Transformer encoder. In the proposed architecture, FC1 reduces the dimensionality of the extracted S3R features before they are passed to the subsequent module.

This transformation serves two purposes. First, it allows feature vectors generated by different S3R extractors to be processed using a common encoder input dimension. Second, reducing the feature dimension can decrease the amount of computation required by the subsequent Transformer encoder. Since the input representations may contain a relatively large number of feature dimensions, the projection operation provides a more compact representation for contextual feature processing.

FC1 is applied to the feature vectors associated with the temporal positions of the input utterance. Therefore, the operation transforms the feature dimension while retaining the sequential organization of the extracted representations. The output of FC1 remains a sequence of feature vectors, with its temporal structure preserved for processing by the Transformer encoder.

The Transformer encoder is the principal feature learning module of the TE-based classifier. It receives the projected S3R feature sequence and processes the relationships among representations at different temporal positions. Through the self-attention mechanism, each position can incorporate information from other positions within the input sequence. The resulting representations therefore contain contextual information derived from the sequence rather than information associated solely with individual feature vectors.

The attention mechanism is particularly relevant to speech processing because the acoustic characteristics of an utterance vary over time. Different segments may contain different phonetic content, and the extent to which a segment provides information useful for distinguishing speakers may also vary. By modeling relationships among temporal positions, the Transformer encoder provides a mechanism for learning contextual patterns from the extracted speech representations.

The multi-head self-attention mechanism allows several sets of relationships to be modeled through different learned projections. The attention outputs are processed together and transformed through the feed-forward sublayers of the encoder. Residual connections and normalization operations support information propagation through the network. As a result, the output of the Transformer encoder consists of a sequence of feature vectors that has undergone contextual feature transformation.

In the proposed framework, the input and output of the Transformer encoder maintain the same sequence length and hidden feature dimension. This property applies to the encoder itself, after the input representations have been transformed by FC1. Although the encoder changes the numerical values and contextual information contained in the feature vectors, it does not change their overall sequence shape.

It should be noted that the Transformer encoder does not directly produce the final speaker classification results. Its role is to transform the projected speech representations into contextualized features that can subsequently be aggregated and classified. The sequence-level output must therefore be converted into a fixed-length representation before it can be supplied to the final classification layer.

The mean module performs this conversion through temporal averaging. It computes the mean value of the Transformer output feature vectors along the temporal dimension. For each feature dimension, the values corresponding to the different time steps are averaged to obtain a single value. The resulting representation contains one aggregated value for each hidden feature dimension.

This operation converts a sequence of contextualized frame-level representations into an utterance-level feature vector. Unlike the input to the mean module, which contains multiple temporal positions, the output is a single fixed-length vector representing the processed utterance. Consequently, the classifier can obtain an utterance-level representation even when the input sequences contain different numbers of temporal positions.

Temporal averaging also provides a relatively simple aggregation strategy. It does not require an additional trainable attention-pooling network or a separate sequence modeling module. Instead, information from the entire Transformer output sequence contributes to the final aggregated representation. However, the mean operation assigns equal averaging weights to the temporal positions, and its effectiveness depends on the quality of the features produced by the preceding Transformer encoder.

The distinction between temporal feature processing and utterance-level aggregation is important in the proposed architecture. The Transformer encoder operates on the sequence of feature vectors and learns contextual relationships across positions, whereas the mean module reduces the temporal dimension and generates a fixed-length representation. These operations serve different purposes and together establish the connection between sequential speech feature learning and speaker classification.

The aggregated feature vector is subsequently passed to FC2. This module maps the utterance-level representation to a vector whose number of output nodes equals the number of speaker classes in the recognition task. Each output node corresponds to one enrolled speaker, and its value represents the classification score assigned to that speaker before probability normalization.

The output dimension of FC2 is therefore determined by the number of target speakers. For a task involving a fixed set of speaker identities, the corresponding classification layer contains one output node for each identity. When the number of speaker classes changes, the required output dimension of FC2 must also be adjusted accordingly. This configuration is consistent with the speaker identification objective adopted in the present study, where the input utterance is assigned to one of the predefined speaker classes.

Although FC1 and FC2 are both fully connected layers, they operate on representations with different structural characteristics. FC1 processes the feature vectors at individual temporal positions and transforms their feature dimensions before contextual learning. FC2, in contrast, processes the fixed-length utterance-level representation obtained after temporal averaging and maps it to the speaker classification scores.

The softmax module is applied after FC2 to convert the classification scores into a probability distribution over the target speaker classes. Each output probability is nonnegative, and the probabilities associated with all speaker classes sum to one. The resulting values indicate the relative classification probabilities assigned by the model to the different speakers.

During inference, the predicted speaker identity is determined by the output node with the highest probability. Thus, an input utterance is assigned to the speaker class associated with the maximum softmax output. The classifier operates within the predefined speaker set and produces a speaker identification result based on the information learned during training.

The TE-based classifier is trained using cross-entropy loss. During the training stage, an input utterance is processed by the S3R extractor and the subsequent classification modules to obtain the predicted speaker probabilities. These probabilities are compared with the corresponding ground-truth speaker label through the cross-entropy objective. The loss measures the discrepancy between the predicted probability distribution and the target speaker identity.

The training process adjusts the parameters of the TE-based classifier to reduce the classification loss. Through this process, the classifier learns how to transform the extracted S3R features into representations that distinguish among the target speakers. The FC1 layer learns the initial feature projection, the Transformer encoder learns contextual transformations, and FC2 learns the mapping between the aggregated utterance-level representation and the speaker classes.

Once training is completed, the resulting classifier can be used to identify speakers from input utterances according to the learned classification parameters. Each test utterance is processed through the same sequence of modules used during training. The S3R extractor first generates the pretrained speech representations, FC1 transforms their feature dimension, the Transformer encoder processes the contextual relationships, and the mean operation produces a fixed-length utterance-level representation. Finally, FC2 generates the speaker classification scores, and softmax converts these scores into probabilities.

The proposed framework is intended for few-shot speaker identification, in which the classifier is trained using a limited number of labeled speech samples for each speaker. The pretrained S3R extractor provides acoustic representations learned before the downstream classification task, while the TE-based classifier learns to distinguish the available speaker identities using those representations. This division between representation extraction and classification is particularly relevant when the labeled dataset is relatively small.

Nevertheless, the effectiveness of the framework depends on the information contained in the extracted representations and the ability of the classifier to learn appropriate decision boundaries from the available samples. The use of a pretrained representation does not eliminate the difficulties associated with limited labeled data, and the Transformer encoder may still be affected by the size and distribution of the training dataset. Therefore, the proposed method is evaluated experimentally using different input representations rather than assuming that the combination will necessarily produce superior recognition results.

In summary, the TE-based classifier transforms sequential S3Rs into utterance-level speaker classification probabilities through five successive processing modules. FC1 performs feature projection, the Transformer encoder learns contextual relationships, the mean operation aggregates temporal information, FC2 maps the aggregated representation to the speaker classes, and softmax produces the final probability distribution. Together with the S3R extractor, these modules form the complete few-shot speaker recognition framework illustrated in Figure 2. The recognition performance of the proposed method is examined in Section 4.

This study proposed a few-shot speaker recognition framework that combines S3Rs with a TE-based classifier. Two pretrained speech representations, Mockingjay and WavLM, were separately employed to extract acoustic features from input utterances. The resulting representations were processed by a classification network consisting of two fully connected layers, a Transformer encoder, a temporal mean pooling module, and a softmax layer. The first fully connected layer projected the extracted representations into a lower-dimensional feature space, while the Transformer encoder learned contextual relationships within the feature sequences. Temporal mean pooling subsequently aggregated the learned features into fixed-length utterance-level representations for speaker classification.

The experimental results demonstrated the feasibility of applying pretrained S3Rs to few-shot speaker identification. The proposed framework achieved recognition accuracy values of 94.80% and 92.29% when using Mockingjay and WavLM representations, respectively. Under the evaluated experimental conditions, Mockingjay produced higher recognition accuracy than WavLM. Furthermore, the comparison with conventional Mel-spectrogram features indicated that S3Rs provided more effective input features for the proposed classification framework. These findings suggest that pretrained speech representations can support speaker discrimination when only limited labeled training samples are available.

Nevertheless, the reported results are specific to the experimental settings considered in this study. Further investigation is required to assess the generalization of the framework across different speech datasets, recording conditions, and numbers of labeled samples per speaker. Future work may also examine the computational requirements of the classifier and the performance of additional pretrained speech representation models under few-shot recognition conditions.