A Novel Context-Aware Deep Learning Algorithm for Enhanced Movie Recommendation Systems

With the burgeoning of the Internet and advancements in information technology, an exponential surge in data across network platforms has been observed [1]. Such proliferation not only furnishes users with a wealth of intriguing information but paradoxically engenders a quandary of information overload. Consequently, sifting through this deluge to pinpoint pertinent information has become an arduous task [2]. This conundrum is mirrored in the film industry, where recommendation systems have been posited as an efficacious remedy. At the heart of these systems lie recommendation algorithms, meticulously developed to tailor to users' specificities [3]. Optimal algorithms are pivotal as they not only streamline users' quest for intriguing films, thereby catalysing their consumption vigour, but also cascade into monumental economic windfalls for movie purveyors.

Historically, binary relations between users and projects have predominantly been leveraged for modelling in traditional recommendation systems [4]. Despite the commendable efficacy of some of these conventional algorithms, a glaring oversight is the neglect of context, a pivotal facet influencing user decisions. Such contexts are indispensable for enhancing recommendation accuracy, as they frequently shape user predilections [4]. For instance, a solitary movie viewer might be well-served by existing systems, but the same might falter when the viewer is accompanied by children, with an inclination towards animation genres. Similarly, prevailing emotional states, such as despondency, might skew choices towards comedic films, while romantic companions might prompt a tilt towards romance-themed films. Evidently, the immediate milieu in which a user is ensconced invariably impacts their final verdict [5].

It becomes unequivocally apparent that the infusion of contextual features into recommendation algorithms is paramount to bolster both the precision and relevancy of recommendations. In this vein, the CAWTF algorithm is introduced, designed meticulously to discern correlations between context and ratings, subsequently determining the weights attributed to various contextual features. Through such a procedure, the conundrums of data sparsity endemic to context-aware systems are assuaged. Given the prowess of deep learning in discerning intricate feature relationships, a fusion of CAWTF with neural networks culminated in the birth of the CAW-NeuMF model.

This manuscript is structured to first delineate the theoretical foundation underpinning context-aware recommendation systems. The ensuing section elucidates the high-order tensor decomposition algorithm predicated on context weight and presents the context-aware recommendation system delineated herein. The penultimate section is dedicated to empirical validation of the introduced algorithm and model, attesting to their efficacy and practicality.

In the exploration of context-aware recommendation systems, it has been observed that the focus on contextual features was not exhaustive [18]. All contextual features were considered, yet the treatment of these features lacked thoroughness. Additionally, the introduction of context presented significant sparsity challenges [19]. Notably, the introduction of contextual information, based on users and projects, expands the system from a two-dimensional matrix to a multidimensional one, as illustrated in Figure 1. This section initially deliberates upon the context-based weight allocation algorithm, subsequently introducing the CAW-NeuMF model, which amalgamates user, project, and contextual feature information to establish a deep learning recommendation system.

Traditionally, the dimensionality was reduced mainly through tensor decomposition or matrix decomposition. In this discourse, a novel method for dimensionality reduction is presented: a high-order tensor decomposition algorithm centred on contextual features to determine weight allocation. The algorithm's emphasis, first and foremost, is the calculation of context feature weights and the subsequent weight allocation algorithm.

image003.png

Figure 1. Multidimensional context space

The process is bifurcated into two distinct phases. Initially, the weight of each context is determined, followed by the alignment of the context features most closely associated with user relevance. It has been hypothesised that within identical contextual settings, varying users' impacts on project ratings differ significantly. For instance, when an individual watches a film accompanied by a significant other, preferences may lean towards romantic or family genres. In such scenarios, correlations between users and the context of social relationships become paramount, while ties to other contextual features diminish.

By calculating the standard deviation of each user's context rating, the most pertinent context in relation to the instance user is discerned. If the standard deviation of the user in one context exceeds that in another, it is inferred that the former context exhibits higher correlation with the user, subsequently neglecting the least correlating contextual feature. Historical scoring data can be exemplified as follows in Table 1.

Table 1. Data instance 1

In the pursuit of discerning which context holds a robust correlation with the rating, the standard deviation of each user was calculated. The elucidation of the specific algorithmic steps is presented as follows:

Step 1: It is posited that the dataset t comprises t comprises $t \in\{1,2 \ldots, m\}$ contextual features, symbolised as C_t. Dependent on varying criteria, the dataset is partitioned into distinct D $\in$ {D₁, D₂…, Dx} segments.

Step 2: From each instance of context t, data is arbitrarily selected from the partitions delineated in the first step. This data is subsequently divided into $\left|C_{t} \right|$ replicas. Within every instance of the context, the pertinent mean value $r_{z_k}$ is computed.

Step 3: Following the acquisition of the mean values for each instance as delineated in the second step, the overall average score for all instances of the context C_t is computed in accordance with Eq. (1).

$\overline{r_{C_t}}=\frac{1}{\left|C_t\right|} \sum r_{z_k}$                (1)

Step 4: In alignment with Eq. (2), the standard deviation pertaining to the context C_t score for every specific item, denoted as i, is computed.

$M_{U_{i t}}=\frac{\left[\sum_{c_k \in c_t}\left(r_{z_k}-\overline{r_{C_t}}\right) /\left(\left|C_t\right|-1\right)\right]^2}{\overline{r_{C_t}}}$                 (2)

To exemplify the calculation of standard deviations within the user-context interaction, consider the ratings of user U₁ for the context feature time. Initially, the data was divided into distinct sections based on varying items. The average ratings were ascertained for context time=Afternoon, r₁=(4+3)⁄2=3.5, time=Morning, r₂=2, and time=Night, r₃=3. Following this, the composite average rating for each instance of Time, $\overline{r_{t i m e}}=(3.5+2+3) / 3=2.83$ was computed. Utilising Eq. (3), the standard deviation pertaining to user U₁'s ratings in the context Time was identified as (2.83-3)²+(2-2.83)²+(3-2.83)²/(3-1)]²⁄2.83=0.120. In a parallel manner, the standard deviation associated with user U₁ in the context Weather was calculated, yielding a value of 0.107. This result indicated a pronounced correlation between the ratings of user U₁ and the context Time. Drawing from these insights, the top K contextual features demonstrating the highest relevance to the score were identified. Subsequently, weights corresponding to each context were determined using Eq. (3). It was observed that an elevated weight signified heightened importance of the respective context.

$W_{c t}=\frac{M_{U_{i t}}}{\sum_{t=1}^{t=k} M_{U_{i t}}} t \in[1, k]$              (3)

3.1 High-order tensor decomposition algorithm based on context weight

Previously, the method for context weight calculation was delineated, aligning the most pertinent context features based on this weight. However, post-introduction of context in recommendation systems, it has been noted that many users exhibit no connection with most contextual instances, leading to pronounced data sparsity. Conventional matrix decomposition approaches become unsuitable. High-order tensor decomposition has been efficaciously utilised for high-dimensional data processing and analysis and finds application in the domain of personalized recommendation systems [20]. Wang et al. [20] utilised higher-order singular value decomposition to address data sparsity in context-aware systems, while Karatzoglou and others incorporated tensor decomposition into context-aware systems, treating all context features collectively. This approach did not account for varied impacts of distinct contexts on recommendation systems. Hence, with an escalating number of context features, tensor decomposition not only becomes computationally intricate but also encounters data sparsity challenges.

Grounded on the context selection method delineated earlier, the most rating-relevant top K contextual features are ascertained. Weights for each context, as per Eq. (3), are then procured. In the ensuing discourse, the second segment of the algorithm will be addressed. By integrating context weights, the third-order tensor corresponding to each context is decomposed, yielding user, item, and context feature matrices.

For elucidative purposes, it is assumed that user historical rating data encapsulating contextual information is presented in Table 2.

Table 2. Data instance 2

To construct the third-order tensor of context C_i from the dataset, each element a_u,i,c in the tensor is considered to represent the user u rating of an item i under a given context C_i=c. For instance, a_1,2,1 is indicative of a rating of 5 given by user 1 for the item 2 under the context c=1. For an N-order tensor $A \in R^{I_1 * I_2 * \ldots * I_n * I_N}$, it is expanded into a matrix prediction, $A_{(n)} \in R^{I_n \times\left(I_{n+1} I_{n+2} \ldots I_N I_1 I_2 \cdots \times I_{n-1}\right)}$. In the matrix's specified line i_n, the value residing in column (i_n+1-1)I_n+2I_n+3…I_NI₁I₂…I_n-1+(i_n+2-1)I_n+3I_n+4…I_NI₁I₂I_n-1+…+(i_N-1)I₁I₂…I_n-1+(i₁-1)I₂I₃…I_n-1+(i₂-1)I₃I₄…I_n-1+…+i_n-1 was identified as $a_{i_1, i_2, \ldots, i_N}$. Guided by the previously delineated expansion protocols, three distinct two-dimensional feature matrices were derived. Subsequently, tensor decomposition was executed on each of these matrices. As a result, the feature matrix $U \in R^{I^* m}$ symbolising the user, the feature matrix $I \in R^{J^* n}$ characterising the project, and the feature matrix $C \in R^{K^* p}$ denoting the context were all acquired. Additionally, the core vector, referred to as core $=R^{m \times n \times p}$, was ascertained.

3.2 Model network structure

Upon analysis of existing technologies and limitations within the recommendation field, deep learning has been applied, resulting in the introduction of the CAW-NeuMF model. As illustrated in Figure 2, this model employs the user feature matrix U, item feature matrix I, and context feature matrix C as neural network inputs. Initially, basic data encompassing users, items, and ratings are utilised to determine the first K context-weight relationships associated with users via context allocation algorithms. Following dimensionality reduction via high-order tensor decomposition, the comprehensive user, item, and context feature matrices are input into the NeuMF neural network for learning and training.

image015.png

Figure 2. A personalized recommendation model based on deep learning

The NeuMF model, as proposed by He et al. [21], is a variant of Neural Collaborative Filtering (NCF). The General Matrix Decomposition (GMF) extracts low-order (user-item) relations, whilst the Multi-Layer Perceptron (MLP) learns (user-context) relations. The original NeuMF decomposition model, lacking the extraction of efficient auxiliary information for feature modelling, has been enhanced in this study. Contextual auxiliary information is integrated, leading to the formation of CAW-NeuMF, encompassing the refined CAW-NeuGMF and the improved MLP CAW-NeuMLP.

In the CAW-GMF model, the extracted feature vector P_uof user u and feature vector Q_i of item i are element-wise multiplied. Subsequently, the prediction score is derived through a fully connected layer and an activation function as per the given equation.

$\varphi_a\left(P_u, Q_i\right)=P_u \odot Q_i=\left[u_1 i_1, u_2 i_2, \ldots, u_m i_m\right]$              (4)

$\hat{y}_{u i}=\sigma\left(\left(P_u \odot Q_i\right)^T h\right)$              (5)

where, n is the dimension of the characteristic matrix, σ is the activation function, and h is the weight matrix of the full connection layer.

Within the CAW-MLP model, it becomes essential to concatenate the extracted user feature vector P_u, item i feature vector, and context feature vector R_c with weight data. The Eq. (6) is as follows:

$\begin{gathered}z_1=\varphi_1\left(P_u, Q_i, R_c\right) =\left[u_1, \ldots, u_m, i_1, \ldots, i_n, c_1, \ldots, c_t\right]\end{gathered}$                      (6)

Post concatenation, the vector of length m+n+t undergoes MLP processing, and prediction scores are then derived using a fully connected layer alongside an activation function. The calculation process is shown in the following Eqs. (7)-(9).

$\varphi_2\left(z_1\right)=\sigma_2\left(z_1^T w_2+b_2\right)$                      (7)

...

$\varphi_X\left(z_{X-1}\right)=\sigma_X\left(z_{X-1}^T \omega_X+b_x\right)$                      (8)

$\hat{y}_{u i c}=\sigma_X\left(\varphi_X\left(z_{X-1}\right)^T h\right)$                      (9)

where, σ_i is the activation function. For each layer, the RELU function was selected as the activation function. ω_i and b_i are denoted as the weight and bias of the fully connected layer, respectively.

The NeuMF model, designed to amalgamate the strengths of GMF and MLP models, was implemented. Vectors φ_GMF and φ_MLP, derived through GMF and MLP, respectively, were concatenated and subsequently channeled into the terminal NeuMF layer. The underlying principle of this operation was to map the vectors to a one-dimensional space, culminating in the ensuing predictive equations:

$Z_{i j}=\left[\varphi_{i j}^{G M F}, \varphi_{i j}^{M L P}\right]$                      (10)

$\gamma_{u, i, c}=\sigma\left(h^T\left(Z_{u i}\right)\right)$                      (11)

To circumvent the pitfall of overfitting, a loss function enriched with an L2 regularization term was employed in training the model parameters. Within this framework, ω and b represent the discrepancies in weights across various layers of the neural network, while λ serves as the coefficient of the regularization term. The specific configuration of the loss function is delineated in:

Loss $=\frac{\sum n\left(\gamma_{u, i, c}-\tilde{y}_{u, i, c}\right)}{n}+\lambda\left(\omega^2+b^2\right)$                      (12)

For optimization of u_i and v_j, the Adam optimizer was harnessed. Its mode of updating is explicated in the subsequent formula.

$u_i=u_i-\eta \frac{\partial L(U, V)}{\partial u_i}$                      (13)

$v_j=v_j-\eta \frac{\partial L(U, V)}{\partial v_i}$                      (14)

Within the referenced formula, the learning rate, denoted as η, is initially set at 0.001. Unlike conventional random gradient descent optimisation algorithms in machine learning, which maintain a constant learning rate throughout the training, Adam is distinguished by its capacity to prescribe adaptive learning rates for individual parameters. This trait gives Adam an edge over the Stochastic Gradient Descent (SGD). In essence, the Adam optimisation algorithm assimilates the merits of both the adaptive gradient algorithm and the root mean square propagation algorithm. Table 3 is the training procedure predicated on the deep learning context weight algorithm:

Table 3. Training of the CAW-NeuMF personalized recommendation model via deep learning