Design and Application of a Prediction Model for User Purchase Intention Based on Big Data Analysis

With the proliferation of information technology, more and more enterprises have attached importance to the Internet and the Internet of things (IoT) in management and product marketing [1-5]. To increase the return rate of marketing, an enterprise must step up the investment in precision marketing plans, and establish a sound personalized recommendation system. The key to precision marketing lies in the prediction of user purchase intentions [6-8]. However, it is impossible to predict these intentions accurately, unless useful information has been mined from the massive historical data on users and products. The accurate prediction of user purchase intentions enables enterprises to quickly identify potential sales targets [9-11].

The user purchase behaviors are usually predicted by algorithms like decision tree (DT), k-nearest neighbors (k-NN), support vector machine (SVM), Bayesian network, and logistic regression [12-17], and their combinations with machine learning (ML) [18, 19]. After dividing users by purchase rate, Kaneko and Yada [20] constructed an online prediction model for user purchase rate based on beta geometric/negative binomial distribution (BG/NBD), which can accurately forecast user purchase behaviors in 2.5 years. To predict the purchase behaviors of Taobao users, Kulkarni [21] added three concomitant variables, namely, the number of reviews, the number of favorites, and the repeat purchase rate, to the hybrid model of SVM and HIPP. Robinson et al. [22] introduced the recency-frequency-monetary (RFM) model into the association rules of the traditional BG/NBD model, which, coupled with the update of weight coefficients, can process online purchase information, and predict user purchase trend in real time. Mistry et al. [23] combined Markov chain theory and collaborative filtering recommendation algorithm, and examined the relationship between purchase intention and non-purchase intention according to the information of shopping cart and favorites.

The existing studies mainly tackle the influencing factors of user purchase behaviors. It takes a long time for the current prediction models to construct features and predict behaviors. These models are not suitable for predicting user purchase intentions, without hypothetical limitations on historical purchase data.

This paper presents a brand-new prediction model for user purchase intentions. Firstly, the historical data on user purchases were subject to quality and feature analyses, and preprocessed by a self-designed procedure. Under the framework of DenseNet, the various features of user purchase behaviors were summarized and optimized, and the features of user purchase intentions for products were derived and described. On this basis, a DenseNet purchase intention prediction model was established on Bagging strategy. Then, the authors clarified the structure of the random forest (RF) unit of Bagging strategy, and specified the workflow of gcForest algorithm. The proposed prediction model was proved effective through experiments.

3.1 Model framework

Figure 2 shows the architecture of the DenseNet model that integrates basic features, purchase features, and sales features to predict user purchase intentions. First, the historical data on user purchases were preprocessed, the features that may affect user purchase intentions (e.g. number of purchases, total sales, number of repurchases, and total secondary sales) were imported to the dense blocks and transition layers of the DenseNet for optimization. The salient features were extracted from the optimized purchase data sequence and sales data sequence, and used to predict user purchase intentions by gcForest algorithm under the Bagging strategy. The multi-dimensional scanning module in gcForest compresses the high-dimensional purchase information and sales information into a set of multi-size eigenvectors. Then, the cascading structure outputs the class distribution of each input.

2.png

Figure 2. DenseNet purchase intention prediction model

3.2 Feature optimization

Table 1. Features of user purchase behaviors

The historical data on user purchases cover various features, including but not limited to basic features, purchase features and sales features. Through preliminary analysis, many features were found useful for predicting user purchase intentions. Due to the sheer number of such features, it is impossible to predict the intentions effectively with DenseNet model.

Based on the statistical analysis of available features, this paper summarizes the typical predictive features of user purchase intentions. The features shown in Table 1 above were imported to the DenseNet model to simplify the training process, making the information processing faster. The feature optimization process is detailed as follows:

Let User={u₁, u₂, …u_n} be the set of users, Product={p₁, p₂, …p_m} be the set of products, Behavior be the set of user behaviors, and t be behavior time. Then, the correlations between users and products can be expressed as:

$Relevance⊆User $$\times$ $Product $$\times$ $Behavior$ $\times t$     (1)

where, Behavior＝{1, 2, 3, 4} with 1, 2, 3 and 4 being browsing, favorite, purchase and repurchase, respectively.

The purchase features of user u_i can be characterized by the number of purchases u_i×|Behavior₃+Behavior₄|, and the ratio of number of purchases to the total number of behaviors u_i×|Behavior₃+Behavior₄| / |Behavior|.

The sales features of product p_j can be characterized by the total number of browses p_j×|Behavior₁|, and the total sales p_j×|Behavior₃+Behavior₄|.ssssss

Let Time_h be the set of behavior times h days away from the predicted purchase times. Then, the total number of behaviors k that are l days away from the predicted times can be counted by:

$C_{h}^{k}=\left\{t \mid t \in \bigcup_{h=1}^{l} \operatorname{Time}_{h}\right\}$    (2)

The correlations between users and products can be characterized by the total number of the four types of user behaviors that are j days away from the predicted times:

$C_{h}=\bigcup_{k=1}^{4} C_{h}^{k}$     (3)

Then, the purchase intention of user u_i for product p_j can be described by:

$u_{i} \times p_{i} \times \mid$Behavior$_{3}+$Behavior$_{4} \mid \times C_{h}$     (4)

3.3 Bagging-based DenseNet prediction of purchase intentions

If user purchase intentions are predicted by the traditional CNN-based deep learning, the hyperparameters of the CNN need to be adjusted repeatedly, and a long time is consumed in each training of model parameters. To make matters worse, the distributions of user and product samples are imbalanced in the complex structure of massive data. After all, different kinds of user behaviors (i.e. browsing, favorite, purchase and repurchase) differ greatly in quantity.

The above problems can be solved effectively by the DenseNet, a D-CNN, under the Bagging strategy. This model can optimize the user and product features that enhance purchase intentions, and effectively identify and process the high-dimensional ones through multi-dimensional scanning and cascading structure.

The DenseNet has a unique densely connected feature transfer structure (Figure 3), which mitigates the vanishing gradient problem in the training of such a deep network. Following the idea of cross-layer connection, all the features are directly connected between the deep and shallow layers, eliminating the need to judge or classify them in the last layer. Since the features are fully transferred between deep and shallow layers, the decision function of DenseNet acquires strong generalization ability.

3.png

Figure 3. Densely connected feature transfer structure

As shown in Figure 3, each layer of the DenseNet involves batch normalization (BN), the activation function Leaky ReLU (rectified linear unit), and convolution. Let f_r be the operation function of the r-th layer. Then, the output y_r of the r-th layer can be expressed as:

${{y}_{r}}={{f}_{r}}\left\{ \left[ y_{_{1}}^{r},y_{2}^{r},,y_{s}^{r} \right] \right\}$     (5)

In the DenseNet, the input data of a layer contain the output features of the previous layer, as well as the features of the original data and all the other layers.

It is assumed that the DenseNet has a total of A layers. If the weight parameters are pretrained for the first a layers, then the structure and weight parameters of the last A-a layers should be adjusted as per the predicted purchase intentions.

Let P_O(u_i, p_j) be the old prediction model of the purchase intention of user u_i for product p_j, that is, the source domain of training, and P_N(u_i, p_j) be the new prediction model of the purchase intention of user u_i for product p_j, that is, the target domain of training. Then, the source domain can be expressed as:

$\begin{align}  & {{P}_{O}}({{u}_{i}},{{p}_{i}}) \\ & =\{{{I}_{O}}({{u}_{i}},{{p}_{i}}),L{}_{O}({{u}_{i}},{{p}_{i}})\},{{P}_{N}}({{u}_{i}},{{p}_{i}}) \\ & =\{{{I}_{N}}({{u}_{i}},{{p}_{i}}),{{L}_{N}}({{u}_{i}},{{p}_{i}})\} \\\end{align}$     (6)

where, I_O, L_O, and D_O are the predicted intention, feature number and feature distribution of the old model, respectively; I_N, L_N, and D_N are the predicted intention, feature number and feature distribution of the new model, respectively.

Since a user either intends to or does not intend to purchase a product, the prediction of user purchase intentions is a binary classification problem. Then, P_O(u_i, p_j) and P_N(u_i, p_j) can be predicted and classified by the DenseNet as follows:

$\begin{align}  & {{{\hat{U}}}_{O}}({{u}_{i}},{{p}_{i}}) \\ & ={{\Phi }_{O}}\left[ {{I}_{O}}({{u}_{i}},{{p}_{i}})\left| {{w}_{O}} \right. \right],\text{  }{{{\hat{U}}}_{N}}({{u}_{i}},{{p}_{i}}) \\ & ={{\Phi }_{N}}\left[ {{I}_{N}}({{u}_{i}},{{p}_{i}})\left| {{w}_{N}} \right. \right] \\\end{align}$     (7)

where, $\Phi_{O}$ and $\Phi_{N}$ are the old model and new model of the DenseNet, respectively; U_O and U_N are the outputs of the old model and new model, respectively.

The error between predicted purchases and actual purchases can be characterized by cross-entropy loss:

$M({{u}_{i}},{{p}_{j}})=-\sum\limits_{i=0,j=0}^{n,m}{P({{u}_{i}},{{p}_{j}})\log R({{u}_{i}},{{p}_{j}})}$     (8)

where, R(u_i, p_j) is the actual purchase or repurchase of user u_i for product p_j. Let w_O and w_N be the weight parameters of the old model and new model, respectively. The training process can be described by:

$\begin{align}  & {{w}_{O}} \\ & =\underset{{{w}_{O}}}{\mathop{\arg \min }}\,\left\{ M\left\langle {{U}_{O}}({{u}_{i}},{{p}_{i}})-{{\Phi }_{O}}\left[ {{I}_{O}}({{u}_{i}},{{p}_{i}})\left| {{w}_{O}} \right. \right] \right\rangle  \right\} \\\end{align}$     (9)

Throughout the DenseNet training, the weight parameters of the first a layers are transferred to the new prediction task. Suppose w_O(1: a) and w_N(1: a) are the weight parameters of the first a layers and the target domain models in the old model and new model, respectively. Then, we have:

${{w}_{N}}(1:a)={{w}_{O}}(1:a)$     (10)

Based on the features of the new prediction task, the weight parameters of the last A-a layers in the new model are trained by:

$w_{N}(1 * A)=\left[w_{N}(1 * a), w_{N}(a * A)\right]$$=\underset{{{w}_{N}}(a*A)}{\mathop{\text{argmin}}}\,\left\{ M\left\langle {{U}_{N}}-{{\Phi }_{N}}\left[ {{I}_{N}}({{u}_{i}},{{p}_{i}})\left| \begin{align}  & {{w}_{N}}(1*a), \\ & {{w}_{N}}(a*A) \\\end{align} \right. \right] \right\rangle  \right\}$    (11)

where, w_n(1: A) is all the weight parameters of the new model.

The historical data on user purchases can be divided into multiple training subsets, according to the distance to the predicted times. Therefore, the feature subsets generated by single networks must be combined before DenseNet sample training. Figure 4 shows the structure of a RF unit based on Bagging strategy.

4.png

Figure 4. Structure of a RF unit based on Bagging strategy

However, a single RF unit cannot adapt its complexity or computing load to multi-dimensional or fuzzy abstract eigenvectors. For the robustness of prediction, this paper cascades multiple RF units into the gcForest, and combined it with DenseNet into a hybrid model.

Before clustering the output features of DenseNet, gcForest makes multiple parallel predictions on feature information. The feature information is high-dimensional and abstract for the prediction of user purchase intentions. Thus, the voting mechanism was selected to cluster the results. Since the prediction is a binary classification problem, each Forest outputs a two-dimensional (2D) vector. The basic flow of the gcForest algorithm is as follows:

Step 1. Initialize u_i =0 and p_j =0;

Step 2. Obtain two complete RFs and two RFs through P_g(u_i, p_j) training, and combine them into the g-th layer of gcForest;

Step 3. Train the 2D class vector outputted by each RF on the g-th layer separately, and combine the results into a 2D class vector set Y for sample subset;

Step 4. Compute the mean value Y of the 2D class vector set, and determine the class argmax`Y of the predicted result of the g-th layer;

Step 5. Set up a voting mechanism with threshold δ=0.5, and verify the predicted results on test set; if the classification accuracy μ_i is not greater than δ, output purchase; otherwise, output non-purchase;

Step 6. Combine the 2D class vectors outputted by the RFs on the g-th layer into a new prediction subset, iteratively process P_g(u_i, p_j), and repeat Step 2.

This paper mainly designs a new prediction model for user purchase intentions. Firstly, the historical data on user purchases were subjected to quality and feature analyses, and preprocessed through cleaning, fusion, and feature update. Based on DenseNet, the architecture of the prediction model was designed, and the features of purchase behaviors, including basic features, purchase features and sales features, were summarized and optimized. Then, the features that affect the purchase intentions were derived and depicted in details. On this basis, a DenseNet purchase intention prediction model was developed under the Bagging strategy, the structure of the RF unit was constructed, and the basic flow of gcForest algorithm was illustrated. Finally, our model was compared with three classification and prediction algorithms through experiments. The comparison shows that the prediction of our model was closer to reality and better in accuracy than that of any contrastive algorithm. Therefore, our model is effective in predicting user purchase intentions.