Fuzzy Wavelet Dynamic Neural Network Model for Modeling the Number of Tourist Visits to West Nusatenggara Province

2.1 Neural network

The Neural Network (NN) method, developed by Warren McCulloch and Walter Pitts in 1943, has been widely applied and successful in solving various data analysis problems. The characteristics of an NN model are determined by several factors, such as its architecture, activation functions, the learning process it employs, and the optimization principles used to achieve convergence during the learning process [22].

The architecture of an NN model describes the working procedure of the model from input to output. The complexity of the architecture is characterized by the number of layers used and the number of neurons in each layer. Activation functions are one of the crucial variables in an NN model, especially concerning the convergence speed of the model. Activation functions determine whether a neuron is activated or not through a comparison process with a certain threshold value. Generally, activation functions come in several types, such as the step function, piecewise linear function, and sigmoid function.

In several studies, the advantages of wavelet functions in denoising, data compression, and multi-resolution [23] have led to their use in enhancing the performance of NN models. Wavelet functions can play a role in NN models as a method for data pre-processing and/or as activation functions. This integration of wavelet functions into NN models can help improve the model's ability to handle noisy or multi-resolution data effectively.

2.2 Fuzzy inference system

Fuzzy sets were introduced by Lotfi A. Zadeh in 1965 . The fundamental difference between fuzzy sets and crisp sets lies in the definition of their membership values, which are within closed intervals $[0,1] \subseteq \mathbb{R}$, crisp sets only have values of 0 or 1. Fuzzy set A is typically denoted by: $A=\left\{\left(x, \mu_A(x)\right) \mid x \in\right.$ $X\}$, with $\mu_A(x)$ representing the element $x \in A$ in $[0,1] \subseteq \mathbb{R}$.

Inference rules in fuzzy sets typically take the form of IF...THEN. To simplify the problem, let's assume $\left(x_1^0, x_2^0, \cdots, x_n^0\right)$ as input and $\left(x_1^{0^*}, x_2^{0^*}, \cdots, x_n^{0^*}\right)$ fuzzification from $\left(x_1^0, x_2^0, \cdots, x_n^0\right)$ and provide $n$ form $R_k, k=1,2, \cdots, n$ as hypotheses, with

$\begin{gathered}R_k: \text { if } x_1 \in A_1 \wedge x_2 \in A_2 \wedge \cdots \wedge x_n \in A_n \\ \text { then } z_k \in C_k\end{gathered}$     (1)

with $x_k:=x_0^{* k}, k=1,2, \cdots, n$.

So that we obtain the conclusion $z \in C$.

The measure of the contribution of the k-th rule in fuzzy inference methods is known as the "fire level" [24]. In general, the implementation of fuzzy inference rules refers to one of the fuzzy inference rules proposed by Mamdani, Tsukamoto, or Sugeno (Takagi, Sugeno, and Kang, TSK). Each of these methods has its advantages and disadvantages depending on the subject and/or research data. For time series data, the TSK method is quite commonly used due to its ease of calculation and its output in the form of linear equations.

Based on Eq. (1), If the fire level is determined by the equality $\quad \alpha_k=A_1\left(x_1^{0^*}\right) \wedge A_2\left(x_2^{0^*}\right) \wedge \cdots \wedge A_n\left(x_n^{0^*}\right), \quad k=$ $1,2, \cdots, n$, in that case, the individual output is given by $z_k^*=$ $a_1 x_1^{0^*}+a_2 x_2^{0^*}+\cdots+a_n x_n^{0^*}+r_k, k=1,2, \cdots, n$. And the overall system output is given by Eq. (2)

$z=\sum\limits_{k=1}^{n}{{{\alpha }_{k}}}z_{k}^{*}/\sum\limits_{k=1}^{n}{{{\alpha }_{k}}}$     (2)

2.3 Wavelet transformation

Wavelet is a class of functions that can localize a function in two aspects: position (time) and scale (frequency). This ability is what makes wavelet transformation advantageous compared to Fourier transformation, and it is widely applied in data processing, such as signal processing and time series analysis.

Mathematically, wavelets are a family of functions constructed from the processes of translation and dilation of a function, defined as follows [25]:

$\psi_{a, b}(t)=|a|^{-\frac{1}{2}} \psi\left(\frac{t-b}{a}\right), a, b \in \mathbb{R}$ and $a \neq 0$     (3)

with $\psi$ mother wavelet, $a$ represents the scale parameter (dilation), determining the degree of compression or scale, and $b$ represents the translation parameter, determining the time location in the wavelet transformation.

In development and application, several wavelet functions are commonly used, including Haar Wavelet, Daubechies, Mexican Hat, Morlet, B-Spline Wavelet, and others. Each type of wavelet has its advantages over the others based on its function and characteristics. The Haar Wavelet function is the simplest type of wavelet and is widely applied because of its simpler calculation characteristics. It is well-suited for analyzing discrete and linear data types. However, for continuous and nonlinear data types, Haar Wavelet may not provide optimal results. In such cases, it is necessary to choose another wavelet function with nonlinear characteristics, such as Daubechies, Mexican Hat, Morlet, B-Spline, and others.

The Daubechies Wavelet is a type of wavelet with compact support developed by Ingrid Daubechies around the 1990s. Unlike the Haar Wavelet, the Daubechies Wavelet has several families known as Daubechies Wavelets of order N (DbN) for N being a natural number and N≥2. The Daubechies Wavelet of order N has 2N vanishing moments and has compact support in the interval [0, 2N-1]. The Daubechies Wavelet is constructed based on the Daubechies Polynomial of order N-1, defined as follows:

${{P}_{N-1}}(y)=\sum\limits_{k=0}^{N-1}{\left( \begin{matrix}   2N-1  \\   k  \\\end{matrix} \right)}{{y}^{k}}{{(-y)}^{N-1-k}}$     (4)

On the other hand, B-Spline Wavelet is a family of wavelets obtained recursively from the convolution of the Haar scaling function [26]. B-Spline Wavelet has several advantages that make it a suitable choice for analysis, including having an explicit formula in both the time and frequency domains, having a symmetric shape, having compact support, and a straightforward manipulation process [27]. Based on the approximation of the Gabor function (cosine-modulated Gaussian), Unser [27] derived the approximation formula for B-Spline wavelets of order n as follows:

$B S^n(x)=\frac{4 b^{n+1}}{\sqrt{2 \pi(n+1) \sigma_w^2}} \cos \left(2 \pi f_0(2 x-1)\right) \exp \left(\frac{-(2 x-1)^2}{2 \sigma_w^2(n+1)}\right)$     (5)

with "$n$" representing the order of the B-Spline wavelet, the constant b=0.657066, $f_0$=0.409177 and $\sigma_w^2$= 0.561145.

4.1 Descriptive analysis

Tourists visiting a region, including West Nusa Tenggara, Indonesia, can be categorized into two groups: foreign tourists and local/domestic tourists. The pattern of tourist visits to West Nusa Tenggara Province from January 2014 to December 2022 is shown in Figure 1. Based on Figure 1, it can be observed that tourist visits to West Nusa Tenggara Province consistently experience an increase around the months of June to September each year, except in 2018 when it advanced from around April to July. The Covid-19 pandemic also had an impact on the decrease in the number of tourist visits to NTB, especially among foreign tourists. Government policies related to restrictions on foreign tourist visits to Indonesia, including NTB at that time, resulted in stagnant foreign tourist numbers until May 2022.

Statistically, as shown in Table 1, during the period from January 2014 to December 2020, the average monthly tourist visits to NTB were 184,772 people, with a breakdown of 70,502 foreign tourists and 114,271 domestic tourists. The maximum number of visits was 544,237 people in September 2016, and the minimum number of visits was 2,610 people in May 2020.

1.png

Figure 1. The number of tourist visits to NTB: total tourists (blue), domestic tourists (gray), and foreign tourists (red)

Table 1. Statistics on tourist visit data visiting NTB

4.2 Modeling the number of tourist visits with a neural network model

The modeling of tourist visits to West Nusa Tenggara Province (NTB) in this research uses a hybrid model that combines the Dynamic Neural Network Model as the core of the model and the Wavelet and Fuzzy Inference Method as tools to optimize the model. In this study, the proposed hybrid model is called the Fuzzy Wavelet Dynamic Neural Network (FWDNN) model. The architecture of the proposed FWDNN model is shown in Figure 2.

2.png

Figure 2. The proposed of FW-DNN architecture

4.3 The algorithm of the proposed FWDNN model

The process of modeling the number of tourist visits to West Nusa Tenggara Province (NTB) using the Neural Network Model is provided by the following FW-DNN algorithm:

The proposed fwdnn algorithm:

Step 1: The input data is organized using Eq. (6), referred to as matrix X.

Step 2: The input matrix X has a size of $n\times m$ and is transformed using the normalization method with the formula:

${{x}_{k}}=\frac{x_{k}^{\prime }-x_{\min }^{\prime }}{x_{_{\max }}^{\prime }-x_{\min }^{\prime }}$     (7)

with $x_k^{\prime} \quad x_{\max }^{\prime}$ and $x_{\min }^{\prime}$ respectively denoted $k$-th data, maximum data value and minimum data value. Then, the data is further processed with a Wavelet Daubechies of 2-nd order transformation (Eq. (4)).

Step 3: The data, after normalization and Wavelet Daubechies of 2-nd order transformation, is aggregated with the weight matrix W of size $n\times m$ using the formula:

${{q}_{k}}=\sum\limits_{j=1}^{n}{{{w}_{jk}}}{{x}_{j}},k=1,2,\cdots ,m$     (8)

Step 4: Value $q_k, k=1,2, \cdots, m$ is activated using the Wavelet B-Spline function of order m=3, as given by Eq. (5), resulting in:

${{z}_{k}}=B{{S}^{m}}\left( {{q}_{k}} \right),k=1,2,\cdots ,n$     (9)

Step 5: For each $k=1,2, \cdots, m$, value $z_k$ It is defuzzied using Gaussian-type fuzzy membership functions with the formula:

${{p}_{k}}=\operatorname{Gauss}(x,[\mu ,\sigma ])=\exp \left( -\frac{1}{2}\times {{\left( \frac{x-\mu }{\sigma } \right)}^{2}} \right)$     (10)

with $\mu$ and $\sigma$ respectively mean and standard deviation from $x_k$.

The Gaussian results are then inferred using the fuzzy TSK method with the rules:

if ${{p}_{1}}\in {{A}_{1}}\wedge {{p}_{2}}\in {{A}_{2}}\wedge \cdots \wedge {{p}_{k}}\in {{A}_{k}}$ then ${{h}_{k}}=\sum\limits_{j=1}^{12}{{{u}_{jk}}}{{p}_{j}}+{{c}_{0}}$     (11)

for some real constant c₀.

Step 6: The output of the FW-DNN model is obtained using weighted aggregation with the formula:

$y=\alpha \times \sum\limits_{k=1}^{12}{{{v}_{k}}}{{h}_{k}}+\beta $     (12)

for some real constant $\alpha$ and $\beta$.

4.4 The learning process of the FW-DNN model

The optimization of parameters for learning the FW-DNN model is performed in the backpropagation step. In this case, the parameters being optimized include the weight matrices W, U, and V. The optimization of each of these parameters is carried out using the gradient descent with momentum method to minimize the cost function:

$E=\frac{1}{N}\sum\limits_{j=1}^{N}{{{\left( {{y}_{j}}-y_{j}^{d} \right)}^{2}}}$     (13)

where, $N$ represents the number of data rows, $y_j$ and $y_j^d$ respectively denote the output values of the proposed FWDNN model and the target data value for the $k$-th row, where $k$ ranges from 1 to $N$.

The optimization process of weight parameters is carried out based on the following partial differential equations:

$\frac{\partial E}{\partial W}=\frac{\partial E}{\partial y}\frac{\partial y}{\partial h}\frac{\partial h}{\partial p}\frac{\partial p}{\partial z}\frac{\partial z}{\partial q}\frac{\partial q}{\partial W}$     (14)

$\frac{\partial E}{\partial U}=\frac{\partial E}{\partial y}\frac{\partial y}{\partial h}\frac{\partial h}{\partial U}$     (15)

$\frac{\partial E}{\partial V}=\frac{\partial E}{\partial y}\frac{\partial y}{\partial V}$     (16)

Next, the weight adjustment process for each weight matrix is showed by $W=\left[w_{i j}\right], U=\left[u_{i j}\right], V=\left[v_{i j}\right]$, using the following equation:

${{w}_{ij}}=w_{ij}^{0}+dW,\ \ \ i=1,2,\cdots ,mdanj=1,2,\cdots ,12.$     (17)

${{u}_{ij}}=u_{ij}^{0}+dU,\ \ \ i,j=1,2,\cdots ,12$     (18)

${{v}_{i}}=v_{i}^{0}+dV,\ \ \ i=1,2,\cdots ,12$     (19)

with $w_{i j}^0, u_{i j}^0, v_i^0$ respectively show $w_{i j}, u_{i j}, v_i$ in the previous steps, and

$dW=m\times {{w}_{ij}}-{{\eta }_{r}}\times (1-m)\times \partial {{w}_{ij}}$      (20)

$dU=m\times {{u}_{ij}}-{{\eta }_{r}}\times (1-m)\times \partial {{u}_{ij}}$     (21)

$dV=m\times {{v}_{i}}-{{\eta }_{r}}\times (1-m)\times \partial {{v}_{i}}$     (22)

and $m, \eta_r, \partial w_{i j}, \partial u_{i j}, \partial v_i$ respectively represent the momentum parameter, learning rate, and weight change in the matrix $W=\left[w_{i j}\right], U=\left[u_{i j}\right]$ and $V=\left[v_{i j}\right]$ on Eqs. (17)-(19).

4.5 The FW-DNN model for modeling the number of tourist visits to NTB

4.5.1 The FW-DNN model for tourist visits to NTB

Based on the time series data of tourist visits to West Nusa Tenggara Province as shown in Figure 1, an autocorrelation analysis was conducted, and the results are as follows:

$f\left( {{x}_{t}} \right)=f\left( {{x}_{t-1}},{{x}_{t-2}},{{x}_{t-3}} \right)$

with $f\left(x_t\right)$ is the number of consecutive tourist visits at time $t$ and $f\left(x_{t-1}\right), f\left(x_{t-2}\right), f\left(x_{t-3}\right)$ respectively number of tourist visits at a specific time $(t-1),(t-2)$, and $(t-3)$.

Modeling using the FW-DNN model proposed in this research involves a total of 209 parameters, including 204 weight parameters from matrices W, U, and V, 3 learning rate parameters, and 2 parameters for Gaussian membership functions. The modeling process with FW-DNN learning for 10,000 iterations produces results as shown in Figure 3, with the model pattern adequately representing the patterns of tourist visitation data to West Nusa Tenggara Province.  Based on the root of mean square error (RMSE) criteria, the performance of the FW-DNN model generated for in-sample data is 95,185.09, and for out-sample data is 22,615.54.

3.png

Figure 3. The number of tourist visits to NTB from January 2014 to December 2020 using the FW-DNN model blue dashed line and actual data (red solid line)

4.5.2 The FW-DNN model for international tourist visits to NTB

Based on the time series data of international tourist visits to West Nusa Tenggara Province as shown in Figure 1, an autocorrelation analysis was conducted, and the results are as follows:

$f\left( {{x}_{t}} \right)=f\left( {{x}_{t-1}},{{x}_{t-2}},{{x}_{t-3}},{{x}_{t-4}} \right)$

with $f\left(x_t\right)$ is the number of consecutive tourist visits at time $t$ and $f\left(x_{t-1}\right), f\left(x_{t-2}\right), f\left(x_{t-3}\right), f\left(x_{t-4}\right)$ respectively number of tourist visits at a specific time $(t-1),(t-2),(t-$ $3)$, and $(t-4)$.

Modeling using the FW-DNN model proposed in this research involves a total of 209 parameters, consisting of 204 weight parameters from matrices W, U, and V, 3 learning rate parameters, and 2 parameters for Gaussian membership functions. The modeling process with FW-DNN learning for 10,000 iterations produces results as shown in Figure 4, with the model pattern adequately representing the patterns of tourist visitation data to West Nusa Tenggara Province. Based on the root of mean square error (RMSE) criteria, the performance of the FW-DNN model generated for in-sample data is 39,848.94, and for out-sample data is 5,223.86.

4.png

Figure 4. The number of international tourist visits to NTB from January 2014 to December 2020 using the FW-DNN model (blue dashed line) and actual data (red solid line)

4.5.3 The FW-DNN model for domestic tourist visits to NTB

Based on the time series data of domestic tourist visits to West Nusa Tenggara Province as shown in Figure 1, an autocorrelation analysis was conducted, and the results are as follows:

$f\left( {{x}_{t}} \right)=f\left( {{x}_{t-1}},{{x}_{t-2}},{{x}_{t-3}} \right)$

with $f\left(x_t\right)$ is the number of consecutive tourist visits at time $t$ and $f\left(x_{t-1}\right), f\left(x_{t-2}\right), f\left(x_{t-3}\right)$ respectively number of tourist visits at a specific time $(t-1),(t-2)$, and $(t-3)$.

Modeling using the FW-DNN model proposed in this research involves a total of 209 parameters, including 204 weight parameters from matrices W, U, and V, 3 learning rate parameters, and 2 Gaussian membership function parameters. The modeling process with FW-DNN learning for 10,000 iterations produces results as shown in Figure 5, with the model pattern adequately representing the patterns of tourist visitation data to West Nusa Tenggara Province. Based on the root of mean square error (RMSE) criteria, the performance of the FW-DNN model generated for in-sample data is 71,855.35, and for out-sample data is 17,301.58.

5.png

Figure 5. The number of domestic tourist visits to NTB from January 2014 to December 2020 using the FW-DNN model (blue dashed line) and actual data (red solid line)

Table 2. Tourist visitation statistics for those who visited West Nusa Tenggara Province

Statistically, the application of the FW-DNN model to tourist visitation data as a whole, domestic tourists, as well as international tourists visiting West Nusa Tenggara Province is presented in Table 2.

The results of our research are in line with and strengthen the results of research that has been conducted by several researchers. These researchers include Xu et al. [28] who showed that the combination of fuzzy inference with an artificial neural network model was able to increase the accuracy of the model in terms of classifying four types of objects resulting from the discrimination of reservoir water-flooded states based on physical quantity values such as resistivity, acoustic level, and radioactivity level in the oil layer. Likewise, research conducted by the study [29] shows that the fuzzy wavelet neural network (FWNN) method with a combination of PSO techniques and gradient descent optimization provides more efficient model performance results and has higher precision for short-term wind power forecasting. In addition, research by the study [30] combining artificial neural networks (ANN) and fuzzy logic resulted in efficient and significant model performance for estimating porosity in the petroleum industry. Finally, these results specifically strengthen the results of our previous research [5] that optimization of the wavelet neural network (WNN) model with TSK type fuzzy inference is able to improve the performance of the WNN model in case studies of several types of univariate time series data.