Novel Response Relation Method for Sensor Data Analysis of Complex Engineering Systems

Data-Driven Modeling has increased momentum with huge growth of articles utilising a variety of algorithms. Yan and Zhang [1] proposed a method for analysing and selecting features from correlated gas sensor data. To achieve a promising performance, this model's recursive feature removal technique used a correlation bias reduction strategy with linear and nonlinear SVM-RFE algorithms. Day et al. [2] presented the concept of autonomic feature selection through the lens of two concepts: representation and transfer learning. A system's representations for various forms of monitoring data are learnt, and the resulting knowledge is shared and reused. Su et al. [3] investigated the sensor data's feature engineering elements. Their approach entailed gathering sensor data correlation changes in order to enhance the detection of IoT (Internet of Things) equipment anomalies. Mosallam et al. [4] established a comprehensive foundation for RUL Prediction that is applicable to a wide variety of issues via the Bayesian approach. Li et al. [5] proposed a novel promising strategy for remaining useful life projections that requires no prior experience or signal processing. They used a deep convolutional neural network-based method. They examined the effect of critical factors on model parameter optimization for prognostic performance. Yu et al. [6] established a cluster-based data analysis framework based on recursive principal component analysis (R-PCA) that aggregates correlated sensor data fast and accurately with outlier identification. Hromic et al. [7] extended the OpenIoTmiddleware's functionality for stream processing, providing real-time demand analytics on data streams. Ren et al. [8] proposed an integrated deep neural network approach for estimating the remaining useful life of a rolling bearing by combining time domain and frequency domain information. Le Son et al. [9] described a probabilistic approach to prognosis by integrating a data analysis technique (Principal Component Analysis) with a stochastic process (Wiener process) and applying it to the 2008 PHM Conference Challenge data. It simulates component deterioration in order to determine the RUL. Liao et al. [10] employed a logistic regression model and a proportional hazards model to develop a relationship between the numerous degradation characteristics of sensor inputs and the unit's specific reliability indices, allowing for the prediction of the unit's RUL. Liu et al. [11] developed a data-driven approach for RUL prediction that relies on sensor anomaly detection and data recovery. The associated algorithm detects and recovers aberrant sensor data in order to provide input to the RUL prediction algorithm. Mutual information, least squares support vector machine (LS-SVM), kernel principal component analysis (KPCA), and Gaussian process regression are all used (GPR). Medjaher et al. [12] developed a data-driven prognostics technique using Dynamic Bayesian Networks (DBNs) in conjunction with Mixture of Gaussian Hidden Markov Models (MoG-HMMs). The RUL assessment is based on the crucial component that has been identified and the sensors that have been deployed. The prognostics method was divided into two phases: a learning phase in which the behaviour model was generated, and an exploitation phase in which the present health state was estimated and the RUL was computed. Zhang et al. [13] discussed the evolution of Wiener-process-based approaches for analysing degradation data and estimating RUL. By taking into account nonlinearity, multi-source variability, confounders, and multivariate, the degradation process concentrated on Wiener process variations. Wang et al. [14] introduced a unique strategy for RUL estimation termed functional Multilayer Perceptron (functional MLP). This is a revolutionary Functional Data Analysis (FDA) technique. Wang et al. [15] suggested a method for obtaining massive run-to-failure data for an engineering system. With data from numerous units within the same subsystem, a library of deterioration patterns is built. Estimation is based on the actual mapped units to the library's patterns. Loutas et al. [16] present a data-driven approach for estimating the remaining usable life (RUL) of rolling element bearings using Support Vector Regression (SVR). Massive research on feature selection necessitates a pre-learning procedure that is difficult to scale for high-dimensional data analytics that needs dynamic feature selection. As a result, Hoi et al. [17] recommended two distinct research directions for feature selection approaches. One can infer that the number of features remains constant throughout while the rest change. Wang et al. [18] proposed an Online Feature Selection (OFS) technique based on the assumption that data instances are provided consecutively and that feature selection occurs as each data instance arrives. Wu et al. [19] introduced a straightforward but intelligent second-order online feature selection algorithm that is exceptionally efficient, scalable to large scale, and capable of handling extremely high dimensional data. Perkins and Theiler [20] presented the Grafting algorithm for this type of online feature selection. It is based on a stage-wise gradient descent approach. It regards convenient feature selection as an inherent aspect of learning a predictor in a regularised learning framework and gradually expanding a feature set while incrementally iteratively training a predictor model using gradient descent. Zhou et al. [21] proposed Alpha-investing as a new feature. Alpha-investing, on the other hand, requires previous knowledge of the original feature set and never assesses the redundancy among the selected features over time. Sunkara et al. [22] presented an integrated framework for IoT Systems that is suitable to engineering and industrial systems, with an emphasis on business and IoT intelligence integration. The current article expands on parts of integrated analysis by addressing the interdependence of sensor variables. Yu et al. [23] described a Scalable and Accurate Online Approach (SAOLA) technique for handling feature selection problems with exceptionally high dimensionality. It conducted a theoretical analysis and derived a lower bound on the correlations between features for paired comparisons, as well as proposing a set of online pairwise comparisons for the purpose of maintaining an economic model over time. Minor et al. [24] addressed the considerable issues associated in converting large-scale sensing data into judgments for real-world applications. Liu et al. [25] proposed a quantitative selection method based on information theory for sensor data in order to determine remaining useful life. Mosallam et al. [26] described a strategy for selecting unsupervised variables. Different health indicators (HIs) describe the degradation over time and are saved as reference models in the offline database. The approach identifies the offline HI that is the most comparable to the online HI in the online phase, and uses the k-nearest neighbour classifier as an RUL predictor. Djeziri et al. [27] studied wind turbine fault prognosis in the presence of numerous faults. The project sought to develop a physical model that could be utilised for structure analysis, sensor positioning, and cluster creation. Si et al. [28] discussed recent advances in modelling for calculating the RUL using statistical data-driven methodologies. They are divided between models that incorporate directly observable asset state information and those that do not. Zhang et al. [29] proposed a method for assembling multiobjective deep belief networks (MODBNE). The MODBNE approach combines a multiobjective evolutionary algorithm with the classic DBN training strategy to evolve several DBNs concurrently while balancing the competing aims of accuracy and diversity. Wu et al. [30] developed a quicker version of the Online Streaming Feature Selection (OSFS) algorithm, dubbed the Fast OSFS algorithm. Katukam et al. [31] employed a neural network approach to forecast the status of industrial equipment such as refrigerators, and this model contains an optimization strategy for guaranteeing that the equipment operates at the lowest possible energy and time consumption. Moraru et al. [32] used machine learning models to the processing of sensor data using the sensor node technique. The node senses parameters such as temperature (C), humidity (percent), light (Lux), and pressure (hPa), which are then processed by the internet gateway. Taha et al. [33] investigated the use of machine learning to create models for simulated aircraft sensor data provided by a turbofan aircraft engine. Although the engine is composed of several subsystems that each have their own set of parameters such as pressure, temperature, and flow rate, the suggested methodology estimates the remaining useful life (RUL) using only a fraction of the overall sensor data. The accuracy of the RUL forecast can be increased by considering as many parameters as possible. The model is constructed by removing the sensors' extreme values in order to approximate a Gaussian distribution. Okoh et al. [34] presented both regression-based and deep learning algorithms for estimating the remaining useful life. The computing costs of the methods discussed above become prohibitively expensive when the dimensionality is extremely high, on the order of millions or more. To summarize, the existing research of feature engineering seldom focuses on correlation changes, thus limiting the characteristics of correlation changes. Hence, the selection of good condition data is crucial for applying the data-driven methodology to be useful for the system with degradation characteristics.

In this proposed methodology, a novel solution as illustrated in Figure 5 is presented to perform feature engineering, find the critical sensors and the relationship among them using the following ensemble of algorithms. Initially, sensor readings are supplied to the framework in a suitable data format. Then the distribution of all variables is studied using statistical tools, and statistical properties are evaluated. A time or cyclic variation of the data is derived using the difference of consecutive data points. The standard deviation of the whole dataset is computed initially, and the same is classified into zero or non-zeros clusters. All non-zero standard deviation variables are grouped to be eligible for participation in further computation. For a non-zero group of sensors, a relationship matrix is formed. The computed relationship matrix is based on the Vector Auto Regression approach. Vector Auto Regression approach is a multi-variable probabilistic model that will give the probabilistic relationships among two variables through the impulse response function. VAR computation methodology is a standard library. Hence the explanation of the same is beyond the scope of the current paper. VAR considers the Lead /Lag relationship and influence probability between two variables to build the relationship matrix. From the relationship table, a ranking algorithm is used to find the variables with higher relation. These chosen sets of variables will be further used for developing machine learning algorithms. An error criterion like Root Mean Square Error (RMSE) is used to cross-check the model’s accuracy for a chosen set of sensor values.

4.1 Response relation algorithm

Input: Multivariate Time Series Data.

Output: Significant features to be used in prediction.

Step 1: Consider time series data S=[s1,s2,s3…sn]

            For each Si in S

               Calculate Std(Si)

                    If  Std(Si==0)

              zs=append.Si //append to Zero Cluster of variables

                   else

     nzs=append.S //append to Nonzero  Cluster of Variables

Step 2:  For each Si in nzs

 res= ADFtest(Si)

             If res is stationary

               goto step3

            else

              apply transformations and goto step 2

Step 3: Granger casuality test (nzs) 

            For each Si in nzs

                 For each Sj in nzs

                       grangercasuality(Si,Sj)

             If (P-Value<=0.05) && (Fvalue>Fmean)

                   Add(Si.Sj) to selected variables

Step 4: For each Si in selected variables list

                  sumlag(Si)              //find the sum of all lags

Step 5: Rank them based on the highest value

Step 6: Choose the top most ranked variables to participate in the prediction of RUL.

5.png

Figure 5. Response Relation methodology

Given a variable, several statistical tools and machine learning models are used to forecast its future value. These are models capable of forecasting for a single or numerous variables. Typically, models are constructed using simple linear approaches or sophisticated neural networks and begin with a fundamental mathematical relationship between one or more independent variables and the dependent variable. Linear Regression is a machine learning approach that uses supervised learning to perform regression tasks. Regression modelling is used to create a target prediction value based on independent variables. It is mostly used to establish relationships between variables and to forecast their future values. The regression models vary in terms of the number of independent variables and the relationship between the independent and dependent variables. Linear regression models are more appropriate when the parameters are linearly dependent on one another. The current research is aimed at precisely calculating the remaining time to failure for all engines. As a result, the dependent variable is defined as the time to failure (TTF). A linear regression model is a critical component of any predictive study since it provides the initial estimate.

Table 3 indicates the statistical relationship value among all the variables after applying the selection ranking criteria as 0.05. This ranking can be chosen based on the dataset and domain expertise.

Table 3. Relationship table after selection

From the above table, seven sensors are selected before feeding the machine learning prediction algorithm. These sensors are namely setting2, s4, s6, s8, s11, s13, and s20.

Industrial IoT is an emerging technology that can significantly assist in enhancing machine’s secure and intelligent operation. A practical application of such technology integrates machine learning models facilitating the business benefits. An integrated framework is developed for Industrial IoT with feature selection as one of the critical aspects of IoT system development and application. Feature elimination ranges from using a standard deviation-based approach to complex energy-based indicators unlike Physics-driven models which rely on the interrelation of variables that are participating in a phenomenon. In data-driven approach, the relationship needs to be established using interrelation among sensor variables in response and relation. In the present study, the approach is to choose few critical sensors from all the available sensors combined with machine learning models like KNN, Decision Tree, Logistic Regression, and SVC. Relation Response method applied to both Binary and Multi-Classification methods indicate that Relative Response method matches well with all variable methods. Further, this method when applied to Binary and Multi-class classification proved to be very effective.