The Role of Recurrent Convolutional Neural Network in IoT for Building a Security Artificial Intelligence and Home Assistance System

Recurrent Convolutional Neural Network (RCNN) is an algorithm that can be tested with a moving object recognition dataset [1]. A process that requires only a few parameters, RCNN achieves better results than state-of-the-art CNN on these data sets, which validates the advantages of RCNN over CNN [2]. Various RCNN studies such as those carried out by conducting research [3] with RCNN where the basic process of labeling the scene is to be able to detect objects by considering the feedforward distance which is calculated based on the distance from the label in the presence of a control system by ANN. Besides that [4] performing moving object detection with RCNN shows that RCNN is able to improve CNN performance by including more facts in this dataset.

From various studies that arise, of course, it is able to do good detection on RCNN [5]. The benefits of the results and the neural network method are not only limited to the accuracy produced, but also the results of the method are embodied in hardware both in computer networks and energy [6]. At present, hundreds of millions of devices such as those that have developed on IoT (Internet of Things) are deployed in various application fields, realizing the "data => information => knowledge" process through big data analysis [7]. The Internet of Things (IoT) is a new paradigm that enables communication between electronic devices and sensors via the Internet, making our lives more convenient [8]. IoT uses smart devices and the Internet to provide innovative solutions to various challenges and issues related to various public/private businesses, governments, and industries around the world [9].

With the increasing amalgamation of IoT devices and technologies, we are seeing major changes in our daily lives [10]. One of these IoT developments is the concept of smart home systems (SHS) and devices, which consist of internet-based devices, home automation systems, and reliable energy management systems [11]. Apart from that, another important achievement of IoT is the Smart Health Sensing System (SHSS) [12]. SHSS combines small smart appliances and devices to support human health [13].

IoT technology continues to be developed and integrated like human performance [14]. This activity is called Artificial Intelligence [9]. Artificial Intelligence or artificial intelligence is intelligence that is added to a system that can be regulated in a scientific context [15]. AI was introduced after the creation of the idea of a digital computing machine that aims to determine whether a machine can think or can perform human tasks [16].

There are several large sectors that have supported the development of AI [17]. This means providing data from e-commerce, business, and government to help improve machine learning solutions and algorithms, especially those related to system security. [18]. Moreover, China and Russia recognize the importance of AI for system security and overall competitiveness. As such, the Chinese government has recognized the importance of AI in relation to home security and seeks to become an authority in this area [19].

Each security offered has its own advantages. The thing that really needs to be considered is the existence of an internet connection and a power source at home or at the office [20]. Over time, this has received attention and development whereby if there is no electricity you can still use solar energy (solar panels) instead as well as a source of internet connection with the 5G network for access anywhere and anytime [21]. AI systems with sensors in IoT make homes think and make them live like humans [22].

From this explanation, this paper will build a security system with artificial intelligence on a Home Assistance system using a serverless where the surveillance center is located in the cloud where the device has been embedded with an RCNN algorithm to help quickly detect it. The process of running this application is that the devices stored at home are only IoT sensors and network devices so that they consume little electric power and minimal connections. The security that was built uses a surveillance camera (CCTV) that is integrated with AI which can always pay attention and capture existing movements and can see and recognize the objects it captures.

3.1 RCNN process in IoT

RCNN in its application in this paper is used to detect moving objects such as humans in an environment that is supported in the image. Basically, the RCNN model consists of three main components: a convolution layer, a proposal layer, and a classification layer. The convolution layer will extract the features from the image, then the proposal layer will select several regions from the image that are likely to contain objects. Furthermore, the classification layer will determine the labels and bounding boxes of each selected area.

To detect moving objects, RCNN can be trained using image datasets containing human images that have been labeled. Once trained, the model will be able to recognize features commonly associated with humans, such as the shape of the body, head and feet. The results of the training and testing calculations obtained are based on Table 2 below:

Table 2. Training process

From Table 2, the results of carrying out the data set in the training and testing process of a model which consists of 3 stages, namely the Convolutional layer, Pooling layer, and fully connected layer. The results of the data set using RCNN form a model and data record that is ready for further detection. After successfully detecting accuracy in reading moving objects, then determine the accuracy of the dataset that has been tested. Testing is carried out on the system by preparing the data set first. The dataset that has recorded 100 moving objects taking distances of 0.5 meters, 1 meter, 1.5 meters and 2 meters and the measurement results are shown in Table 3 as follows.

Table 3. Moving object detection accuracy test

However, as with other object detection, the success of moving object detection with RCNN will depend on many factors, such as the amount of training data, image quality, and the ability of the model to extract relevant features. In addition, human detection in complex or unexpected situations, such as darkness or partial closure of human objects, can be a challenge for the RCNN. The detection process has been carried out on various platforms. However, this paper focuses on the application of security. Where the algorithm process on RCNN is poured into hardware such as IoT. The RCNN algorithm carries out a training and testing process involving adaptive datasets to strengthen the method in IoT devices.

The success and running of the IoT system at this stage affects the performance of solar panels in a day and analyzes the needs and consumption that solar panels receive from batteries during charging. Weather factors can affect battery charging performance through solar panels. Furthermore, the performance of the Home Assistant application in securing whether it can capture objects that are visible on surveillance cameras and can be accessed via smartphone applications as carrying out IoT functions. Installation of solar panels in accordance with the circuit that we have previously designed. It is intended that the electric power sent and received to each device can be seen in Figure 4.

Figure 4 explaining the Solar Panels used are of the 100WP type as many as 2 pieces. Testing of solar panels on batteries and home assistant devices was carried out all day with sunny weather. The following is a table of power consumption and battery charging results from solar panels. This activity is seen from the MPPT Solar Charger Controller device based on the indicators that appear on the display.

4.png

Figure 4. The arrangement of the solar panel circuit

As in Figure 4, explaining the Solar Panels used are of the 100WP type as many as 2 pieces. Testing of solar panels on batteries and home assistant devices was carried out all day with sunny weather. The following is a table of power consumption and battery charging results from solar panels. This activity is seen from the MPPT Solar Charger Controller device based on the indicators that appear on the display shown in Table 4.

Table 4. Battery and solar panel activity

After the solar panel system is installed, then run the Home Assistant device and configure it so that it can be accessed through the system cloud. The device arrangement is shown in Figure 5.

5.png

Figure 5. Arrangement of cloud-based home assistant devices

Figure 5 shows that the IoT device has been successfully installed and ready to use. To access or open the Home Assistant application via a web browser and android application. Home Assistant runs on internet port 8123 so it's written as http://ipaddress:8123. The process is shown in Figure 6.

6.png

Figure 6. Home Assistant display after logging in

Figure 6 is the initial setting process of the SHA and then giving a signal to the client which can be operated on a mobile device in order to retrieve the data that has been stored. The display on mobile devices is as follows:

7.png

Figure 7. Home assistant mobile

Figure 7 is the result of setting the mobile Home Assistant in the RCNN implementation.

3.2 Device testing

This stage tests the Home Assistant application to capture objects that pass through the camera and record them. This is so that every object that is determined is visible and then stored in storage, so that home security can run. Objects detected can also record the detection time of the object. The application package for Home Assistant is MQTT and frigate.

8.png

Figure 8. Security home assistance system

In Figure 8, the Frigate package on the Home Assistant can detect moving objects. In addition, the Frigate feature in classifying objects other than humans can also be detected. In this case, the classification uses 4 objects, namely person, bicycle, paint, and car. So that each object that is detected is given a score value which is useful for accuracy in assessing objects.

9.png

Figure 9. Daylight testing

In Figure 9, it can be seen that the test was carried out during the day where the light was clear and the RCNN algorithm was able to capture and detect moving objects on the camera in real time. Then it was tested at night as shown in Figure 10.

10.png

Figure 10. Night testing

From Figure 10 it can be seen that the process is going well where there are no problems with the application during the day and at night because the RCNN system and algorithm can be implemented on various cameras. RCNN has been successfully integrated into IoT devices by utilizing cloud computing technology. One way to implement RCNN on IoT devices is to use an edge-cloud architecture, where computing processes are performed at the edge of the network (edge) and in the cloud. IoT devices can be used to collect image and video data from the surrounding environment. This data is then sent to a cloud server that integrates RCNN. RCNN is then used to detect the desired object in image or video data, such as the moving object in this paper, namely monitoring in home security.

Once a human object is detected, the detection results will be sent back to the IoT device to trigger certain actions, such as activating the security system, giving a warning or displaying information on the screen. An example of implementing RCNN on IoT devices is in an automatic door security system that can open doors automatically when human detection is carried out by RCNN. In this paper, RCNN can be used to provide intelligent services to IoT devices and enable IoT devices to make more accurate and intelligent decisions based on data collected from their surroundings. This can increase the security and convenience of IoT device users.