Design of Automated Visual Inspection System for Beverage Industry Production Line

Design of Automated Visual Inspection System for Beverage Industry Production Line

Subhransu Padhee Durgesh Nandan

Department of Electrical and Electronics Engineering, Aditya Engineering College, Surampalem 533437, Andhra Pradesh, India

Accendere Knowledge Management Services Pvt. Ltd., CL Educate Ltd., New Delhi 110044, India

Corresponding Author Email: 
durgeshnandano51@gmail.com
Page: 
461-466
|
DOI: 
https://doi.org/10.18280/ts.380225
Received: 
12 November 2019
|
Revised: 
3 February 2021
|
Accepted: 
20 February 2021
|
Available online: 
30 April 2021
| Citation

© 2021 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

This paper provides an overall design and implementation perspective of a laboratory-scale automated visual inspection system for the beverage industry's production line. A case study has been undertaken where the image processing algorithm inspects the beverage bottle for any defects. Different defects such as improper labeling and improper liquid level can be detected using the image processing algorithm. A laboratory prototype of the conveyor belt has been built, and a prototype filling plant has been established to verify the simulation results.

Keywords: 

automated visual inspection system, coverage industry production line, visual inspection, image processing

1. Introduction

Quality inspection of the finished product is a significant challenge in the production process of an industrial manufacturing plant. A multistaged inspection comprising of different parts, sub-parts, sub-assemblies, and final products is required to improve the finished product’s quality and reliability in a mass production facility. Visual inspection of the finished product quality is the industry’s norm because using visual inspection, a functional defect or a cosmetic defect can be ascertained easily. A human-based visual inspection system is slow, erratic, expensive, and less accurate. Therefore, the modern manufacturing industry focuses on automated visual inspection systems using high-end imaging devices and high-power processing platforms [1].

A review of the industrial vision system can be found in the studies [2-8]. Figure 1 presents the flow of product in an assembly line where the raw material is converted to a finished product in different steps, and after the finished product is available, a set of the inspection system is initiated to inspect the quality of the finished product. Once the quality of the finished product is ascertained, then it is packed and dispatched to the market.

Visual inspection system can have four types of inspection, such as:

  • Inspection of dimensional quality
  • Inspection of surface quality
  • Inspection of correct assembling
  • Inspection of accurate or correct operation

Designing an automated visual inspection system is complicated as it requires multiple high-resolution cameras with high-speed capturing and processing facilities. Dedicated illumination and optical system play a vital role in the automated visual inspection system. Multiple vision platforms are used for the inspection of the different features of the finished product. Every vision platform comprises multiple camera and illumination sources used for image acquisition and image processing. For image acquisition, geometric camera calibration is one of the essential steps. Camera parameters include intrinsic, extrinsic, and distortion coefficients.

Figure 1. Flow of product in an assembly line

Pinhole camera model and lens distortion model of camera calibration is widely used [9, 10]. Machine vision-based can-end inspection system has been discussed by Chen et al. [11] and vision inspection system for beer bottle has been discussed by Duan et al. [12]. Feature extraction for fill level and bottle cap inspection in the bottling machine has been proposed by Yazdi et al. [13]. Vision-based liquid particle inspection of pharmaceutical injection has been proposed by Zhang et al. [14]. A vision-based system for empty bottle inspection has been reported by Huang et al. [15]. Saliency detection and template matching techniques have been used for visual inspection of the bottom of the glass bottle [16]. Morphological analysis and machine learning have been used to detect liquid impurities [17]. The empty bottle detection technique has been discussed by Duan et al. [18]. Vision-based inspection of olive oil bottling plant has been discussed by Abdelhedi et al. [19]. A programmable logic controller (PLC) has been used to control the conveyor belt of a bottling plant [20]. A detailed discussion on industrial vision system with multiple cameras, host PC, PLC, manipulators and cameras are discussed by Martínez et al. [21].

A comprehensive survey of modularity of PLC and PLC based software for automated production system has been outlined in the study [22]. Recently deep learning based technique is used to classify bottles using machine vision approach [23, 24].

This paper provides a tutorial aspect of the design of an automated visual inspection system. The dataset used for simulation is provided by Solomon and Breckon [25]. The dataset comprises 141 images of the bottle leaving a hypothetical bottling facility. A high-speed camera has captured the images at a near-constant lighting condition. The motion blur of the image has been eliminated. A visual inspection system has been developed in software where the algorithm detects the faulty bottle. To validate the simulation result, a laboratory prototype vision inspection model has been developed and real-time image acquisition and processing has been carried out.

This paper is organized as follows. Section 2 provides the problem formulation for an automated visual inspection system. Section 3 provides the simulation results. Section 4 describes the experimental setup and results obtained from experimentation. Section V concludes the paper.

2. Problem Formulation

This paper considers a hypothetical bottling facility and captures the image of Coca-Cola bottle with the help of a high-speed camera when the bottle is leaving the conveyor belt. There is a near constant lighting condition. There are all total 141 captured images [25].

There can be many kinds of defects in the Coca-Cola bottle. Some of the faults are as follows:

  • Bottle under-filled or not filled at all
  • Bottle over-filled
  • Bottle has label missing
  • Bottle has label but label printing has failed
  • Bottle label is not straight
  • Bottle cap is missing
  • Bottle is deformed

The objectives of the paper are:

  1. To develop an image processing algorithm which can detect the above mentioned faults.
  2. To develop a hardware prototype which can emulate the working of real-time machine vision system.
3. Simulation Results

Figure 2 shows the original image database used in this study. The image database comprises of multiple images which indicate different faults in the bottle (discussed in Section 2).

Figure 2. Image database [25] (a) Underfilled, (b) Overfilled, (c) No label, (d) Faulty label, (e) Label not straight, (f) Missing Cap, (g) Deformed bottle, (h) Ideal bottle

For image processing operation, first the image is converted to greyscale and binary image. The image contrast has been enhanced using the following equation:

$g(x, y)=\left\{\begin{array}{cc}\frac{11}{27}(f(x, y)-120)+200 & 120 \leq f(x, y) \leq 255 \\ \frac{37}{18}(f(x, y)-30)+15 & 30 \leq f(x, y) \leq 120 \\ \frac{1}{2} f(x, y) & 0 \leq f(x, y) \leq 30\end{array}\right.$  (1)

where, f(x, y) is the original image and g(x, y) is the output image which is a grayscale image. The coordinates of a particular bottle in the image are computed and the compared with the coordinates of the other bottles in the image. A threshold is computed and the coordinate of the damaged bottle is computed where region of interest (ROI) is found. The image processing steps followed are simple, accurate and takes less computational time.

Figure 3 presents the output of image processing algorithm where the algorithm detects the faults. In Figure 3(a) shows the detection of overfilled bottle as well as the bottle which has improper labeling. Figure 3(b) presents the detection of a void space between two consecutive bottles. Figure 3(c) detects multiple faults such as bottle with no cap and underfilled or empty bottle with cap. Figure 3(d) detects the overfilled bottle. Figure 3(e) and Figure 3(f) detects empty or underfilled bottle and bottle with no labelling.

Figure 3. Output of vision system with fault detection at each case

Different performance metrices have been calculated for the simulation program and these are summarized in Table 1. From the performance metrices it can be ascertained that the sensitivity and precision of the image processing algorithm is high and false discovery rate is low.

Table 1. Performance metrices

Performance Index

Formula

Results

Precision

$\frac{T P}{T P+F P}$

0.9884

False discovery rate (FDR)

$\frac{F P}{T P+F P}$

0.12

Sensitivity or Recall

$\frac{T P}{T P+F N}$

1

Fallout (FPR)

$\frac{F P}{T N+F P}$

0.018

Specificity

$\frac{T N}{T N+F P}$

0.918

Accuracy

$\frac{T P+T N}{T P+F N+T N+F P}$

0.98

Prevalence

$\frac{T P+F P}{T P+F N+T N+F P}$

0.18

where, TP stands for true positive, TN stands for true negative, FP stands for false positive and FN stands for false negative.

4. Experimental Results

To demonstrate the real-time vision based inspection system, a laboratory prototype has been developed. Figure 4 presents the block diagram of vision based inspection of beverage system. A conveyor belt is used to emulate the conveyor belt of bottling plant. Geared DC motor is used in the conveyor. Table 2 presents the technical specification of experimental setup. Microsoft lifecam is used for capturing images at 30 frames per seconds. The camera is place above a tripod at a particular position. Proper lighting condition has been assured for image acquisition. To remove any background subtraction issue, a uniform background has been used. In the hardware setup, only one camera is used to capture the front view of the object whereas multiple cameras can be used to capture different views of the object.

Figure 4. Block diagram of vision based inspection of beverage system

Figure 5 presents the flow diagram of experimental setup developed. If the photoelectic switch detects the presence of any object, then the camera captures the image of the object and performs the image processing operation. If the object passes the machine vision test then it is deemed as fit for packaging or else it is deemed unfit for the packaging purpose.

Figure 5. Flow chart of experimental setup

Table 2. Technical specification of experimental setup

Part name

Manufacturer

Specification

Camera

Microsoft LifeCam

30 fps

Geared DC Motor

Pittman

Supply: 24 V DC

Ratio: 65.5:1

CPR: 500

Photoelectric switch (Diffused type)

YOS-18-P-U-X-3

10 V to 30 V DC

Data Acquisition System

NI USB 6008

National Instruments

8 channel 12 bit ADC

Maximum sampling rate of single channel: 10 kS/sec

4.1 Camera calibration

The camera calibration algorithm calculates the camera matrix using extrinsic parameters (rigid transformation from 3-D world coordinate to 3-D camera coordinate system) and intrinsic parameters (projective transformation from 3-D camera coordinate to 2-D image coordinate). The extrinsic parameter comprises rotation and translation, whereas the intrinsic parameter consists of focal length and optical center. Camera calibration is carried out using the in-built tool of LabVIEW.

4.2 Sensor and data acquisition system

Diffused type photoelectric switch acts as a proximity sensor. The photoelectric sensor comprises an Emitter for emitting light and a Receiver for receiving light. When emitted light is interrupted or reflected by the sensing object, it changes the light that arrives at the Receiver. The Receiver detects this change and converts it to an electrical output. The output of the photoelectric switch is provided to the data acquisition card (in this case, NI USB- 6008). The data acquisition system’s output and the camera are interfaced to the personal computer where Laboratory Virtual Instrument Engineering Workbench (NI-LabVIEW) program is developed. Figure 6 provides the photograph of the experimental setup.

Figure 6. Experimental setup for vision based inspection of beverage system

Figure 7. LabVIEW front panel for visual inspection system (Unable to detect the bottle cap)

Figure 8. LabVIEW front panel for vision inspection system (Perfect detection of bottle cap)

Figure 9. LabVIEW time computation

The LabVIEW program provides the vision-based inspection system. Figure 7 provide a case where the image processing algorithm is not able to detect the bottle cap whereas in Figure 8 the image processing algorithm is able to detect the bottle cap. Figure 9 provides the estimated time required by the NI vision assistant to perform the image processing task.

5. Conclusions

This paper provides a design procedure to develop an automated visual inspection system to detect faults in the finished products in the beverage industry. This paper considers a case study of the Coca-Cola bottling plant where image databases are provided, and the image processing algorithm detects the different types of faults in the image. In addition to the software-based vision system, this paper includes design details of a laboratory prototype model of a vision system where a conveyor belt is used to replicate the bottling plant. A photoelectric sensor senses the object’s proximity, and the camera acquires and processes the image to detect any fault.

  References

[1] Chin, R.T., Harlow, C.A. (1982). Automated visual inspection: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, (6): 557-573. https://doi.org/10.1109/TPAMI.1982.4767309

[2] Malamas, E.N., Petrakis, E.G., Zervakis, M., Petit, L., Legat, J.D. (2003). A survey on industrial vision systems, applications and tools. Image and Vision Computing, 21(2): 171-188. https://doi.org/10.1016/S0262-8856(02)00152-X

[3] Newman, T.S., Jain, A.K. (1995). A survey of automated visual inspection. Computer Vision and Image Understanding, 61(2): 231-262. https://doi.org/10.1006/cviu.1995.1017

[4] Golnabi, H., Asadpour, A. (2007). Design and application of industrial machine vision systems. Robotics and Computer-Integrated Manufacturing, 23(6): 630-637. https://doi.org/10.1016/j.rcim.2007.02.005

[5] Du, C.J., Sun, D.W. (2004). Recent developments in the applications of image processing techniques for food quality evaluation. Trends in Food Science & Technology, 15(5): 230-249. https://doi.org/10.1016/j.tifs.2003.10.006

[6] Smith C., Adendorff, K. (2011). Advantages and limitations of an automated visual inspection system. The South African Journal of Industrial Engineering, 5(1). http://hdl.handle.net/2263/20486

[7] Morishita, I., Okumura, M. (1983). Automated visual inspection systems for industrial applications. Measurement, 1(2): 59-67. https://doi.org/10.1016/S0263-2241(83)80012-8

[8] Shirmohammadi, S., Ferrero, A. (2014). Camera as the instrument: The rising trend of vision based measurement. IEEE Instrumentation & Measurement Magazine, 17(3): 41-47. https://doi.org/10.1109/MIM.2014.6825388

[9] Heikkila, J., Silvén, O. (1997). A four-step camera calibration procedure with implicit image correction. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106-1112. https://doi.org/10.1109/CVPR.1997.609468

[10] D’Emilia, G., Di Gasbarro, D. (2017). Review of techniques for 2D camera calibration suitable for industrial vision systems. Journal of Physics: Conference Series, 841(1): 012030. https://doi.org/10.1088/1742-6596/841/1/012030

[11] Chen, T., Wang, Y., Xiao, C., Wu, Q.J. (2016). A machine vision apparatus and method for can-end inspection. IEEE Transactions on Instrumentation and Measurement, 65(9): 2055-2066. https://doi.org/10.1109/TIM.2016.2566442

[12] Duan, F., Wang, Y.N., Liu, H.J., Li, Y.G. (2007). A machine vision inspector for beer bottle. Engineering Applications of Artificial Intelligence, 20(7): 1013-1021. https://doi.org/10.1016/j.engappai.2006.12.008

[13] Yazdi, L., Prabuwono, A.S., Golkar, E. (2011). Feature extraction algorithm for fill level and cap inspection in bottling machine. In 2011 International Conference on Pattern Analysis and Intelligence Robotics, Kuala Lumpur, Malaysia, pp. 47-52. https://doi.org/10.1109/ICPAIR.2011.5976910

[14] Zhang, H., Li, X., Zhong, H., Yang, Y., Wu, Q.J., Ge, J., Wang, Y. (2018). Automated machine vision system for liquid particle inspection of pharmaceutical injection. IEEE Transactions on Instrumentation and Measurement, 67(6): 1278-1297. https://doi.org/10.1109/TIM.2018.2800258

[15] Huang, B., Ma, S., Wang, P., Wang, H., Yang, J., Guo, X., Zhang, W., Wang, H. (2018). Research and implementation of machine vision technologies for empty bottle inspection systems. Engineering Science and Technology, an International Journal, 21(1): 159-169. https://doi.org/10.1016/j.jestch.2018.01.004

[16] Zhou, X., Wang, Y., Xiao, C., Zhu, Q., Lu, X., Zhang, H., Ge, J., Zhao, H. (2019). Automated visual inspection of glass bottle bottom with saliency detection and template matching. IEEE Transactions on Instrumentation and Measurement, 68(11): 4253-4267. https://doi.org/10.1109/TIM.2018.2886977

[17] Li, X., Qiao, T., Pang, Y., Zhang, H., Yan, G. (2018). A new machine vision real-time detection system for liquid impurities based on dynamic morphological characteristic analysis and machine learning. Measurement, 124: 130-137. https://doi.org/10.1016/j.measurement.2018.04.015

[18] Duan, F., Wang, Y.N., Liu, H.J., Tan, W. (2004). Empty bottle inspector based on machine vision. In Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No. 04EX826), Shanghai, China, pp. 3845-3850. https://doi.org/10.1109/ICMLC.2004.1380507

[19] Abdelhedi, S., Taouil, K., Hadjkacem, B. (2012). Design of automatic vision-based inspection system for monitoring in an olive oil bottling line. International Journal of Computer Applications, 51(21).

[20] Zhang, T., Dong, F., Yuan, H. (2012). Application of PLC for arranging bottle in beer filling production line. In 2012 24th Chinese Control and Decision Conference (CCDC), Taiyuan, China, pp. 1215-1220. https://doi.org/10.1109/CCDC.2012.6242996

[21] Martínez, S.S., Ortega, J.G., García, J.G., García, A.S., Estévez, E.E. (2013). An industrial vision system for surface quality inspection of transparent parts. The International Journal of Advanced Manufacturing Technology, 68(5-8): 1123-1136.

[22] Vogel-Heuser, B., Fischer, J., Feldmann, S., Ulewicz, S., Rösch, S. (2017). Modularity and architecture of PLC-based software for automated production systems: an analysis in industrial companies. Journal of Systems and Software, 131: 35-62. https://doi.org/10.1016/j.jss.2017.05.051

[23] Wang, J., Fu, P., Gao, R.X. (2019). Machine vision intelligence for product defect inspection based on deep learning and Hough transform. Journal of Manufacturing Systems, 51: 52-60. https://doi.org/10.1016/j.jmsy.2019.03.002

[24] Benbarrad, T., Salhaoui, M., Kenitar, S.B., Arioua, M. (2021). Intelligent machine vision model for defective product inspection based on machine learning. Journal of Sensor and Actuator Networks, 10(1): 7. https://doi.org/10.3390/jsan10010007

[25] Solomon, C., Breckon, T. (2011). Fundamentals of Digital Image Processing: A Practical Approach with Examples in Matlab. John Wiley & Sons.