Defect detection technologies for quality control in manufacturing

Computer vision is a subclass of the set of techniques under the umbrella of artificial intelligence. As its name suggests, it is the development of vision capabilities in machines, which can be considered as similar to the human perception of analyzing items through vision. Applications include object and defect detection, classification, transformation, image and video processing to solve real-world problems. One of our recent challenges is based on the application of this technology.

Two applications of defect detection in processes using computer vision techniques can be found in this article. These applications are intended to reduce the number of operators performing visual inspections to identify defects in manufactured products by making their job easier and faster.

1. Ceramic tile defect detection

Building materials companies, including ceramic tile manufacturers, need to perform visual inspections for defect detection of their products.

By implementing artificial vision algorithms, these manufacturers can save a lot of time and money. The accuracy and speed provided by artificial intelligence-based visual inspection cannot be matched by any other technique.

These inspections identify defects such as bumps, depressions, color shades or unwanted lines. In addition, the images from the vision system allow automation engineers to further fine-tune the detection of acceptable or unacceptable aesthetic defects.

It is time-consuming to formulate rules to consider all possible visual defects that could occur during an automated inspection:

• It is necessary to identify any type of defect, to determine its patterns, colors, textures and eventually its values, grades, usability, and selling prices.
• Not every imperfection caused by defects or irregularities in color is cause for rejection. Some are within the acceptable range.
• It is best if defects are detected before the clay dries, the material can be recycled, and waste is eliminated.

Just as a self-driving car incorporates different sensors (cameras, radar, ultrasonic sensors) to interpret reality through the processing of the images provided, by combining this information, it is possible to use different sensors and technologies to automate the quality control of ceramic tiles:

a. Imaging cameras and OpenVINO

To solve these inspection challenges, visual inspection algorithms combined with an in-line scanning camera and LED lighting can be implemented.

The hardware must be up to the task:

• High-performance CPU to enable tile inspection at a speed of up to 30,000 pieces/hour.
• Extensive I/O interface with support for multiple cameras at an image resolution of at least 3840 x 2160.
• Fast GbE interface for fast networking.

As for software, various tools and algorithms are available, such as:

• Intel's Open Visual Inference and Neural Network Optimization integrated deep learning toolkit, OpenVINO together with Myriad X VPUs for vision processing, which offers seamless software/hardware integration with the Intel Movidius engine, and includes model optimizer, the inference engine, pre-trained models, computer vision libraries, code samples and other tools, or.
• Cognex's VisionPro Deep Learning system, a proprietary deep learning solution, which identifies multiple aesthetic defects on the original plate based on a small sample of about 100 approved images. Inspecting a 200 x 32 cm material takes about two seconds.

b. X-rays and Detectron2

Another possibility is to use x-rays. Limitations in the resolution and sharpness of boundaries and interfaces in images reconstructed from data collected by X-ray computed tomography (CT) make it difficult to extract and segment features of interest. 

To circumvent this, the Faster RCNN module of the open source object detection algorithm Detectron2 developed by Facebook's artificial intelligence team is employed. This approach employs multiple convolutional neural networks configured in series for real-time object detection to identify defect positions (such as cracks) in two-dimensional slices of the 3D tomography dataset, capable of producing detection results with an accuracy of greater than 85%.

To train the detection algorithm, the user must manually incorporate several images for use in the training processes.

c. Low-cost spectral radiation

Another more novel possibility is the implementation of terahertz cameras, which emit a non-ionizing, innocuous radiation, and allow the ceramic pieces to pass through and measure their apparent density, allowing the inspection of the tiles, so that the same density can be maintained in all the pieces, achieving a high quality of production and a significant reduction in costs.

In addition, the application of these advanced optical sensors has shown that they can also be used to detect the presence and concentration of the inks applied in the decoration of the tiles, without the need to wait for the material to be fired. They can therefore be used to detect faults in the decoration of the tiles and optimize the printing process.

2. Visual inspection of fruit quality

Appearance and ripeness are key factors in the marketing of fruit. These factors can be determined by characteristics such as color, size, weight, texture and shape. In this way, growers or distributors can differentiate fruits and classify them into standard or low grades.

While years ago quality control methods based on sensory evaluation processes were applied, the most modern methods are based on spectroscopy, chemometrics and artificial vision.

Computer vision is used in this sector to inspect the quality of fruit, using images and applying classification and detection techniques.

Image processing and analysis is complex because fruit, unlike manufactured products, can have heterogeneous and complex color, size and shape, even if it was harvested on the same day and from the same tree. 

A good image greatly facilitates subsequent analysis. The quality of the images is therefore key and is closely related to the capture system used, the camera and the lighting:

• Cameras translate images into digital information by means of a greater or lesser number of photosensors (pixels) depending on the resolution of the camera. Matrix cameras are the most widespread and acquire a scene by means of a CCD integrated circuit. Color cameras can be built with sensors sensitive to the primary bands (RGB) and their combination generates color images.
• The illumination must be mounted depending on the application and the geometry of the object to be inspected, providing uniform radiation throughout the scene, and avoiding glare or shadows.  If the scene is not adequately illuminated, uncertainty and error in parameter extraction will increase.
• Polarizing filters are also used to avoid unwanted glare and reflections when this cannot be achieved directly with the illumination system. 

The analysis of the images allows to classify and determine the quality of the fruit:

• Deep learning methods such as SVM and decision trees are applied for classification.
• To determine the quality of a fruit, very specific information such as color and shape at a specific point is required.

To ensure analysis, each pixel of the fruit is treated separately using an instance segmentation technique such as the R-CNN mask. This mask takes an image as input and returns only the specific pixels needed for the inspection process.

Instance segmentation can be done using the "labelme" tool which provides the possibility to label specific pixels in the images. The trained model can determine the abnormal pixels that can lead to the detection of rotten, non-fresh or, in general, substandard quality fruit.

The implementation of this system will automate the quality control process with an accuracy of 85-90% depending on the quality of the data set.

The higher the number of imaging parameters used, the better the result will be. Up to 1500 image features have been reported, achieving success rates of over 95%.

The widespread use of hyperspectral, ultrasound and X-ray images for both internal and external quality assessment are technological trends that are also being implemented.

There are also apps for smartphones that use the camera together with a portable sensor (a spectrometer), to perform monitoring with conventional photographic images and then send the information to the cloud (such as fruit brix, acidity, firmness, color, size) where an algorithm compares the measurements with the thousands of samples in the database, returning clear information such as quality, freshness and ripeness.

AclaroMeter APP

At ennomotive, we are seeking to solve a new challenge about the inspection of leather for the interior of automobiles. Do you believe that the solutions used in the horticultural or ceramic sector could be used? What complete solution would you propose for leather inspection? What capture system, illumination and analysis algorithms would you use?

If you are interested in publishing, discovering or participating in new engineering challenges, please register at ennomotive.com.