Surface Inspection And Novelty Detection

The surface defects are broadly divided into two types. One is local textural irregularities which is the main concern for most visual surface inspection applications. The other is global deviation of colour and/or texture, where local pattern or texture does not exhibit abnormalities. Automatic surface inspection for quality control has largely employed graylevel image processing techniques, for example in textile and wafer inspection. There are rising demands in the quality control industry for colour analysis to fulfil its vital role in visual inspection, e.g. in ceramic tile manufacturing. In this work, we first systematically review recent advances in surface inspection using computer vision and image processing techniques, particularly those based on texture analysis methods. The aim is to review the state-of-the-art techniques for the purposes of visual inspection and decision making schemes that are able to discriminate the features extracted from normal and defective regions. Then, we present our work on developing texture analysis techniques in application to the detection of abnormalities in colour texture surfaces, in particular ceramic tile surfaces on which patterns are regularly of a random nature. Those abnormalities we concerned can be divided into two categories: colour tonality defects and textural abnormalities.

Example defects on different types of surfaces: Steel, stone, textile, wood, and ceramic tiles.

Textural Defect Detection

The techniques used to inspect textural abnormalities are classified into four categories: statistical approaches, structural approaches, filter based methods, and model based approaches.

  • Statistical approaches
  1. Histogram properties
  2. Co-occurrence matrix
  3. Local binary pattern
  4. Other graylevel statistics
  5. Autocorrelation
  6. Registration-based
  • Structural approaches
  1. Primitive measurement
  2. Edge Features
  3. Skeleton representation
  4. Morphological operations
  • Filter-based approaches
  1. Spatial domain filtering
  2. Frequency domain analysis
  3. Joint spatial/spatial-frequency
  • Model-based approaches
  1. Fractal models
  2. Random field model
  3. Texem model

Tonality Inspection

In industrial quality inspection of colour textured surfaces, such as ceramic tiles or fabrics, it is also important to maintain consistent tonality during production. Tonality inspection can be carried out on both uniform pattern surfaces and randomly textured surfaces. Existing methods include:

  1. Statistical approaches
  2. Wavelet analysis based approaches
  3. Colour histogram based approaches

Colour Texture Analysis for Defect Detection

Due to rising demand and practice of colour texture analysis in application to visual inspection, those works that are dealing with colour texture analysis are discussed separately. Most colour texture analysis techniques are borrowed from methods designed for graylevel images, such as co-occurrence matrices and LBP. This extension of graylevel texture analysis techniques to deal with colour images usually takes one of the following forms:

  1. Processing each channel individually by directly applying graylevel based methods,
  2. Decomposing image into luminance and chromatic channels,
  3. Combining spatial interaction within each channel and interaction between spectral channels.

Other techniques independent of graylevel methods attempt to apply fully three dimensional models to analyse colour textures.

Classification and Novelty Detection

The primary goals of visual inspection are detection and classification. This involves choosing an appropriate decision making scheme which is usually referred to as pattern classification. Generally, this can be divided into supervised classification and unsupervised (or semi-supervised) classification.

Observations

  1. A significant amount of reported works are based on statistical and filter based approaches in visual inspection. This also could due to the fact that more texture analysis techniques fall in these categories than the others.
  2. Filter bank based methods have been very popular in textural defect detection. The filters can be manipulated and designed in all sorts of directions and scales to decompose textures in order to highlight defects. However, it is notable that recent researches suggest contextual analysis which directly based on local neighbourhoods without dedicated filtering is a promising alternative approach.
  3. Tonality defect detection, as a new emerging topic, should be viewed differently to textural defect detection. Its importance will be increasingly notable as more colour vision system will be used in practice.
  4. There are significant and increasing amount of work on colour texture analysis, however, limited work has so far been reported in visual inspection using colour texture analysis The majority of the existing methods decompose the colour image into separate channels and process them independently or with limited interactions.
  5. It is also notable that novelty detection is important in visual inspection where knowledge of defective patterns is usually incomplete and/or unavailable. However, when good prior knowledge is available, supervised classification scheme should be preferred as they often deliver better results.
  6. In order to understand the formation and nature of the defects, it is important to be able to accurately localise the defective regions rather than classifying the surface as a whole. This can provide possibilities of classifying the defects and further studies of the characteristics of the defects.
  7. Real time performance is highly desirable for industrial application.
  8. There is also a clear need of some standard datasets and well-defined experimental protocols in order to carry out fair comparative analysis.

Colour Tonality Inspection

Colour tonality refers to global chromatic appearance of a surface. Its variation from surface to surface may be understated, but becomes significant once the surfaces are placed together. A multidimensional histogramming method is developed to detect subtle tonality changes by incorporating local chromatic information into global chromatic characteristics. Principal component analysis (PCA) is used to reveal the nonlinear noise interference introduced by the imaging system. Vector directional processing and reference eigenspace feature selection are proposed to obtain salient colour tonality representation.

Flow chart of proposed method.

Random Textured Surface Inspection

Textural quality inspection involves the detection and localisation of various chromatic and textural imperfections. We suggest that although some textures have a random appearance, there are textural primitives that govern the global appearance. A novel two-layer generative model is proposed to represent an image or a family of images. In this model, random (or regular) texture images in the first layer are assumed to be generated from a collection of texture exemplars, or texems, in the second layer. A bottom-up texem generation process is proposed based on pixel neighbourhoods. This local contextual analysis using texems is applied to a large set of graylevel ceramic tile images, in which graylevel analysis is sufficient to detect textural abnormalities. Then, different schemes are explored to extend graylevel texems to colour images. This results in two different formulations and inference procedures for the texem model with different computational complexity. Spatial detection and localisation accuracy of the proposed methods is measured and compared using texture collage images. The texem model is also compared against the multiscale, multidirectional Gabor filter for defect detection. Both the colour tonality inspection and textural defect detection methods are implemented in novelty detection schemes to cope with the variety and unpredictable nature of defective samples.

Example results - same texture family

Defective samples
Localised defects

Example results - different textures

Defective samples
Localised defects

Example results - VisTex Collages

Original collage images
Escofet et al.’s method [Escofet'98]
Gray-level texems in RGB channels
Gray-level texems in PCA decorrelated RGB eigenchannels
Full color texem model

Funding

This project was funded by the European Commission under grant G1RD-CT- 2002-00783-MONOTONE.

Publications