Applied problems solved using machine vision systems.

Applied problems solved with the help of machine vision systems..

SEMIN Mikhail Sergeevich

APPLIED PROBLEMS SOLVED WITH THE HELP OF MACHINE VISION SYSTEMS  

Recently, the progress of microelectronics in the development of image recording devices has been obvious. The scope of their application has expanded: television cameras are already installed on parachutists' helmets, Formula 1 racing cars, and even in keyholes.

And this is serial technology!

At the same time, technologies for the production of special image receivers are developing just as intensively. Matrix optical receivers with a resolution of 4000×4000 elements and more have long been mastered, and recently matrices have appeared that combine high resolution — more than 1000×1000 elements — and a shooting speed that is orders of magnitude higher than the usual television 25 frames per second, as well as matrices that allow recording individual photons.

Along with the rapid development of image recording devices, mathematical support for image processing is being created, which allows us to talk about machine vision systems (MVS), the final “product” of which is not the image itself, but the parameters of the controlled process.

Thus, for any specific case, it is possible to create a machine vision system that greatly exceeds the capabilities of the human eye, and sometimes even a person as an image analyzer. At the same time, the use of special algorithms for processing the resulting image sometimes allows us to achieve completely unexpected solutions in terms of efficiency, seemingly in dead-end situations.

There is a common belief that demand creates supply. However, in times of technological breakthroughs, the supply of new solutions often outstrips demand for the simple reason that potential consumers of these innovations either do not know about them or, due to the inertia of thinking, “cannot guess” to apply them in their practice.

STZ in thermal imaging and thermal measurement technologies
Developer NPP “TERMOTEKH”

To obtain temperature fields of various objects with a temperature of 700 °C and above, the hardware and software complex “TERMO” has been developed, which performs the following main functions:

• calculation of temperature, radiation coefficient, integral density of radiation flux at each visible point of the object;
• presentation of the temperature picture of the object in real time according to the following parameters: radiation coefficient of the object, transmission of the medium, reflection of the mirror, temperature of surrounding objects;
• signal about temperature going beyond the specified limits in the specified areas of the thermal image;
• recording a series of frames to disk with the possibility of subsequent playback and processing.

Thermal image analysis is performed in the following forms:

• color thermogram (several palettes, dynamic palette boundaries);
• distribution profiles at specified points along specified directions;
• temperature distribution at selected points over time;
• isotherms;
• statistics for the selected region (maximum, minimum, average, standard deviation, etc.);
• histogram of values ​​in the region.

All analysis functions are available in the “live” image mode.

Fig. 1. Examples of using a thermal imager (thermovisor) to monitor the parameters of the technological process of burning by-products in oil refining (the presence of a flare, the volume of combusted gas and the combustion temperature are monitored)

X-ray flaw detection
Developed by the Russian Research Center “Kurchatov Institute”

The Department for Development of Detection Systems has created and introduced into production an X-ray diagnostic complex for monitoring welds and wall thickness of steel pipes. The use of a digital X-ray television system allows for monitoring a weld length of about 4 meters in just 8 minutes and monitoring the wall thickness of pipes at a speed of up to 3 m/min. Using traditional film radiography technology, these processes take several hours.

Gamma-image recorder
Developer: RRC «Kurchatov Institute»

This device is designed to obtain gamma images (images in gamma rays), which can be used to detect radioactive sources from a safe distance and determine the distribution of radioactivity.

The device (Fig. 2) consists of a collimator that forms an image; a scintillator that emits light when it absorbs gamma quanta; a light amplifier based on a microchannel plate (MCP) and a digital CCD camera.

Frames of defective areas can be saved for subsequent analysis.

Fig. 2. Schematic diagram of the device for obtaining gamma images

Photo 1a
Image of the gamma source Am-241, obtained from a distance of 5 m in 100 seconds. The source gives an increase in the dose rate of the order of the natural background at the observation site

Photo 1b
Overlay of intensity distribution in a gamma source (blue palette) on a video image of a contaminated object

Fig. 3. Another way to present the results of measuring the distribution of radioactivity of a radioactive spot on the floor of a room.
On the right is the distribution of radioactivity in the spot,
on the left is the overlay of the obtained distribution
(in the form of isolines of equal intensity) on the video image of the room

Visualization of hydroaerodynamic flows
Developer of the aerodynamic laboratory of the Central Research Institute named after Academician A.N. Krylov

Due to its clarity and high information content, visualization of flows in hydroaerodynamic experiments is finding increasing application, which allows for prompt and high-quality solutions to issues of improving ship designs and obtaining reliable information about complex processes occurring in hydroaerodynamic flows.

The hydrodynamic flow visualization stand complex includes: a hydrodynamic tube with a transparent working section (cross-section — 15×15 cm, length — 60 cm); a set of models of mixing chambers and technological devices of water supply stations; a set of models of ship premises with transparent walls for hydraulic modeling of convective flows at the initial stage of a fire and ventilation processes at a scale of 1:10.

The hydrodynamic flow visualization stand is equipped with modern recording film, photo, video equipment, allowing for recording and subsequent processing of images with high accuracy. The equipment includes: an IAB-451 optical device, 35 and 60 mm film and photo cameras, and a video complex.

The stand conducts fundamental and applied research aimed at solving practical problems of improving ship designs.

The main areas of fundamental research:

• separated flows and vortex formation in the boundary layer;
• spatial separated flows;
• interaction of the jet with the drift flow;
• vortex formation in an unsteady flow;
• development of convective flows in a limited space.

Fig. 4. Fundamental research (the flow is directed from right to left). The interaction of the flow with a pair of rotating cylinders:
a) the direction of rotation of the cylinders is the same;
b) the cylinders rotate in opposite directions,
creating a jet going across the flow

Applied research is aimed at improving ship designs:

• reducing smoke emissions from transport vessels;
• vortex formation on protruding elements of the hull of a submarine, causes of non-uniform flow in the propeller disk;
• the influence of the operating modes of the propeller, angles of attack and drift of the submarine on the structure of the flow around the aft end;
• vortex formation above the flight deck of an aircraft carrier during pitching;
• vortex formation on the grates of heat exchangers;
• studies of convective flows at the initial stages of a fire in ship spaces in order to optimize the installation locations of fire alarm sensors.

Fig. 5. Applied research:
a), b) – smoke generation on the bridge of a transport vessel with a low chimney;

c), d) – stages of vortex formation over the flight deck of an aircraft carrier during pitching

A flaw detector projectile for monitoring the internal surfaces of smoke and ventilation pipes of industrial enterprises
Developer: TsIEKS EMERCOM of the Russian Federation

The task of monitoring the condition of smoke pipes without their preliminary cooling is very relevant, especially where continuous technological processes are used.

To examine the condition of the lining of the internal surface of smoke pipes, the developer has created a flaw detector projectile (photo 2).

Photo 2. Flaw detector projectile

The design of this projectile with installed ring aerodynamic stabilizers ensures its stable position during descent/ascent and, together with a set of auxiliary lifting and transport mechanisms, allows work to be carried out without stopping the technological process or with partial cooling of the pipe to a temperature of 200 °C. The lower part of the body contains an image recording unit (developed by NPK VIDEOSCAN), which includes 12 television cameras installed in a ring and an on-board computing system that provides: when the flaw detector is lowered, a series of ring images of the inner surface of the pipe are obtained and recorded on an HDD, and after the flaw detector is removed from the pipe, the obtained data are read via a standard communication channel to an external computer for analysis and archiving.

Photo 3. Fragment of the image of the inner wall of the chimney

Photo 3 shows fragments of images of the inner wall of the chimney near the mouth. Cracks and other defects that formed during the operation of the pipe are clearly visible.

Automatic selection and reading of bar codes
Developed by IIT and Intermec Corp.

The device is intended for use in automated warehouse control systems, as well as for automatic sorting and registration of goods, parcels, etc.

An image of an object with an applied bar code is recorded by a video camera and entered into a personal computer. During the analysis of the obtained digital image, the available bar codes are detected and read. In this case, all bar codes in the camera's field of view are detected and read, regardless of their size, position, orientation and geometric properties of the surface to which they are applied; partial contamination and erasure of bar codes, as well as cellophane coating of the object over the codes, are allowed; bar codes are recognized on any complex structured background.

Photo 4. The frames show the detected and read barcodes

Automatic selection and filtering of particle traces
Developer IIT and ICT RAS

The system enables automation of the processing and analysis of 12-bit digital images of particle tracks obtained by high-speed photography, and ensures reliable particle track extraction with subpixel accuracy, as well as their filtering according to the required parameters (particle track length, maximum intensity along the track length, transverse size of the particle track taking into account the track angle, angle of inclination of the particle track relative to the image boundaries).

Photo 5. Particle tracks are extracted, satisfying the given parameters

Automatic human face detection and feature tracking
Developed by IIT and SPIRIT Corp.

The task is to automatically detect a face and its characteristic large elements (eyes, mouth, eyebrows, nose), as well as track these objects and display them in real time on a personal computer such as IBM PC. A web camera is used as an input device, providing an input data stream in RGB format (24 bit) at a rate of at least 10 frames per second. Input image size: 320×240 pixels. The user's face is located full-face to the camera and is not covered by any other objects. Face rotations and tilts within 10 degrees are allowed. Normal office illumination. The results of eyebrow, nose and mouth boundary detection are approximated by splines at several characteristic points. The results of eye detection are characterized by the “eye open/closed” flag, as well as the centers and radii of the corresponding circles (Fig. 6).

Fig. 6. Automatic detection of a face and its characteristic elements

The selected information about the face and its features is shown on top of the original images, presented in the form of vector graphic elements for transmission over networks or further visualization of virtual characters by means of computer graphics in real time.

Detection of obstacles on the road in front of a moving vehicle
Developer GosNIIAS and NC «Modul»

To solve the problem of automatic vehicle control, a system for detecting obstacles on high-speed highways has been developed. Its operation is based on the analysis of digital stereo images obtained using two CCD video cameras. Obstacles are detected within the vehicle's own lane in real time. The system ensures stable detection of various types of obstacles at a distance of up to 100 m.

Photo 6. Results of identifying own traffic lane and obstacles observed on the road in different lighting conditions and various traffic situations

Desktop forensic complex GENETIKA-02
Developer NIIIN MNPO «SPEKTR»

The complex is designed for in-depth examination of documents, banknotes, securities, etc.

Composition of the complex: forensic unit; remote infrared video magnifier; color video monitor, as well as an optimal set of tools for operational control of documents and providing communication with a PC for processing and archiving information.

It is possible to retrofit with additional functional units and blocks.

Photo 7. a), b) identification of additions and corrections;

c), d) visualization of the image under dye filling.

Main functions of the complex:

• quality control of paper, typographic and handwritten design of watermarks, luminescent fibers, security marks of luminescent, metameric and other dyes — sheets of various formats, including large ones;
• detection of erasures, erasures, forgeries of text, seals, traces of glue and pressure of writing instruments;
• detection of traces of exposure to etching compounds with visualization of destroyed parts;
• visualization of design details filled with dyes and details on the back of the sheet and the next sheet.

Below are some examples of the complex's operation (IR radiation registration band in the range of 720 — 1000 nm).

Banknote authenticity check:

Photo 8. Image of a Russian banknote in reflected radiation:
a) in the visible range;

b) in IR radiation

Uncooled small-sized thermal imagers on pyroelectric and microbolometric matrices “TN-4604 MP” and TN-4604 MB”
Developer NIIIN MNPO “SPEKTR”

Thermal imagers are designed for effective surveillance of protected and controlled objects and zones at any time of the day in various, including difficult weather conditions. Provide visualization of heat-emitting objects, observation of heat exchange dynamics. Can be used to solve special problems. Their main areas of application:

• ensuring man-made safety:
• thermal imaging diagnostics of energy-saturated objects, buildings, structures and vehicles;
• search for leaks and ruptures in oil pipelines and heating mains, as well as from the air for hidden fires and underground peat fires;
• identification of fire- and explosion-hazardous places of overheating in power electrical equipment, etc.;
• search and inspection activities:
• search, detection and covert surveillance of the movement and actions of remote objects;
• aerial search for people in the open sea, mountains, deserts and other hard-to-reach places;
• an effective mobile means for round-the-clock external security of buildings, structures and extended objects;
• detection of hidden anomalies (voids, foreign inclusions).

Photo 9. Residual heating (on completely cooled areas) and reflection from dry asphalt of wheel hubs and muffler 20 minutes after stopping the car (delta T = 5 °C)

Multichannel recording of ultra-fast processes
Developer: JSC “NPK Videoscan”

The solution to the problem is shown using the example of recording images of a bullet ricochet when firing from a Kalashnikov assault rifle at an acute angle to the surface of the water (photo 10).

Photo 10. Registration of the image of the ultrafast process

The images were recorded by three electron-optical cameras of the Nanogate Frame-4 system [1], installed along the bullet's flight trajectory. The exposure of each camera is 300 ns. The interval between images is 200 μs.

The first phase: the bullet flies up to the surface of the water, and the strong pitch of the bullet is caused by the loss of stabilization when passing the polyurethane wall of the cuvette.

The second phase: the bullet enters the water. It is interesting to note that the disturbance of the water is ahead of the bullet's flight.

The third phase: the bullet leaves the water.

The use of systems for recording ultra-fast processes is relevant in various fields of science and technology.

The use of STZ in industrial production (using the medical industry as an example)
Developer: JSC «NPK Videoscan»

For the prevention and treatment of attacks in patients with bronchial asthma, the company «Pulmomed» has created a prolonged-release drug «Saltos», the most important feature of which is its dosed release. To do this, a hole of a strictly defined size is burned in the insoluble shell of each tablet with a laser beam. Since any deviation from the specified parameter can be fatal for the patient, the Pulmomed company decided to ensure 100% fully automated control of the size of the holes in the shells of the produced tablets by using the STZ means.

Photo 11. The image of the hole in the shell of the tablets was obtained by the VSСТТ-075-2001 video system, equipped with a microscopic lens

Despite the fact that the average brightness of the image in the area of ​​the holes is almost no different from the average brightness values ​​in the rest of the image (photo 11), NPK Videoscan developed algorithms that solve the problems of identifying the area of ​​the tablet surface containing the hole and determining the area of ​​the hole at the speed of the production line — up to 2 tablets per second. Examples of this algorithm in action are shown in Photo 12.

Photo 12. Examples of hole area allocation

The results of hole area measurement are used in the industrial line to regulate the laser installation radiation power and product rejection.

 Literature

1. Krutik M.I. Multichannel software-controlled electron-optical complexes for high-speed recording of a series of images of a fast-flowing process.//Special equipment, 2002, No. 1, p. 36 – 41.