TARASOV Viktor Vasilievich, Doctor of Technical Sciences
YAKUSHENKOV Yuri Grigorievich, Doctor of Technical Sciences, Professor
SOME WAYS TO IMPROVE THERMAL IMAGING SYSTEMS
The ever-increasing need for new optoelectronic systems used in a wide variety of fields of science and technology provides their developers with ample opportunities for creativity. This also applies to infrared (IR) technology and its important component – thermal imaging systems (TIS). The recent emergence of new technologies and the creation of modern components and modules based on them often allows developers to take a new approach to designing these systems and, as a result, achieve fundamentally new high-quality results.
The main task that TIS developers solve is the visualization of images created by optical systems in the IR region of the electromagnetic radiation spectrum, i.e. at wavelengths greater than 0.76 μm.
In the late 1990s, these systems entered a new important stage of their development, which was caused by the creation of matrix (two-dimensional) multi-element radiation receivers (MER), which made it possible to implement the “looking” mode of operation of these systems, i.e. to abandon the optical-mechanical scanning devices. In this case, it turned out to be especially important to create small-sized uncooled infrared matrix MRE of a sufficiently large format, as well as the corresponding CCD and CMOS circuits for reading and primary signal processing.
The transition from systems with optical-mechanical scanning to systems of the “looking” type (with electronic scanning) provided tangible advantages, but at the same time led to the need to take into account a number of new factors and solve previously unencountered tasks and even problems associated, for example, with a sharp increase in the volume of information that needs to be processed in real time, with the peculiarities of high-resolution spatial and temporal sampling, and a number of other problems.
The appearance of false low-frequency components in the spectrum of the sampled signal (aliasing) due to the discrete structure of the MPI in infrared systems (IRS) of the viewing type is a fundamental drawback of such systems, since it leads to distortion of the image spectrum, i.e. the video signal, and then the signal on the display system screen. An effective means of combating aliasing is microscanning, which is carried out by shifting the image (frame) relative to the MPI by some part of the period of the MPI element location and subsequent sampling of this image (subframe or subimage). Then the obtained digital images (subframes) are combined, forming one full frame. In known systems, a shift by half the period of the MPI element location for each coordinate is most often used, i.e. four-position microscanning [1, 2]. In this case, a double sampling frequency is achieved, compared to the frequency determined by the period of the MPI element location, and the spatial-frequency characteristic of the entire system remains the same, i.e. corresponds to the geometric optical parameters of the microscanning device. The overlapping of spectra and the accompanying distortions are noticeably reduced, i.e. high spatial frequencies in the image spectrum (or the small details corresponding to them) are successfully resolved. Microscanning allows for lowering the requirements for the microscanning device, i.e. for using a smaller format microscanning device with a larger pixel period or a lens with a shorter focal length in the system, which is important in connection with the desire to simplify the design and reduce the cost of both the microscanning device and the IR scanner as a whole.
Microscanning increases the geometric resolution of the microscanning device proportionally to the number of positions (offsets) occupied by the image relative to the microscanning device raster, while maintaining the microscanning device format and the angular field value the same as in a system without microscanning. Therefore, it is very effective in systems that use a microscanning device with a relatively small fill factor.
The trajectory of shifts during microscanning can be very different. For example, a four-position trajectory is quite common in practice: the initial position, to the right by 1/2 pixel of the microscanning range, down by 1/2 pixel, to the left by 1/2 pixel, up by 1/2 pixel, i.e. to the initial position.
The number of MPI positions involved in one cycle (period) of microscanning is often determined by the need to have a number of individual image samples equal to the format of the display system. Thus, with the help of microscanning, it is possible to match the formats of the MPI and the display system.
Although microscanning increases the sampling frequency and, therefore, allows for an increase in spatial resolution, it simultaneously leads to a decrease in the accumulation time tи.
Microscanning increases the sampling frequency of a static image. But it cannot improve either the optical transfer function of the system preceding the MPI, or the transfer function of the entire system.
The positive effect of microscanning is weakened or completely lost if the scanning process is non-stationary during the integration of the signal generated by the image by the receiver (during charge accumulation). The relative movement of the image and the microscanner during the time the image elements are on the microscanner elements leads to image blurring. The overall transfer function of the entire ICS is deteriorated if there is a mismatch between the microscanner pixels performing the sampling and the pixels of the display system (display).
In microscanning systems, the charge accumulation time must be sufficient to create the required signal-to-noise ratio for each intermediate image, i.e., the subimage obtained with each image shift. This may limit the frame rate formed in the display system. Therefore, it is easiest to provide the required frame rate of the integrated image in MPI systems that have a sufficiently high quantum efficiency, which allows for a shorter charge accumulation time and signal readout time from the MPI elements. The maximum permissible period of frames observed using the display system can be approximately determined as the product of the required charge accumulation and readout time in the photodetector cell (PDC) and the number of positions occupied by the image during microscanning. Thus, with a frame rate of 30 Hz and four-position microscanning (image shift along each axis by 0.5 pixel period), the microscanning frequency is 120 Hz, and the corresponding charge accumulation time is 8.33 ms.
Microscanning can reduce the effect of frequency overlap in the first-order spectrum, but this phenomenon also persists for higher-order spectra. It was proposed to use additional microscanning stages to eliminate the overlap of higher-order spectra, but these proposals did not find practical application due to the complexity of the design, time limitations, and the increased complexity of processing the signals generated in this case. With microscanning, the time to obtain an image increases proportionally to the number of microscanning stages, and the frame rate is inversely proportional to this number. The main limitation of microscanning, as noted above, is its inefficiency in the case of moving objects, the image of which is unstable during the detection or recognition time. Therefore, if achieving high spatial resolution is not very important, but it is necessary to eliminate harmful side images arising from frequency overlap, then in the case of sufficiently bright objects, preliminary spatial optical filtration is often used instead of microscanning, in particular, artificial blurring or defocusing of images.
Inclined rotating plane-parallel plates are used as a scanner in the TVS developed by AEG Infrafot-Module GmbH, Germany [2]. This system uses a 384×288 MCT-based MPI with a pixel pitch of 24 μm and 2×2 pixel microscanning (Fig. 1).
Fig. 1. Four-position microscanning using a rotating disk
At a frame rate of 25 Hz and an accumulation time of up to 2 ms, an output image of 768×576 format is formed here, i.e. the resolution is doubled due to microscanning. The rotation frequency of the disk with plates, equal to 50 Hz, is synchronized with the frequency of the video signal. Each time, when the plane-parallel plate is in front of the FPU in one of the 4 positions, the synchronization system issues a sync pulse to the image processing unit, which issues a command to begin the accumulation of charges by the pixels of the 384×288 frame. After accumulation, the signals are read and the non-uniformity is corrected. Then, having received the corrected 4 frames from the FPU, the full image of 768×576 pixels is returned to the high-speed processor — the image processing unit.
The system described in [2] demonstrated in practice the possibility of very good correction of the inhomogeneity of individual FPU elements, which is fundamentally important for systems with microscanning. The magnitude of the noise-equivalent temperature difference D Tp did not exceed 20 mK with an accumulation time of 1 ms and an aperture value of the lens K = 2.
Another example of a “looking” type ICS with microscanning is the development by Cincinnati Electronics Corporation of a system based on an InSb microscanner of 256×256 elements with a fill factor of 0.25 [3]. The use of four-position microscanning with a piezoelectric drive made it possible, while maintaining the microscanner element pitch of 30 μm and the focal length of 250 mm, to increase the Nyquist frequency from 4.16 mrad-1 to 8.33 mrad-1. At the same time, the frame rate remained equal to 30 Hz. The minimum resolvable temperature difference D Tr at a Nyquist frequency of 8.33 mrad-1 was 100 mK.
Many modern MPIs and multiplexers included with them in the FPU, used to read signals from individual MPI elements, were initially developed within the framework of projects to create new weapons systems. At the same time, until recently, each new multiplexer developed for a specific type of MPI required the development of an original circuit to ensure control of the MPI and FPU. This increased the time and cost of development, and in addition, there was a risk of making an error, i.e. choosing a non-optimal solution.
It should also be noted that the improvement of a number of the most important quality indicators of the ICS began to be constrained not so much by the difficulties of improving the parameters and characteristics of the MPI, but by the limitations introduced by the electronic circuit, in particular, the multiplexer. As an example, we can mention the limited dynamic range or the excessively high output noise of many multiplexers.
In this regard, it is important to note the importance of developing standardized series multiplexers (indigosystems) that have a common interface with the signal processing circuit taken from the MPI elements (with processing electronics) and are already used in TVS, for example, ISC 9705 from Indigo Syst. Corp. Due to the use of a flexible, programmable multi-pipeline architecture of the multiplexer, high values of its parameters and the ability to operate in various modes are ensured. This allows using the multiplexer in a fairly simple mode, used by default, without external adjustments — a signal in NTSC or PAL format is output to one output. In the programmable command mode, such important functions as dynamic image transposition, working with subframes, working on several outputs at increased frame rates and signal scanning are supported. In both modes, it is possible to adjust the gain and perform reading both with signal accumulation and by their simultaneous reading from all pixels after accumulation. For large format MPI, the multiplexer circuit provides a larger number of clock pulses per frame.
The generalized functional diagram of such multiplexers is shown in Fig. 2 [4], and the functions and technical characteristics of the multiplexers (standard readout chips – ROIC) from Indigo System Corp. are presented in Tables 1 and 2.
Fig. 2. Generalized functional diagram of multiplexers from Indigo System
Table 1. Standard readout chips from INDIGO SYSTEMS (functions)
Functions and specifications are subject to change without prior notice
Table 2. INDIGO SYSTEMS Readout ICs (ROIC) (technical specifications)
The use of a standard series of multiplexers in design and production leads to the development of standard and uniform processing electronics for a number of TVS and its placement in the form of a large integrated circuit (LSI) on a single crystal with a multiplexer.
Analyzing this and other trends in the development of IR technology [5], we can predict the appearance in the near future of a thermal imaging converter (TIC) operating in the spectral ranges of 3…5 µm and 8…12 µm, to the input of which IR radiation is received, converted into standard analog and digital video signals, and then into a video image on a microdisplay located in the same single structure. The dimensions of this TIC will be comparable to the dimensions of modern electron-optical converters (EOC) or hybrid-modular converters (EOC plus CCD).
A prototype of such a converter was developed and manufactured at JSC TsNII «Cyclone» and is shown schematically in Fig. 3. The device also contains temperature sensors and special shutters for autocalibration. In the near future, it is proposed to make the processing circuit boards 5, 6, 7, as well as the interface circuit board 8 in the form of a separate microcircuit, which can later be combined with the microbolometric matrix multiplexer.
Fig. 3. Thermal imaging image converter (TIC) operating in the range of 8 – 14 µm:
1 – 320×240 pixel microbolometric matrix;
2 – input germanium window;
3 – sealant;
4 – housing;
5, 6, 7 – signal processing circuit boards;
8 – interface board of the processing circuit block and the monitor;
9 – monitor;
10 – light-emitting monitor screen;
11 – contact pads for supplying voltage and connecting the interface
To fully implement the circuit of such a converter, i.e. to convert a video signal into a high-quality image, it is important to have a high-quality small-sized monitor — a display system, which can be a microdisplay. As is known, microdisplays (miniature high-resolution displays designed for use with a magnifying optical system) are usually used in projection systems such as portable projectors, virtual displays of video camera viewfinders or helmet-mounted or head-mounted (ocular) displays.
Currently, several types of ocular displays are known: liquid crystal displays (LCD) with liquid crystals fixed with polyimide and using ultraviolet treatment and ion-beam bombardment; active matrix electroluminescent microdisplays (AMEL) in digital and analog versions, with the latter providing control over changes in brightness and noise level using electronics built into their design; organic light-emitting diodes (OLEDs), which make it possible to create flexible microdisplays on plastic substrates and with high resolution, both active and passive [6, 7].
It seems that the most promising for new TVPs are OLED-based microdisplays, as they are characterized by high reliability, a wide range of operating temperatures, instant switching on and insensitivity to vibrations. Most OLEDs have a diagonal size of less than 7 mm and a resolution of 320×240 pixels and higher. At the same time, the cost of such a display fits well into the total cost of the entire TVP. Displays of 800×600 pixel format based on silicon wafers containing all the control electronics [7] are known, serially produced by Magin Corp. (USA) and used in helmet-mounted TVS. Fig. 4 shows the device and structural control circuit of such an OLED [8].
Fig. 4. Microdisplay on organic light-emitting diodes with a control circuit
More advanced OLEDs are currently being developed – on light-emitting polymers (LEPs), which are the basis for next-generation flat panel displays. They will be thin, lightweight structures with low supply voltage, low power consumption, high contrast, wide viewing angle and high speed.
The operation diagram of a display on light-emitting polymers is very simple (Fig. 5) [8].
Fig. 5. Operation diagram of a display on LEPs.
Polymer layers are connected between two metal contacts, at least one of which is transparent. When a voltage greater than the threshold bias (2 to 3 V) is applied, current begins to flow. Electrons move from the cathode to the LUMO (lowest, unfilled molecular orbital level) of the polymer layer, and holes move from the anode to the HOMO (highest, filled molecular orbital level), and the structure emits light. An electron leaving the cathode recombines with a hole, emitting light with a wavelength determined by the band gap of the polymer. Due to the high disorder inherent in polymer films, the conductivity in electroluminescent polymers is several orders of magnitude lower than that of inorganic compounds, but their small thickness (50 to 150 nm) allows the operating voltages to remain low. The anode is usually a thin transparent conductive layer of indium tin oxide applied to a glass or plastic substrate (Fig. 6). The cathode can be made of calcium.
Fig. 6. Display structure on SIP
To control the brightness of the image on the display, it is necessary to control the current passing through each pixel. Both passive and active matrices are used for this. Passive matrices control perpendicularly located anodes and cathodes — in rows and columns, i.e. they supply a data signal to the columns with sequential addressing of the rows. As the number of rows in the display increases, each pixel must be supplied with a pulse of higher brightness, which can exceed 20,000 cd/m2. The value of the current required to achieve high brightness levels limits the size of the screen.
With an active matrix, a thin-film transistor made of polycrystalline silicon on a substrate addresses each pixel individually. In displays with an active matrix, there are no limitations associated with high currents, current problems.
Gyricon [10] and E-Ink [11, 12] have developed a display with transistors on a flexible base, which are used to control image parameters. The display consists of sheets of thin-layer elastic material filled with two-color spheres [10] (Fig. 7).
Fig. 7. Structure of a display with two-color electrostatic balls
Each sphere is located inside a separate cavity filled with silicon oil, which allows the sphere to rotate. The spheres are made so that one hemisphere is white, and the other hemisphere is a different color (usually black). Both hemispheres have opposite permanent charges. In a stable position, the spheres are attracted to one side of the cavity and pressed against the wall. When an electric field is applied to the elastic sheet, the sphere detaches from the wall and rotates so as to establish an equilibrium field. If the image is converted into a picture of the distribution of voltages on transistors, the electric field stimulates the rotation of the spheres. Light reflected from them reproduces this image. Each sphere has a diameter of about 30 microns. The surface of the monitor screen has a wide angle of view in reflected light (like paper). At the same time, the contrast is about 12:1. The display consumes very little power.
Literature
1. Cabanski W., Breiter R., Mauk K-H.. Miniaturized high performance starring thermal imaging system. – SPIE Proc., vol. 4028 (2000), p.p. 208 – 219.
2. Cabanski W., Breiter R., Koch R. et al. Third generation focal plane array IR detection modules at AIM — SPIE Proc., vol.4369 (2001), pp.547 – 558.
3. Greiner M.E., Davis M., Sanders J.G. Resolution performance improvements in staring imaging systems using micro-scanning and a reticulated, selectable fill factor InSb FPA-Report Cincinnatti Electronics Corporation SF 298, Mason, Ohio, USA IRIS Conference, February 1999, pp.p.1 – 15.
4. Hoelter T.R., Retronio S.M., Carralejo R.J. et al. Flexible high performance IR camera systems. – SPIE Proc., vol. 3698 (1999), pp.p 837 – 846.
5. Tarasov V.V., Yakushenkov Yu.G. Trends in the development of infrared systems of the “looking type”.//Special equipment, 2004, No. 1, p. 24 – 37.
6. Hardin W.. Technology on display. — SPIE OE Magazine, December 2001, p.p. 18 – 19.
7. Chinnok C. Revisiting the near-eye microdisplay market. – SPIE OE Magazine, June 2002, p.p. 18 – 19.
8. Lewotsky K. Disruption gets practical. – SPIE OE Magazine, June 2002, p. 13.
9. Leadbeater M. Polymers shine the light. — SPIE OE Magazine, June 2002, p.p. 14 – 15.
10. Sprague R. Electronic paper combines programmability, portability, and reusability. SPIE OE Magazine, December 2001, p.30.
11. http://vokruginfo.ru/news/news1132.html.
12. http://cnews.ru/newtop.