Television observations in difficult conditions.

Television observations in difficult conditions.

Kulikov Alexander Nikolaevich

TELEVISION OBSERVATIONS IN DIFFICULT CONDITIONS.

Quite recently, in 1970, a microprocessor and a charge-coupled device (CCD) appeared simultaneously [1].

Modern life is already unthinkable without computers and CCD cameras.

The electronic revolution continues and amazes us every year with new products and unprecedented price reductions.

It is unlikely Did anyone in the 70s think that several miniature TV cameras could fit into a matchbox and cost less than an electric kettle, all with excellent image quality and high reliability?

Today, most cameras are used in security television systems. Security cameras operate in different surveillance conditions.

They are often simple, when objects are well lit and there is no noticeable interference. But there are also complex situations when a television camera monitors in the light of the sun and stars, in conditions of haze, fog and rain, under water, in the presence of intense electromagnetic interference, radiation, etc. In such conditions, conventional CCD cameras no longer provide reliable surveillance.

This requires special highly sensitive CCD matrices, wide-range aspherical lenses, special methods of signal adaptation and processing, and much more.

This article examines the possibilities of television surveillance in the most difficult conditions — at the edges of the range of working illumination.

1. Methods for expanding the range of working illumination.

Inventors are improving television cameras, trying to bring them to the quality of the natural eye. It turned out that the human eye is a perfect visual instrument.

It has many amazing properties and one of them is the widest range of perceived illumination.

During the day, we can see, slightly squinting, white snow under the sun and clouds with an illumination of more than 100,000 lux.

At night, we easily walk along a road illuminated by the light of the stars (approximately 0.0001 lux). Dividing the first value by the second, we get 109 — the range of illumination perceived by the eye is equal to one billion or 180 dB!

No electronic signal sensor has such a wide dynamic range. This is due to physical limitations — the level of its own noise on the one hand and the level of signal saturation on the other. But as it turned out, natural optical signal sensors do not have such a range either: cones and rods of the human eye.

With a stretch, we can assume that the dynamic range of both natural and artificial light sensors is 1000 (60 dB). Where does one billion come from?

To achieve a range greater than the dynamic range of the signal sensor, it is necessary to build an automatic control or adaptation system.

In television cameras, two adaptation methods are most common.

• In the first method, an adjustable attenuator and a signal amplifier (controlled lens diaphragm and image intensifier, respectively, in the television camera) are installed in series in front of the light sensor.
• In the second method, the photosensitive sensor itself is made controllable and its sensitivity is changed by adapting the parameters.

a) The first method of adaptation to illumination in a television camera

b) The second method of adaptation to illumination in a television camera

Fig. 1 a, b. Methods for expanding the range of working illumination in CCD cameras

Modern television cameras use both methods of adaptation to illumination levels, each of which has its own advantages and disadvantages.

Below, the capabilities and limitations of each method will be considered in relation to the area of ​​extremely high and extremely low illumination.

Fig. 2 Illustration of the expansion of the range of working illuminations
of the photosensitive sensor operating as part of the adaptive photosensitive system.

 2. Night surveillance.

Advertising manipulations with sensitivity.

Every company that produces and sells TV cameras advertises its product. Phrases like “excellent image quality,” “DSP processor,” “Ultra-high sensitivity,” and so on are used. It’s not just about quality. Many are tempted to slightly exaggerate the quantitative parameters of TV cameras.

For example, you can see how the reputable company SONY honestly states a resolution of 370 television lines (TVL) for its SPT-M308CE camera, while the young Korean LILIN in the PIH-752 model indicates 420 TVL.

When choosing which camera to buy, the consumer does not know that both cameras use the same ICX-055BL CCD matrix and frame chips. Other technical characteristics of TV cameras are also exaggerated.

But perhaps no other parameter has been subjected to such advertising manipulation as “sensitivity”. The reason for this is the different interpretation of it by different authors and companies.

Sensitivity characterizes the ability of a television camera to observe at night. Sensitivity is the minimum illumination, expressed in lux, at which the camera is still capable of forming an image.

If we limit ourselves to these phrases, then ambiguity arises, which allows us to declare sensitivity figures for the same camera that differ by more than 100 times.

How can this situation be corrected?

• First, you need to specify where the illumination is measured: on the CCD matrix or on the object. For example, with a lens with a relative aperture of F1.4 and an object reflection coefficient of 0.75, the illumination on the matrix will be 10 times less than on the object.
• Second, when measuring illumination on the object, you should specify the relative aperture of the lens. For example, the sensitivity of the same camera with an aspherical lens F0.8 will be 10 times better than with a small-sized lens F2.0.
• Thirdly, it is necessary to specify: what signal/noise ratio is taken as the threshold when measuring sensitivity. For example, previously, the minimum illumination was understood as that at which the full resolution of the camera was preserved, that is, the signal/noise ratio was approximately 34 — 36 dB. Now the minimum illumination is interpreted as that at which only large details of the image can be distinguished, that is, the signal/noise ratio is 20 — 24 dB. In space and military technology, the threshold sensitivity is often understood as that when the signal amplitude is equal to the amplitude of the noise track, that is, the signal/noise ratio is 5 — 6 dB and almost nothing is visible in the image except noise. In this case, sensitivity values ​​that differ by a factor of 10 can also be specified for the same camera.

A typical example of advertising manipulations of the last two years is the sensitivity declared by the Japanese company Watec for the WAT-902H camera.

In 1999, specialists were amazed by the sensitivity value of 0.0003 lux at F1.4 in the AGC switch position “Hi”.

This is 100 times better than for standard CCD cameras! Many even believed this figure, especially since the camera turned out to be really good. The WAT-902H was the first to use the SONY ICX249AL matrix of the new generation EXWAVEHAD with a 4-fold improved sensitivity. But only four, not a hundred.

After the camera was bought and disassembled into small parts, it turned out that there was nothing special in this camera except the new SONY matrix, and the “Hi” position is a 4-fold increase in the gain, which only increases noise, but not sensitivity.

Now everyone has woken up and is indicating only 0.002 lux for the WAT-902H, as, for example, in the latest English catalog NORBAIN” [4], although this figure is too high and it should be indicated as 0.005 lux, which corresponds to measurements on the object with a signal-to-noise ratio of 20 dB and F1.4.

Methods for improving sensitivity.

There are the following methods for improving the sensitivity of television CCD cameras:

• Using highly sensitive CCD matrices and high-aperture lenses.
• Using electron-optical image intensifiers (EOPs).
• Introduction of adaptive modes of charge accumulation and reading in CCD.

Use of highly sensitive CCD matrices and high-aperture lenses.

First, we will list the factors limiting sensitivity in modern CCD cameras and the possibilities of improving them by using new CCDs and lenses.

Fig. 3. Illustration of various sensitivity limiting factors in a CCD camera.

• Light Losses in the Lens

Not all photons of light that hit the input lens pass to the CCD matrix. Some of them are scattered, and some are absorbed by the lens material.

It should be said that modern aspherical lenses with a relative aperture of 0.8 – 0.75 — have very high characteristics and it is difficult to expect noticeable improvements in their parameters in the near future.

• Losses due to the small relative area of ​​photosensitive elements to the total area of ​​the photosensitive section. Photosensitive cells, especially in small format matrices of 1/3 inch and less, occupy less than 10% of the sensitive surface area. The remaining area is used for charge transfer channels and an anti-blooming system. 10 years ago, this was one of the main limitations of sensitivity. SONY invented and applied transparent microlenses on the CCD surface, which concentrate light from the entire surface onto small photosensitive cells. A year ago, SONY improved these lenses and released a new series of CCD matrices under the EXWAWEHAD CCD brand, which made it possible to additionally increase the sensitivity of TV cameras by 3-4 times. At present, the parameters of the microlens array are close to the theoretical limit, and it is also difficult to expect significant improvements here.

• Photon/electron conversion losses. The quantum yield of the best CCDs approaches 0.5 in the visible and near IR wavelength ranges. The development of new materials and further optimization of the device structure in the future could increase this value, especially in the blue and near ultraviolet ranges, which could improve the sensitivity of cameras. However, serious changes are also hard to expect here.

• Sensitivity limitation due to CCD output device readout noise. Currently, readout noise is the main factor limiting the sensitivity of TV cameras. Its value of 20 — 30 electrons/pixel could theoretically be reduced by 10 times. The limitation here is the area of ​​the gate of the first output transistor. The smaller the area, the less noise, but a gate with a small area is not able to accommodate the pixel charge in the case when there is a lot of light, which will lead to signal limitation in daytime conditions. There are patents that propose placing two output devices in the CCD matrix, one for small and one for large charges, and switching them at night and during the day, respectively. Therefore, we can expect the future emergence of new CCDs with reduced output device noise, which could lead to a further increase in the sensitivity of CCD cameras several times.

• Sensitivity limitation due to glow of transistors of the output device of the CCD matrix. All transistors glow weakly (similar to LEDs and laser diodes), and in CCD matrices this prevents observation of low illumination. As early as 13 years ago, an article was published [2], where a glow was noticed in a cooled astronomical CCD camera in the corner of the image where the output device is located. At that time, this was regarded as a unique phenomenon, which only manifested itself when cooling CCDs operating with a long exposure time. Since then, the sensitivity of CCD matrices has increased 100 times and this effect already prevents observation in the most sensitive cameras of PANASONIC, BAXALL, EVS. Russian specialists managed to photograph glowing transistors using a highly sensitive VNC-702 camera. Two cameras were used for this, and one of them observed the CCD matrix of the other camera, which was in the on state. The image clearly shows both transistors of the two-stage output device of the ICX249AL CCD matrix glowing. Other types of CCD matrices were also tested, and it turned out that the output devices of all the studied matrices glow, but only with different intensities and glow areas. This new serious obstacle, which had not been paid attention to before, forced the EVS company to modify Japanese matrices and seal their output device with opaque material in those cameras where maximum sensitivity was required. There is hope that CCD manufacturers themselves will pay attention to the glow of transistors and cover the glowing elements in a simpler way.

Photo 1. Glow of output transistors in the CCD matrix of SONY ICX-249AL
Published with permission from EVS

Using electron-optical image brightness amplifiers.

Electron-optical image intensifiers have been used in television for a long time. Even before the era of CCD cameras, electronic amplification stages were built into transmitting television tubes, achieving sensitivity on the object of 0.001 lux and higher.

After the disappearance of cameras on cathode-ray tubes, electron-optical converters (EOC) remained, which were used in military applications as night sights and night vision devices. These EOCs began to be coupled with CCD cameras to increase their sensitivity.

A new class of ultra-sensitive television cameras was formed.

However, television cameras of the “CCD+EOC” type are not very common, as they have serious drawbacks.

There are two disadvantages: extremely high cost, reaching $10,000 and more, and low reliability due to the possibility of the image intensifier tube being destroyed by sunlight and from high-voltage leaks and breakdowns. Currently, CCD cameras with generation 3+ image intensifier tubes have unrivaled sensitivity and are used in areas where the importance of reliable night surveillance outweighs the cost.

It should be noted that CCD+EOP cameras are increasingly being replaced by highly sensitive CCD cameras with adaptive “night” modes. For example, the sensitivity of EOP generations 1, 1+ and 2 has been successfully surpassed by night TV cameras from PANASONIC, IKEGAMI, KAMPO, BAXALL, EVS and others.

Therefore, it can be said that cameras with EOP of the first two generations will not appear on the TV camera market today, since they cannot compete with CCD cameras either in sensitivity or in cost.

Cameras with image intensifier tubes of generations 2+, 3 and 3+ still exist as exotics, but after the next technological revolutions of SONY and PANASONIC they will inevitably disappear like mammoths.

Table 1. Comparative characteristics of TV cameras with image intensifier tubes

Note. Since sensitivity figures for cameras with image intensifiers are usually given for good quality images, at full resolution, i.e., at a signal-to-noise ratio of 34–36 dB, then for comparison with CCD cameras, where sensitivity is given at a signal-to-noise ratio of 20–24 dB, the sensitivity figures in Table 1 should be reduced by 5 times (multiply by 0.2).

Introduction of adaptive modes of charge accumulation and reading in a CCD matrix.

When the first CCD matrices appeared, the main task of engineers was to create a reliable solid-state analogue of a cathode-ray tube. And only after some time, attention was paid to the adaptive properties of the new device.

The fundamental capabilities of the CCD to work equally well in a wide range of charge reading clock frequencies, as well as the ability to sum charges from adjacent elements and rows before reading the signal from the device output, turned out to be new.

This made it possible to create an experimental CCD camera back in 1985 without an ARD lens or any light filters with a range of working illumination equal to the human eye [3]. The range of working illuminations of 1 billion was achieved only by reconfiguring the parameters of the still very antediluvian CCD matrices of the 80s.

Currently, using the new SONY EXWAWEHAD series matrices, it is not difficult to significantly exceed the characteristics of the eye.

Let's stipulate that so far this is only possible within the range of working illuminations and contrast sensitivity. In other parameters, it is still very far from the eye.

So, how can the sensitivity of a television camera be improved by adapting the CCD parameters?

Signal accumulation before exposure to noise.

There are different ways to increase the sensitivity of a television camera, but they are all based on one principle: “the principle of signal energy accumulation”.

This principle is based on the fundamental difference between a signal and noise.

A signal is always unipolar (positive in television) and has a limited frequency band.

Noise is always differential with zero mathematical expectation and a significantly wider frequency band.

As a result, simple addition (accumulation) of portions of «signal plus noise» will lead to a linear increase in the signal level and only to a slow (according to the square root law) increase in the average deviation of the noise amplitude.

Every 100 additions improve the signal-to-noise ratio by 10 times. The principle of signal energy accumulation is used in all methods of increasing sensitivity, be it space-time summation or low-pass filtering.

The adaptive properties of CCD matrices allow for the use of a unique method of increasing sensitivity, which can be conventionally called «accumulation before noise».

Its essence is that additional summation (accumulation) of the signal is performed in the CCD matrix itself before the signal enters the output device and is joined by readout noise. As a result, the signal is added without adding up the noise, and the noise is added in the CCD output device once for each sum of signals.

As a result, fourfold addition leads to a fourfold increase in the signal-to-noise ratio, rather than a twofold increase as in conventional methods.

This mode is effective due to the fact that at low signals, the readout noise significantly exceeds the photon noise and the latter has virtually no effect on the accumulation result.

Fig. 4a. Accumulation of signal with noise (standard method)

Fig. 4b. Signal accumulation before noise exposure (in CCD cameras)

One of the first cameras with adaptive signal accumulation was released by PANASONIC.

The mode was called “Electronic sensitivity enhancer” and provided an increase in the accumulation time from 1 to 32 television fields, i.e. from 1/50 to 0.64 seconds, which led to an improvement in sensitivity up to 32 times. Currently, cameras with the “Electronic sensitivity enhancer” mode are produced by many companies, such as IKEGAMI, BAXALL, PCAM, KAMPO and many others.

In these cameras, when using CCD matrices from SONY EXWAVEHAD series and aspherical lenses, sensitivity up to 0.0002 lux is achieved with a signal-to-noise ratio of 20 dB. Despite the excellent characteristics, cameras with the “Electronic sensitivity enhancer” mode have two serious drawbacks.

Firstly, when increasing the exposure, the image of moving objects is blurred, which can cause a fast-moving intruder to be missed, which is unacceptable in security systems. The second drawback is the relatively high cost, since a TV standard converter with frame RAM, ADC, DAC and synchronization system is needed to visualize the image thinned by 32 times on the monitor screen.

As a result, even Korean cameras with the Electronic sensitivity enhancer system cost twice as much as conventional TV cameras.

Another option for adaptive signal accumulation is summing up the charges from adjacent elements of the CCD matrix.

By changing the CCD synchronization mode, it is possible to provide addition of charges of adjacent elements on the gate of the output transistor and adjacent rows on the electrodes of the CCD output register. Just as in the first method, addition of the signal occurs before the effect of noise, and tenfold addition leads to a tenfold improvement in sensitivity. The first and so far the only company that has implemented this mode in its TV cameras is the Russian company EVS. The mode was called “night mode 1” and in CCD cameras of this company it is automatically switched on when the illumination on the object decreases to less than 0.02 lux. CCD cameras of EVS, made on EXWAVEHAD CCD matrices of SONY, in night mode 1 develop sensitivity up to 0.0002 lux (camera VNC-703), which is equivalent to TV cameras with the “Electronic sensitivity enhancer” mode.

Cameras with “night mode 1” operate without increasing inertia, which allows them to reliably observe moving objects, up to illumination corresponding to the illumination from the starry sky.

The cost of cameras with “night mode 1” is only 10% higher than standard ones, since they do not require the use of expensive frame RAM. The disadvantage of “night mode 1” is the deterioration of resolution at night by about three times due to the summation of charges from neighboring elements and lines.

It seems obvious to further increase sensitivity by combining the two modes “Electronic sensitivity enhancer” and “Night mode 1” in one CCD camera. At the end of 1999, the first such camera VNC-702 from EVS appeared.

The advertising materials indicate that the television camera, which develops sensitivity on an object of 0.00004 lux with a signal-to-noise ratio of 20 dB, uses “night modes 1 + 2”. By “night mode 2”, EVS means the “Electronic sensitivity enhancer” mode, which in the VNC-702 camera is limited to 16-fold frame summation for better observation of moving objects.

The unique VNC-702 camera is currently the record holder for sensitivity among CCD cameras and is only a few times inferior to cameras with generation 3 and 3+ image intensifiers. In the maximum sensitivity mode of the VNC-702 camera, the glow of the output transistor of the CCD matrix is ​​clearly visible (see photo 1), which currently prevents further growth of the sensitivity of adaptive CCD cameras.

Table 2. Comparative characteristics of TV cameras with CCD matrices of the EXWAVEHAD series and with electronic modes of increasing sensitivity.

Note.

For ease of comparison, the sensitivity of all cameras is indicated on the object using a high-aperture aspherical lens with a relative aperture of F0.8.

Conclusions.

• Modern CCD cameras with an electronic sensitivity increase mode have practically ousted expensive cameras with generations 1, 2 and 2+ image intensifiers from the market, providing high sensitivity at night and reliability at a reasonable price.
• The weak glow of the transistors of the output device of the CCD matrix limits the sensitivity of the best television cameras with night modes. Work is needed by CCD manufacturers to reduce this glow.
• In the coming years, we can expect a further increase in the sensitivity of CCD cameras due to a decrease in the readout noise of the output devices and an increase in the summation coefficient in cameras with electronic sensitivity enhancement.

Literature.

1. Seken K., Thompset M. Charge-Transfer Devices/Transl. from English. Ed. by V.V. Pospelov, R.A. Suris. — Moscow, Mir, 1978 — 327 p.
2. J.R. Janesick, T. Elliott, S Collins. “Scientific charge-coupled devices.”, Optical Engineering, August 1987, Vol. 28, No. 8, p. 692 – 714.
3. Kulikov A.N. Evaluation of the tuning ranges of the decomposition parameters in a television camera on a CCD matrix, “Communication Equipment”, series. Television Equipment”, 1985, Issue 4, pp. 47 – 54.
4. Catalog of the Norbain Security company “CCTV WAREHOUSE”, October, 1999, p. 37.
5. Advertising catalogs of the companies SONY, PANASONIC, WATEC, TURN, IKEGAMI, PHILIPS, JAI, LILIN, EVS for 1999 and 2000.