Some aspects of automatic recognition of vehicle license plates.

Some aspects of automatic recognition of vehicle license plates.

Some aspects of automatic recognition of vehicle license plates

Currently, there are more than half a billion cars on the roads in the world. All these vehicles have a unique identification number as the main identification mark. The vehicle identification number is, in fact, a registration number that gives the legal right to participate in road traffic.

The problem of identifying a car by its registration plate is an important aspect of traffic control and safety. Products capable of solving this problem are in demand in a variety of areas. Examples include motor transport enterprises, car parks, garage cooperatives, cottage villages, petrol stations, entry control points to the territory of a facility, etc.

When we talk about an automatic number plate recognition system (License Plate Recognition, LPR), we mean a software or hardware-software complex that implements automatic number plate recognition algorithms for recording events related to the movement of vehicles, i.e. for automating data entry and subsequent processing. Strictly speaking, an LPR*system is a device that records the passage of a vehicle, reads its registration number and outputs it to an ASCII data processing system.

Currently, there are quite a lot of LPR* systems with different levels of recognition quality, speed, and range of additional functions provided. Products with high speed and recognition accuracy are usually very expensive. Their high cost does not allow for mass implementation. Let's consider the general principles underlying the recognition of vehicle license plates to understand the reasons for the high cost of such systems.

Algorithms and technologies for recognition of license plates

Undoubtedly, the basis of any LPR* system is the recognition algorithms used. The qualifications of developers in the field of modern higher mathematics, image processing, programming and software optimization technologies, as well as the presence of significant work experience — all these factors determine the characteristics of the LPR system, such as:

• Probability of recognition.
• Processing speed.
• Ability to recognize different types of license plates.
• Ability to work with images of different quality.

Fig. 1.

Recognition of state registration plates is a non-trivial task in the field of technical vision and artificial intelligence. The algorithms used to localize the license plate and recognize it are usually a commercial secret and, naturally, are not published. Only a few companies name their types and publish the sequence of actions.

Let's consider the operation of the LPR*system using the example of «TeleWizard*AUTO», developed by Nordavind CJSC. The key stages of recognizing a car number are listed below:

• Bringing the original image to a form that does not depend on the conditions of image recording (illumination level, uneven distribution of brightness from light sources, blurriness, noise, etc.).
• Isolation of candidate areas on the resulting image that potentially contain a plate with a number.
• Conducting a detailed analysis of candidate areas based on a formal representation of the scale characteristics of the number plate and reducing the space for further search.
• Bringing the graphic image of the license plate to a standard size with image quality correction.
• Preliminary determination of the license plate type (in relation to current standards).
• Extraction of individual characters and their recognition (analysis of characters by key characteristics independent of scale, font used, geometric distortions and breaks).
• Refining the recognition results based on information about the license plate type and the results from previous frames.

The result of the algorithm is information about the vehicle's passage, containing a line with the recognized
number, a still frame with the best image of the vehicle, information about the time of the car's passage, etc.
From the presented sequence of steps, it is clear that the initial data for recognizing the number is not limited
to only a visual image. There are a large number of types of license plates in the world, differing in:

• fonts used (signs with symbols of different sizes, Latin, Cyrillic and other fonts);
• color of the background and symbols (black symbols on a light background or white symbols on a dark background);
• number of lines in the number (single-line, two* and three-line);
• presence or absence of a region designation code or special mark
• etc.

Taking these differences into account in license plates gives a significant “head start” to those developers of LPR* systems who use this additional information in the logic of their recognition algorithms. Information about the structure of the sign and its syntax allows to significantly increase the probability of correct recognition, while simultaneously reducing the requirements for the quality of algorithms for extracting and recognizing individual characters.

Optical Character Recognition Technology
The accuracy of Optical Character Recognition (OCR) technology makes a significant contribution to the overall performance of the LPR*system. To understand the complexity of the task being solved at this stage, consider the following simple example. Let's assume that the LPR*system being developed requires a 95% probability of correctly recognizing a license plate on a static image. Let's determine what the probability of recognizing a single character of the license plate should be.

Let three algorithms precede direct character recognition:

• An algorithm that localizes a license plate in an image. Probability

• A pre-processing algorithm that normalizes contrast and brightness, which corrects the image. Probability

• The character extraction algorithm, which is responsible for finding and extracting individual characters on the sign and passing them to the character recognition algorithm. Probability

The total probability of correct recognition, in which n algorithms are used to achieve the goal, is determined by the formula:

Or, for our conditions:

From here it is easy to find that the total probability of character recognition should be no less than:

For example, the main Russian state registration plates of the new model (with a three-digit region code) have 9 symbols. If the overall accuracy of optical recognition of a license plate must be at least 97.7%, then the accuracy of recognition of an individual symbol must be at least:

That is, it is assumed that out of 1000 symbols fed to the input of the OCR* module, only 3 may not be recognized or recognized
incorrectly!

Today, quite a lot of OCR methods are known. As an example, consider the structural method of recognizing skeletal
images used in one of the solutions of Nordawind CJSC. In structural methods, an object is described as a graph, the nodes of which are the elements of the input object, and the arcs are the spatial relationships between them. Methods that implement this approach usually work with vector images. The structural elements are the lines that make up the symbol.

Fig. 2.Variety of license plate types

First of all, the recognizable symbol undergoes the procedure of obtaining a skeleton, for which any of the well-known algorithms described in the subject literature can be used. Then, for each special point of the obtained skeletal representation of the symbol, a set of topological features is calculated, the main ones being:

• Normalized coordinates of the special point (graph vertex).
• The length of the edge to the next vertex as a percentage of the length of the entire graph.
• Normalized direction from this point to the next special point.
• Normalized direction of entry to the point, exit from the point.
• Curvature of an arc, or more precisely, the “left” and “right” curvature of an arc connecting a special point with the next vertex (the curvature is calculated as the ratio of the maximum distance from the arc points to the line connecting the vertices to the length of the segment connecting the same vertices).

Fig. 3. Example of topological features

Figure 3 shows some of the topological features. The graph has five special points – a0, a1, a2, a3, a4. When traversing the graph along the route a0, a1, a2… the following features are shown at the vertex a1: vector R1 is the direction of entry to the point, vector R2 is the direction of exit from the point, vector R3 is the global direction to the next special point. The bidirectional vector h shows the value of the “left” deviation of the arc (a1, a2) from the straight line; the “right” deviation is zero.

For some codes, the number of special points and, accordingly, the number of topological features is too small. Thus, for the code corresponding to the symbol «0», there are no topological features at all, since there is not a single special point. Therefore, the following additional features can be calculated and used:

• Sizes and positions of components and «holes».
• «Black» and «white» width of the upper half of the symbol.
• Modified straight deflections.

Fig. 4. An attempt to improve the symbol image. a) original symbol image; b) symbol with glued lines

The deflections are calculated as the distances from the points of the skeletal representation to the convex hull of the constructed representation.
Additionally, the position of the maximum deflection points is remembered. For some topological codes, the number of topological features may be large enough to require too large a set of standards for training, so in some cases only part of the features is used in recognition.

The symbol is determined after comparing its description with codes from the database, and the closest topological code is selected.
If the symbol remains unrecognized after passing the recognition cycle, an attempt is made to improve the image (Fig. 4) using the following operations:

• Gluing the ends of lines in directions.
• Gluing the skeleton points that are at a minimum distance from each other.
• Discarding the shortest line.

The method under consideration is not optimal. Its disadvantages include high sensitivity to image defects that disrupt the constituent elements. Vectorization can also add additional defects. In addition, effective automated training procedures have not yet been created for these methods (unlike template and feature methods), so structural descriptions most often have to be created manually.

Real LPR* systems often use complex OCR algorithms that are a synthesis of several methods. From the description provided, it is clear that the effective operation of the OCR algorithm significantly depends on the quality of the image fed to the input.

What is meant by good image quality?

The technology used to obtain the image determines the average image quality that the recognition algorithm will have to work on. Obviously, the higher the image quality, the better the conditions under which the license plate recognition algorithm works and the greater the accuracy that can be achieved by the LPR system.

To obtain good results from the license plate recognition algorithm, the processed images must contain
license plates:

• With acceptably good spatial resolution.
• With acceptably high definition.
• With acceptably high contrast.
• In acceptably good lighting conditions.
• In an acceptably good position and at the right angle.

Of course, «acceptable» is a rather arbitrary definition, although it has a very precise meaning. Some typical
problem images are shown in Figure 5.

Figure 5.Some typical problematic images

Typical solutions

Perhaps one of the most widespread and popular applications of LPR* systems is access control to the territory of a facility and a parking lot. Within the framework of these applications, we can already identify the main type of number plate recognition systems with a typical configuration of technical support and system equipment.

An example of a number plate recognition system is an access control number plate recognition system (it is worth noting that
in most cases, a number plate recognition system is only part of an integrated access control system).

	Fig. 6. The car drives up to the entrance
	Fig. 7. Creating a digitized image of a car
	Fig. 8. Reading a license plate from a digitized image of a vehicle
	Fig. 9. Allowing access and saving data in the database

An example of a number plate recognition system is an access control number plate recognition system (it is worth noting that
in most cases, a number plate recognition system is only part of an integrated access control system).

A car approaches the gate of a protected area, which it wants to enter. At the checkpoint, there is a barrier and a red traffic light, indicating that entry is prohibited. In addition, a CCTV camera is installed at the entrance. The barrier, traffic light and camera are connected to a control computer with an installed LPR system, which coordinates the operations of the access control system.

If the system registers the approach of a vehicle, the LPR module attempts to «read» the license plate within the control zone.

After reading the license plate number, the LPR* module transmits the recognized license plate number for subsequent decision-making. The access control and management system then sends the number to the database module. The database module compares the received number with the access rights lists and returns the «access granted» or «access denied» flag. Depending on the flag type, the access control application opens the barrier and turns on the green light or not.

In addition, the access control application can send additional information to the database module, such as the date or time of access, to be saved in the event log. After the vehicle passes (or leaves), the system returns to its original
position and waits for the next vehicle.

Conclusion

The presented overview of the principles of automatic license plate recognition makes it clear how wide is the range of problems that LPR may face.

I. Svirin
k. t. n., General Director of Nordavind CJSC,
A. Khanin
Chief Specialist of Nordavind CJSC