Discussion on Basic Knowledge of Digital Video

Discussion on Basic Knowledge of Digital Video

Due to the needs of my work, I have been doing development in digital video processing for several years. What I mean by development is the development of the application layer, not the research of the bottom layer. I have developed several products, such as digital interface cameras, embedded video capture cards, Video-VGA video conversion cards, online spot detectors for the papermaking industry, digital pseudo-color processors, etc. During the development, I also learned some knowledge about CMOS area array sensors, CCD area array sensors, CCD line array sensors, high-speed cameras, high-resolution cameras, video encoding and decoding, image compression, etc. I am still doing similar development. The products mentioned above have a certain sales volume, so I often receive calls from users, most of which are to consult some very basic digital video knowledge. Digital video is undoubtedly complex and involves many aspects, so I have the idea of ​​writing this article. I hope to explain some basic issues in the most popular way, hoping to help readers with their development work (Note: In order to make the problem as simple as possible, I have made some simplifications in the description, such as saying U and Cb are U, V and Cr are V, etc. It should also be noted that the various parameters of analog video I am talking about below are for China's television standard - PAL system, while other television systems are slightly different. Readers should pay attention when doing further research). Because my level is very limited, it is difficult to avoid mistakes and omissions. Interested readers can write to me to discuss.
Video is simply a moving image. Movies are also moving images. Movies record a large number of still images on film and display them one by one continuously, which becomes the movie we see. For television video, there are dozens of still images per second, each still image frame consists of hundreds of lines, and each line consists of hundreds of pixels. The number of image frames in 1 second is the frame rate, the total number of lines in 1 second is the line rate, and the total number of pixels in 1 second is actually equivalent to the video bandwidth.
Let's talk about analog video first.
Looking back, under the limited conditions of the time, the predecessors who used electron tubes to make televisions were really admirable. However, from today's perspective, the television technology of the time was really very simple. The frame rate of black and white video signals in China is 25 Hz, which means that 25 images are displayed per second. The reason why it is set to 25 Hz instead of some other value is mainly due to two reasons. One is related to the visual physiological characteristics of human beings. When the refresh rate of the image reaches 5 frames/second, people begin to feel that the image is active, and when it reaches 24 frames/second, people feel that the image is completely continuous and smooth (the frame rate used in movies is 24 Hz), so the frame rate of the video signal should be greater than or equal to 24 Hz. Theoretically, the higher the frame rate, the better. However, the higher the frame rate, the higher the requirements for the circuit, the more complex the technology, and the higher the cost (some computer color display frame rates have reached 200Hz). Under the conditions at that time, only a value greater than 24 and as small as possible could be selected. Another reason is that the frequency of my country's power grid is 50Hz. When a 25Hz frame rate is used, the field frequency during interlaced scanning is 50Hz, which is exactly the same frequency as the power grid. In this way, the interference of the power supply to the image is fixed, and it is not easy for the human eye to feel it. Or to put it another way: the power supply does not need to be so precise (at that time, switching power supplies were rarely used, and the rectifier power supply lacked voltage stabilization measures, and the ripple was very large). So a 25Hz frame rate was chosen. Speaking of this, we have to talk about "field frequency". When the TV displays an image, it divides a frame into two fields for display. One field consists of odd lines in the frame, called the odd field, and the other field consists of even lines in the frame, called the even field. The reason for doing this is mainly because when the CRT tube displays 25 frames of images per second, the human eye still feels that the continuity is not very good, and there is obvious flicker. After a frame is divided into two fields, the field frequency is 50 Hz, and the image is more continuous. Of course, there are some other reasons related to circuit design.
China's black and white video signal stipulates that each frame of the image has 625 lines, each field has 312.5 lines, the line frequency is 15625 Hz, and the video bandwidth is 6MHz. Among the 312.5 lines in each field, some lines are used for field blanking and do not contain video signals. According to the line numbering method specified in the CCIR656 standard, the line numbers of the odd field are lines 1 to 312.5, and the line numbers of the even field are lines 312.5 to 625. Among them, lines 23.5 to 310 of the odd field contain valid video signals, a total of 287.5 lines, and lines 336 to 622.5 of the even field contain valid video signals, a total of 287.5 lines. Therefore, the total number of valid lines in a frame is 576, which is composed of the top half line, the middle 574 lines, and the bottom half line. (Speaking of this, I have to mention another problem. When experienced readers use a video capture card to capture images on a PC, if the resolution is set to 720 points * 576 lines, they will find that the left half of the top line is black, or the right half of the bottom line is black. This is because a field contains half a line of invalid lines.)
The above is about black and white video signals, so what about color video signals? When scientists began to study color television, black and white televisions were already in widespread use, so

It is best that the color TV signal can be compatible with the black-and-white TV signal so that the color TV signal can play a black-and-white image on a black-and-white TV. This problem is very difficult because the chrominance signal also occupies a large bandwidth, and on the TV radio frequency band, one channel is next to another, and the brightness signal (actually there is also the FM audio signal) has filled the frequency band. Fortunately, this problem was finally solved by using technologies such as compressing the brightness signal bandwidth and large-area coloring. From the perspective of the frequency domain, the chrominance signal (UV color difference signal) is inserted in the gap of the spectrum of the brightness signal, specifically at 4.43MHz, with a bandwidth of 1.3MHz. In the receiver, simply put, the chrominance signal is taken out from the 4.43MHz signal with a bandwidth of 1.3MHz, and the remaining signal is the brightness signal after filtering out the 4.43MHz signal from the received signal. Many hardware engineers may not understand what this means easily, but it doesn't matter. Next, we look at the waveform of one line from the perspective of time domain. As shown in the figure, the brightness signal of black and white video adopts amplitude modulation. The period of one line is 64μs, of which the signal displayed on the screen occupies 52μs, and the rest is line blanking and line synchronization header. For color signals, a small 4.43MHz subcarrier signal is superimposed on the line synchronization header, which is used as the frequency and phase reference of the 4.43MHz signal in the receiver.
The above is how to add the color video signal to the original black and white video signal. So how is the color image restored and displayed? We know that the pixels of black and white images can only be described by brightness (grayscale). The description of color image pixels is more complicated. There are many different methods. For example, the printing industry uses CMYK (cyan, magenta, yellow, black) four-color synthesis method, while the CRT picture tube of computers or TVs uses RGB (red, green, blue) three-primary color synthesis method. I don’t understand why these three colors are chosen instead of other colors. I guess it may be because the human eye contains RGB three color sensing cells, so these three primary colors can synthesize more colors that the human eye can recognize, or because the phosphors of these three primary colors are easier to manufacture. Everyone is familiar with how to synthesize a certain color through the RGB three primary colors, or how a certain color is decomposed into three primary colors. It is indeed very intuitive to use RGB three primary colors to represent color, but if this method is used for image transmission, it is definitely not a good method. The first disadvantage is that it is incompatible with black and white images. The method of converting RGB three primary colors into grayscale is: grayscale = R*0.3+G*0.59+B*0.11, and this conversion process is obviously more complicated. For a TV, it means that the RGB signal must be decoded to get a black and white image, but a black and white TV does not have a decoding function, so compatibility cannot be achieved. The second disadvantage is that it takes up too much bandwidth. When the RGB three primary colors are used to represent the image, the bandwidth of each component is equal, which is approximately equal to the bandwidth of the brightness signal. Therefore, a larger bandwidth must be used to describe each component. The third disadvantage is poor anti-interference ability. Since the G component accounts for 59% of the brightness value, when G is interfered with, the brightness value of the pixel will be greatly affected, and the human eye is very sensitive to changes in brightness values, so the subjective quality of the image will be significantly reduced. For these reasons, the YUV synthesis method is used in video signal transmission. Y represents brightness information, U represents blue color difference (that is, the difference between the blue signal and the brightness signal), and V represents red color difference. Let's take a look at the advantages of using this representation method. The first advantage is compatibility with black and white images. Assuming that a pixel is represented by YUV, we only need to ignore the UV component and take out the Y component to get the brightness value of the pixel, thereby converting the color image into a black and white image. This makes it easy to achieve compatibility between color TV signals and black and white TV signals. The second advantage is saving bandwidth. When talking about this problem, we must first talk about the principle of large-area coloring. Experiments have found that the human eye is sensitive to brightness information, and mainly distinguishes the details of the shape of an object through brightness differences, but is insensitive to color information. The human eye cannot distinguish the slight changes in the color of an object, or it is not easy for the human eye to perceive the changes in the details of the color of the image. Therefore, the brightness signal can be sampled at a higher sampling frequency, while the chrominance signal can be sampled at a lower sampling frequency (or with a lower quantization depth). For example, the brightness values ​​of several adjacent pixels are different, but the same chrominance value can be used. This is the principle of large-area coloring. Based on this principle, in television signal transmission, the bandwidth of the U or V signal is much smaller than the bandwidth of the V signal, which saves bandwidth. To put it another way, for example, in a computer, using RGB to describe a pixel requires a total of 3 bytes of R, G, and B. If described in YUV, for every 2 pixels, Y uses 2 bytes, U takes the same value, uses one byte, and V takes the same value, uses one byte, with an average of 2 bytes per pixel. Or each pixel Y uses one byte, U uses half a byte, and V uses half a byte, for a total of 2 bytes. The third advantage is strong anti-interference ability. Since the brightness signal is represented separately, if the color difference signal is interfered with, it will not affect the brightness, and the subjective perception of noise will not increase significantly.
In a TV, the color video signal is first decomposed into the brightness signal Y and the chrominance signal, and the chrominance signal is then decomposed into the U color difference signal and the V color difference signal. Finally, the three components of YUV are transformed into RGB signals through matrix operations for display on the picture tube. So how is YUV specifically transformed into RGB? This problem is also called "color space conversion", and I will discuss this problem in detail later in this article.
Through the previous discussion, we already know that the black and white video signal has a bandwidth of 6 MHz and consists of frames, fields, lines, pixels, etc., and the pixels are described by the brightness value Y. The color video signal inserts a chrominance signal with a bandwidth of 1.3 MHz into the black and white video signal, and obtains the UV color difference signal from this signal, and finally converts YUV into RGB to describe the pixel.
Now let's analyze the shortcomings of the video signal. 1. Low frame rate. The frame rate of the video signal is only 25 Hz, which will inevitably cause the image to flicker. 2. Low resolution. There are only 576 valid lines in one frame. Due to the use of interlaced scanning, a frame of image must be composed of two consecutive fields, but in fact it is difficult to ensure that the lines in the two fields are accurately staggered (aligned with the gap), which further leads to the loss of vertical resolution. 3. Bright color crosstalk. The brightness signal and the chrominance signal are mixed together, and they cannot be separated well during decoding, resulting in mutual interference between the brightness signal and the chrominance signal. 4. Lack of room for improvement. Unless a new standard is re-established, the three problems mentioned above are difficult to improve on the existing basis. The video signal format has so many shortcomings because it is restricted by the technical conditions when the standard was established. In recent years, the image quality of television has been improved by adding some digital processing methods to television, such as double frequency scanning (100 Hz field frequency) and the use of digital comb filters. The digital television that is being studied now is a brand new standard that has been re-established to obtain film-quality images. It may completely eliminate the current video standards and television equipment. Of course, this is a matter of the future and is not the topic of my discussion.
What I want to discuss is "digital video", while what I have discussed before is analog video. This is because the digital video I am talking about is the digital representation of existing analog video. Once we understand analog video, the following discussion will be very simple.
Video signals were originally stored in the form of analog signals on video tapes, but now with the development of digital technology, they can be converted into digital signals and stored in CDs or computer hard disks. Of course, these are inseparable from powerful computers. In fact, digital video can also be applied in the embedded field, such as using a single-chip microcomputer or DSP to process digital video data. Next, we will discuss the format, conversion, storage, and display of digital video data.
Which question should we start with? Let's first discuss some signal elements contained in the analog video signal after it is decoded and quantized into a continuous digital video stream. We have discussed before that the analog video signal has 25 frames per second, each frame period is 40ms, and each frame is divided into 2 fields, each field is 20ms, the odd field is output first, and then the even field, the odd field line number is from 1 to 312.5, the even field line number is from 312.5 to 625, among which, the odd field line 23.5 to 310 contains valid video signals, and the even field line 336 to 622.5 contains valid video signals. After the analog video signal is decoded into YUV components, A/D quantization sampling is performed separately and converted into a digital video stream, and the time should also be output in the above order. There are some relevant international standards that have made some agreements on this. For example, in 1994, the International Radio Consultative Committee issued the CCIR601 standard, which was mainly formulated for the requirements of the studio. It stipulates that the sampling frequency of the brightness signal is 13.5 MHz, the sampling frequency of the chrominance signal is 6.75 MHz, and 8-bit PCM encoding is used. In this way, the number of Y samples per line is 864, of which the effective number of Y samples is 720. The number of U or V samples per line is 432, and the effective number of U or V samples is 360. The average number of bits of YUV describing each pixel is 8bit, 4bit, and 4bit, respectively, which is also called the YUV422 encoding scheme (of course there are many other schemes, such as YUV411, etc.). The International Radio Consultative Committee also issued the CCIR656 recommendation, which stipulates that the 0 and 255 in the quantization value of video data are reserved, and the order of serial output of quantization data is: U0, Y0, V0, Y1, U2, Y2, V2, Y3, U4, Y5, V4, Y6, and so on. From the above discussion, we can see that the elements that a digital video stream should contain are: odd-even field indicator signal FI (sometimes called ODD), field synchronization signal, line synchronization signal, pixel clock, and YUV data output. Here we calculate the data volume of the digital video stream. The data volume per second = (720 pixels * 576 lines * 25 frames) * 2 bytes = 20736000 bytes, and the data rate is about 165Mbps. This shows that the data volume and data rate of digital video are large!
As you can imagine, the simplest way to save and describe digital video streams is of course to record and describe continuous frames of still images. The simplest format for saving still images is the BMP format, which is a bitmap. Let's analyze the BMP file format now. There are actually many ways to record images in BMP files, and you can even directly record YUV components in them, but I'm not going to discuss that much. A BMP file consists of four parts: file header, image feature description, color table, and image data. For simplicity, the syntax of VB is used below, and the numbers used below are all decimal unless otherwise specified.
The data structure of the BMP file header is as follows:
Type BitMapFileHeader '14 bytes in total
bfType As Integer '2 bytes, filled with the character "BM", that is, 4D42 (hexadecimal)
bfSize As Long '4 bytes, filled with the byte size of the entire BMP file
bfReserverd1 As Integer '2 bytes, reserved, filled with 0
bfReserverd2 As Integer '2 bytes, reserved, filled with 0
bfOffBits As Long '4 bytes, indicating the starting position of the image data in the entire BMP file
End Type
The data structure of the image feature description block is as follows:
Type BitMapInfoHeader '40 bytes in total
biSize As Long '4 bytes, indicating the byte size of this structure, filled with a fixed value of 40
biWidth As Long '4 bytes, filled with the number of pixels in the horizontal direction of the image, its value must be an integer multiple of 4
biHeight As Long '4 bytes, filled with the number of pixels in the vertical direction of the image
biPlanes As Integer '2 bytes, filled with a fixed value of 1
biBitCount As Integer '2 bytes, indicating the number of bits for each pixel, 8 for grayscale images and 24 for 24-bit true color images
biCompression As Long '4 bytes, filled with 0 for no compression
biXSizeImage As Long '4 bytes, indicating the total number of pixels in the image
biXPelsPerMeter As Long '4 bytes, filled with a fixed value of 3780
biYPelsPerMeter As Long '4 bytes, filled with a fixed value of 3780
biClrUsed As Long '4 bytes, filled with a fixed value of 0
biClrlmportant As Long '4 bytes, filled with a fixed value of 0
End Type
The image data recorded in BMP is not necessarily the RGB or Y value, it can be just a "number", and the actual RGB color corresponding to this "number" must be found in the color table. 24-bit true color images do not need a color table because they directly record RGB values. Therefore, there is no color table in 24-bit true color BMP files, but in other cases, there must be a color table. The color table has 256 items in total, each of which is 4 bytes. The first three bytes represent the B, G, and R values, respectively, and the last byte is 0. Let's take an 8-bit grayscale image as an example. An 8-bit grayscale image is a black and white image. What is actually recorded is the brightness component Y. According to the RGB three primary color principle, when R=Y, B=Y, and G=Y, a gray pixel with a brightness of Y is synthesized. In the extreme case, when R=255, G=255, and B=255, it represents the whitest pixel, and when R=0, G=0, and B=0, it represents the darkest pixel. Therefore, the value of the color table should be: 0,0,0,0,1,1,1,0,2,2,2,0,3,3,3,0……..255,255,255,0.
The last part of the BMP file records the image data, which is also the part with the largest amount of data. The order of filling the pixel data of a frame of image into the BMP file is: fill in the bottom row first, and fill in pixels one by one from left to right, and so on until the top row is filled. For 8-bit grayscale images, only one byte of Y value needs to be filled for each pixel data. For 24-bit true color images, 3 bytes need to be filled. Note: fill in the B value first, then the G value, and finally the R value.
To summarize: the BMP file of grayscale image consists of "file header + image feature description block + color table + image data Y". The 24-bit true color image consists of "file header + image feature description block + image data BGR". If the reader is not familiar with these, you can draw a picture in the drawing tool of WINDOWS, save it in the above format, and then use binary editing tools such as UltraEdit to observe and analyze the contents of the file. For more knowledge about BMP files, readers can refer to further materials.
Now let's go back to discuss the format of saving videos as BMP images. For digital video streams, since the number of valid lines in a frame is fixed at 576 lines, if it is converted into a 576-line BMP image, it is of course the simplest and can achieve better results. If other values ​​are taken, such as 600 lines or 400 lines, then interpolation operations must be performed to calculate the imaginary lines based on the existing lines. Such conversion has a large amount of calculation and has a certain loss of image quality. It is generally implemented with special hardware (generally high-end video capture cards have such a function). So does it mean that it can only be converted into 576 lines? Of course not. For example, it can be converted into 288 lines, and every other line is taken, which actually means only one field is collected. In addition, some lines can be discarded, such as only taking the middle 480 lines and discarding 48 lines at the top and bottom. Although the image is cropped, the clarity will not decrease. Let's take a look at how many pixels should be sampled in a line. Since the amplitude of the analog video signal changes continuously in a line, the number of samples is not limited like the number of lines. For example, we can sample 400 pixels, of course, we can also sample 401 pixels, which depends on our requirements for horizontal resolution. The more pixels sampled in a line, the higher the resolution. However, it should be noted that the bandwidth of the brightness signal is limited. After the sampling rate reaches a certain level, it is meaningless to increase it. As mentioned earlier, the CCIR601 standard stipulates that 720 effective pixels are sampled in a line. In addition to this standard, another commonly used standard is to sample 768 effective pixels in a line. By the way, when you use a video capture card to capture images on a computer, the driver provides several fixed image formats, such as 768*576, 720*576, 384*288, 320*288, etc. Why only these formats are provided? Now you know why those readers who didn't understand before know it!
Now we know the format of analog video after it is converted to digital video stream, and we also know how to create and save BMP files, but we can't save the image data captured from the digital video stream as BMP files, because the digital video stream is described in YUV, and the BMP file is described in RGB. How to convert between them? This is the problem of color space conversion.
The correspondence between RGB and YUV is expressed by algebraic equations as follows:
Y = 0.299R + 0.587G + 0.114B
U = - 0.147R- 0.289G + 0.436B
V = 0.615R - 0.515G - 0.100B
Or:
R=Y+1.14V
B=Y+2.03U
G=Y-0.58V-0.39U
The values ​​defined in the CCIR601 standard are slightly different from the above. After considering the non-linear characteristics of the human visual system and CRT displays, the recommended conversion equations are as follows:
R = Y + 1.371 V
G = Y - 0.698 *V - 0.336 U
B = Y + 1.732 U
However, readers should note that the UV value in the above equation is shifted by 128 due to sign extension. The following corrected equation should actually be used:
R = Y + 1.371 * (V - 128)
G = Y - 0.698 * (V - 128) - 0.336 * (U - 128)
B = Y + 1.732 * (U - 128)
There are a few notes on the above equations: 1. Some capture cards will specify U and V as signed or unsigned numbers. Don't consider this when using the above equations, and assume that Y, U, and V are unsigned values ​​between 0 and 255 obtained from the capture card. 2. The R, G, and B values ​​calculated by the above formula may exceed the range of 0 to 255. They should be checked after calculation. If the value is less than 0, it should be corrected to 0. If the value is greater than 255, it should be corrected to 255. The above conversion method has been proven to be effective after my actual use. After
so much empty talk, let's do something more intuitive! Let's take a look at an actual image. The image on the right is obtained using the VC302 embedded video capture card produced by Wuhan Wande Digital Technology Co., Ltd. The signal source is the video output of the TV. The pixel resolution of the image is 320*240 (288 lines in a field are collected, each with 360 pixels, but only 240 lines are intercepted from it, each with 320 pixels). The first picture is a black and white image described by the Y component, the second picture is described by the U component, the third picture is described by the V component, and the fourth picture is a color image synthesized by YUV. It is a bit hard for you to believe that a black and white image can be synthesized with such a blurred UV component map to obtain a color image with such realistic colors? I can't believe it either, but this is the fact! From the comparison of these figures, we can draw the following conclusions: 1. The Y component basically retains the contour details of the color image, and its resolution is relatively high. 2. The UV component is subjectively very fuzzy, with low resolution, and only roughly describes the color of the entire block, which is the so-called "large area coloring". 3. When we observe the synthesized color image, we do not feel that the color of the image is blurred, which shows how poor the human eye's ability to distinguish color details is!
Next, let's discuss the display problem of the image. This problem belongs to the scope of software, and most readers are familiar with it, so I am not going to say too much. One way is to convert the obtained YUV data into RGB and then display it. Another way is to use YUV data directly for display. The current graphics cards have the ability to convert YUV data directly into RGB through hardware. By using Direct Draw technology, the YUV data is directly submitted to the graphics card, which saves the CPU time spent on software conversion. In the embedded field, a digital interface LCD display can be used for display, which should be handled according to its specific situation.
Finally, let's talk about some common problems in video development. 1. First of all, we must pay attention to distinguish the video signal format. The signal sent by my country's TV stations is a PAL signal, so the video output of the TV is PAL. But the TV can also accept signals of other formats. For example, many of the video outputs of VCD machines are NTSC, which can also be played on the TV. Most of the cameras sold in the Chinese market (I am talking about industrial cameras, not home camcorders) are PAL, but some are NTSC, and some can be set to the format through the DIP switch on the body. 2. Some DVD players use some special methods to improve the clarity, such as the so-called "progressive scanning". The signals they output are slightly different from the standard signals. There will be no problem playing on the TV, but some acquisition cards cannot recognize it, resulting in disordered captured images, so be careful when using DVD as a signal source during debugging. 3. Resolution issues. Resolution generally refers to the maximum number of black and white stripes that can be distinguished in the vertical direction and are arranged at equal intervals. Let's first look at analog video. The time for the video signal to travel is 52μs, and the maximum bandwidth of the video signal is 6 MHz. Assuming that 1Hz can describe 2 pixels, the maximum number of lines is 52μ*6M*2=624 lines. In fact, there will be a lot of losses in the editing, storage, transmission, and restoration process, so the image resolution on the TV is much lower than this value, generally around 240-340 lines. For the video output of the camera, some black and white cameras have a nominal resolution of up to 600 lines, which is theoretically possible. The nominal resolution of color cameras has specifications such as 380 lines, 420 lines, and 480 lines. Let's take a look at the digital video stream. The number of pixels sampled in a line of digital video is fixed. The CCIR601 stipulates that it is 720 pixels. If it is expressed in lines, the limit value is 720 lines, which is larger than the resolution of analog video. In security monitoring projects, the most commonly used video source is a 420-line or 480-line color camera. After deducting the loss during transmission, it is good to reach 380 lines in the end. It is more appropriate to take 360 ​​pixels per line when saving as an image. If the number of pixels is increased, the clarity will still be improved, but it is not obvious. In the industrial field, sometimes there are particularly stringent requirements for resolution. At this time, the camera with standard video signal output can no longer meet the requirements. Non-standard cameras are needed. Most of them are line-by-line scanning and use special high-resolution CCD array sensors. Common resolutions are 2048 lines * 2048 pixels/frame, and the frame rate can be as high as 100Hz. Of course, such video signals cannot be displayed on TV. They are generally collected by a special acquisition card and processed on a computer. 4. Field acquisition and frame acquisition. This problem is quite special. In theory, the camera should first take a frame of image, then decompose it into two fields, and when restoring, the two fields are combined into one frame. But in fact, it is often not like this. In order to simplify the design, some cameras collect one field at a time, and there is a 20ms time difference between the two fields. Therefore, when the two fields are combined, the active part of the image will be misaligned. In addition, some video editing equipment also processes the field separately, so this problem also exists. When watching TV, because the image changes continuously, we only feel that the image is active, but if we cut off a frame of the image, the misalignment is very obvious. In order to avoid this problem, when the resolution requirement is not high, you can collect a single field, that is, only one of the two consecutive fields is taken. In fact, many acquisition cards also handle it in this way. In addition, you can collect more pixels in the horizontal direction while collecting only one field, such as collecting 288 lines in total, but 720 pixels per line, so that at least there is a higher resolution in the horizontal direction, which can improve the effect of pattern recognition. When the resolution requirement is higher, you have to use a non-standard camera.