Serializer and camera chip application

introduction

Maxim serializers can connect to and control camera ICs. These devices include the MAX9257 (with a half-duplex UART/I²C control channel), the MAX9259, and the MAX9263 (both with a full-duplex, synchronous control channel). The MAX9263 also supports high-bandwidth digital content protection (HDCP). This application note describes how to convert the RGB or YUV output of a camera into RGB data accepted by a standard display.

Camera output data format

Camera chips, such as the OmniVision® OV10630, can be connected via a serializer. The interface pins of the OV10630 include: pixel clock, PCLK, line valid, HREF, frame sync, VSYNC, and parallel data bits D[9:0]. The data bits remain stable on the rising edge of the clock.

YUV and raw RGB data formats

CMOS camera sensors include millions of photosensitive units, each of which responds to light signals of the entire wavelength. Filters are used to make specific sensors respond only to red, green, or blue light signals. Adjacent photosensitive units are usually arranged in a Bayer structure with twice as many green filters as red or blue filters. This method is used to simulate the light-sensitive characteristics of the human eye. Reading the sensor unit output from left to right and from top to bottom, the original RGB data sequence is blue, green...blue, green (end of the first row), green, red...green, red (end of the second row), and so on, as shown in Figure 1 .

Figure 1. Original RGB data arrangement

RGB data with the same density as the sensor cell is generated by interpolating adjacent cells. In addition, the image can be restored according to specific rules using the colors of adjacent cells. One of the rules for forming the RGB data group for each pixel is to use adjacent cells in the same row, plus the green adjacent cells in the next row (or previous row). The interpolated RGB data sequence is..., red (i-1), green (i-1), blue (i-1), red (i), green (i), blue (i), red (i+1), green (i+1), blue (i+1),... as shown in Figure 2. Each pixel requires a set of RGB data to drive a color display and maintain the highest resolution of the camera sensor. The brightness resolution of the interpolated RGB data is close to the resolution of the sensor cell, but the chrominance resolution is poor. Since the human eye is more sensitive to the grayscale of each pixel than to the color components of the pixel, the perceived resolution is basically the same as the sensor cell resolution.

Figure 2. RGB data arrangement

However, this interpolation algorithm of RGB data triples the data rate. In order to reduce the data rate, especially when image transmission is required, the YUV color space can be used (compressing the analog color TV signal into the frequency band of analog black and white TV). In the following formula, brightness is represented by Y, the color difference between blue and brightness is represented by U, and the color difference between red and brightness is represented by V.

Where the typical color weights are: WR = 0.299, WB = 0.114, WG = 1 - WR - WB = 0.587, and the normalized values ​​are UMAX, VMAX = 0.615.

For camera sensors using Bayer color filters, the U or V data of adjacent pixels is roughly the same, depending on the row index i and the pixel index j (if the rule used is adjacent colors). Using this guide, YUV data can be directly generated from RGB data according to the following formula.

Even row index i and even pixel index j.

Even row index i and even pixel index j.

For odd row index i and even pixel index j.

For odd row index i and even pixel index j.

Even row index i and even pixel index j.

Even row index i and even pixel index j.

For odd row index i and even pixel index j.

For odd row index i and even pixel index j.

Even row index i and even pixel index j.

Even row index i and even pixel index j.

For odd row index i and even pixel index j.

For odd row index i and even pixel index j.

In order to reduce the data rate, the U data with even pixel index and the V data with odd pixel index, as well as the Y data with even and odd pixel index are used. The compressed YUV data is sent according to the arrangement shown in Figure 3 , that is, Y1, U0 and V1 are the data of pixel 1; Y2, U2 and V1 are the data of pixel 2, etc.

Figure 3. YUV422 data arrangement

422 represents the sampling ratio of Y:U:V. The 4:x:x standard is the early color NTSC standard. The chroma is resampled according to 4:1:1, so the color resolution of the image is only one-fourth of the brightness resolution. Currently, only high-end equipment that processes uncompressed signals uses 4:4:4 color resampling, and the resolution of brightness and color information is exactly the same.

Serializer Input Format

The parallel interface of Maxim serializer is designed for 24-bit RGB data, especially MAX9259, which has pixel clock bit (PCLK) and 29 data bits for 24-bit RGB as well as horizontal sync, field sync and 3 control bits. In addition to the parallel data interface, the DRS and BWS pins need to be set high or low to select the data rate and bus width respectively.

Maxim Serializer/Deserializer

The MAX9257 and MAX9258 serializer/deserializers (SerDes) feature 18-bit parallel I/O for YUV data transfer; the MAX9259/MAX9260 chipset features 28-bit parallel I/O for RGB data transfer; and the MAX9263/MAX9264 SerDes feature 28-bit parallel I/O with the addition of HDCP functionality. In addition, the MAX9265 and MAX9268 28-bit SerDes feature a camera link instead of a parallel I/O interface. All 28-bit Maxim serializers and deserializers have the same parallel/serial data mapping and can be used interchangeably. For example, the MAX9259 serializer can be used with the MAX9268 deserializer to transfer RGB data (with the help of an FPGA). Data is sent from the CMOS camera via a serial link to a display at the camera link interface.