Application of high-performance image sensors in industrial barcode readers

summary:

With the trend of lower cost and higher performance industrial cameras today, higher requirements are also put forward for CMOS image sensors, which need to be achieved through the design of system-level chip (SoC). To achieve this goal, multiple image processing tasks need to be integrated into a single device through 3D chip stacking and back side illuminated (BSI) technology. In the future, there will be solutions with sophisticated machine learning and proprietary intelligent computing chips combined with image capture functions to create compact high-speed computing vision systems.

But before new large-scale technology integration can be achieved, two major development obstacles must be overcome - chip thermal management and power consumption.

Today, advanced front side illuminated (FSI) CMOS sensors integrate analog and digital functions to achieve both cost and performance solutions. The key to achieving these goals is to cleverly separate various factors that contribute to system performance and embed them into the image sensor SOC. Here, separating the image system application and the role of existing image processing devices such as CPU, FPGA and DSP is the core factor, because duplication of functions will lead to increased costs. Developing a standard image SOC product that can provide a feasible solution for the target application market to achieve mass production (to achieve the lowest production cost) requires in-depth dialogue between the two modules in advance.

The main market participants of terminal camera products, including hardware, software, system builders and optical engineers, as well as a multi-disciplinary image sensor development team, are contributing their knowledge in semiconductor technology and terminal camera product technology and applications to find innovative product solutions.

This article will introduce a family of CMOS image sensors for barcode readers and other embedded vision applications, their application examples, and some future development trends.

Barcodes – An overview of today’s most popular encoding systems and related reading technologies

In industries such as holiday and seasonal consumer retail and online shopping, logistics, transportation, manufacturing and wholesale industries scan more than 5 billion bar codes every day. With the world's first barcode appearing on a pack of gum in the 1970s, it is now clearly the most popular readable coding system and continues to develop new applications. The 1D barcode was the first barcode to appear (and its area has continued to shrink due to advances in printing technology), and it is still the mainstream technology, which can be seen in UPC (unique product code) applications in retail, transportation and logistics and other industries. 2D barcodes have a variety of different specifications and provide more programmable data than 1D codes: 1D codes can load up to 20-25 characters, depending on the barcode type; 2D codes can load up to more than 2,000 characters. In addition to the general function of writing product information and details, 2D barcodes also write checksums and other correction technologies to ensure greater tolerance for printing errors or damaged barcodes. QR codes have been widely used in some specific industries, such as automated manufacturing and direct part marking (DPM).

The upgrade of 2D barcode reading technology started about 15 years ago because it can read both 2D and 1D codes and has become the mainstream in today's market.

Figure 1: Barcode example

UPC 1D Code

QR Code

PDF417 QR Code

Datamatrix QR Code

Figure 2: Not all QR code readers have the same functionality

Not all QR code readers have the same functionality

Barcode verification and decoding systems are rapidly evolving and continually improving to provide faster, more compact, less expensive, and more powerful reading capabilities.

Although laser-based 1D readers are still in production and use, the most significant technological advancement in reading systems comes from 2D readers. 2D readers use image sensors, which enable them to provide significant computing capabilities and bring additional functions that were previously impossible. These functions include taking photos and recording videos, as well as adding more advanced functions. Common examples include document scanning, optical character recognition (OCR, orthogonal character recognition), object recognition, dimensional measurement, and many other functions.

Teledyne-e2v's image sensor is a unique product for this market, offering many advantages over the various 2D sensor options. One of the main reasons is that it is designed specifically for barcode reading, rather than products typically targeted at the general purpose, consumer or automotive markets. This means an accurate and powerful solution that meets all the requirements of a market-leading barcode reader product.

Teledyne-e2v has recently developed a small pixel low noise global shutter CMOS image sensor series with unique features that can bring significant cost savings and/or performance improvements to the Automatic Data Collection System (ADCS) and Auto Identification (AI) market applications. In this market segment, although the sensor unit cost is the most important factor, cost reduction solutions such as lighting/optical lenses also need to be considered.

Figure 3: Snappy is an innovative, application-oriented approach to CMOS image sensing for industrial barcode readers

2mp and 5mp resolution image sensors

Available in monochrome and color versions CRA 12° and 0°

Small pixel high performance
2.8µm low noise global shutter pixel with in-pixel Correlated Double Sampling (CDS) technology

MIPI CSI-2 interface for adhesive-free embedded imaging systems
4x1.2Gbits channels (optional minimum single channel)

On-chip automatic barcode detection
Intelligent viewfinder function enables fast and accurate barcode recognition

Automatic adjustment Fast self-exposure mode

2D code reading systems require very fast frame capture to avoid smearing. This requires the shortest possible exposure time. On the other hand, to obtain the maximum depth of field (DOF) or scanning range, lenses with very small optical apertures (usually F/8 or less) are often used. The very small number of photons that can enter the image sensor pixel combined with a short integration time means that barcode reading can be performed in low-light applications (see Figure 5). The global shutter is also beneficial for reading moving barcodes.

The key sensor parameters that influence terminal reader performance are therefore specific to barcode reading applications. Figure 4 lists some of the key sensor/barcode reading performance requirements and demonstrates the advantages of the Snappy sensor family as an example of an application-specific CMOS image sensor.

* Dark signal doubles for every 6 to 8°C temperature rise

Figure 4: Snappy’s main specifications for barcode scanning

Signal-to-Noise Ratio

Irradiance (photons/pixel)

Figure 5: The low luminance signal-to-noise ratio of the Snappy sensor provides an advantage in reducing system illumination optics power consumption and cost.

Impact of temperature rise

If one looks more closely at the differences between the various components that make up the noise parameters at 25°C and how they perform at >65°C, some components have limitations in their parameters that should be considered during sensor selection. Spatially fixed row and column read noise is a particularly important parameter for barcode reading. Given that fixed pattern noise has a shape similar to straight and horizontal lines, they can easily be confused with a barcode or add erroneous information to the barcode reading in the image.

Snappy series image sensors use advanced semiconductor processes that produce only a few dark signal photons at 25°C, and only 77 photons per second even at 65°C. This helps the built-in fixed pattern noise cancellation algorithms for rows and columns to achieve fixed pattern noise of only a few percent even at high operating temperatures.

Very low readout noise (combining temporal and spatial elements) with a typical value of 3 photons. It does not deteriorate even at high temperatures. If the sensor performance degrades at high temperatures, it means that more illumination is needed, which increases the system cost.

Unique black and white + color pixel filter mode - combines the advantages of low-light high-sensitivity data of black and white and color pixels

Sensors can use color pixels to add additional object/tag recognition capabilities, providing more security features to prevent spoofing or situations where the barcode itself cannot be read. However, due to the lower transmission characteristics of the organic color filter of the color sensor, and the need to combine red, green and blue pixels to create a "color" pixel, this means that color Zen View Chess has lower spatial resolution and lower sensitivity than monochrome image sensors. Teledyne-e2v's Jade image sensor is an interesting innovation that uses monochrome pixels but adds one color pixel to every four monochrome pixels. This preserves the spatial resolution and sensitivity that are critical for reading barcodes, while allowing a lower resolution color image to be captured simultaneously.

Figure 6: Innovative color sensing applications do not require compromises in reading performance

Innovative embedded application-specific functions

Achieving snappy barcode reading is not simply a result of frame readout rate. While fixed noise is a limiting factor, Snappy sensors do not compromise this. The sensors offer outstanding performance at nearly 120 frames per second at 8 bit depth. A unique power-on mode ensures that the device is powered on or ready when the first image (or fast self-exposure sub-image) is captured at the SNR specification. This is not a standard feature of global shutter CMOS sensors for general automotive or other applications, as it means that the system must discard multiple full-frame images before fully settling and achieving the datasheet SNR capability. This unique first-frame read after power-on capability can provide a differentiating factor for cameras, enabling the highest speed barcode reading, providing "snapshot" scanning to the end user and higher productivity for the enterprise. The following are two of the most innovative patented features invented by the Teledyne-e2v imaging team in the Snappy sensor family: they are designed for ultra-high-speed barcode reading, recognition and decoding applications for end-product scanning.

1. Fast Self Exposure (FSE) mode (for Snappy 2MP and 5MP CMOS sensors):

Fast self-exposure mode allows for optimized exposure time under changing light conditions (see Figure 7). Compared to traditional auto-exposure modes, FSE brings more advantages in terms of integration time and powerful features, including full user programmability and providing stable and fast reading advantages to the end user, adaptive to any light source or dynamic lighting environment, and almost no impact on frame rate.

Figure 7: New on-chip automatic exposure approach for barcode reading and all machine vision applications

The patented FSE mode uses multiple on-chip components to achieve the following functions:

(a) A unique vertical analog-to-digital converter (ADC) allows four different exposure periods to be set in consecutive row segments, which are then repeated across the entire array, producing four low-resolution images with different exposure values. This feature can also be used as a powerful high dynamic range image acquisition function.

(b) Horizontal incremental subsampling, maximum value 1/64 line

(c) On-chip statistics include saturated pixel values ​​and provide a 16-bin histogram output, which can be directly read in the image footer to open the frame or area data

(d) Viewport (ROI) mode supports FSE subframes, multiple regions, and regions within regions

(e) Fine-grained control using histogram values, averages, or a combination of both

(f) Programmable buffer provides intuitive user control and settings

These features bring scanning speed advantages to terminal applications, because the FSE mode generally uses less than 10% of the frame period. Traditional embedded automatic exposure control (AEC) of other CMOS sensors uses asymptotic techniques to avoid flashing and provide target images, making image fusion slower. The entire process consumes a lot of frame width, making the speed unable to meet the requirements of barcode reading applications.

2. Smart ROI mode (for Snappy 5MP sensor):

Smart ROI uses on-chip algorithms to detect one or more barcodes on the image. The barcode decoding image processing system needs to separate the range containing the barcode from the rest of the image so as to process the useful part. This work is usually performed on FPGA or CPU because this task requires a large number of gates/real-time clocks (RTC) and processing power, resulting in additional high costs and complex technical limitations when selecting the processing engine.

Figure 8: Detecting multiple moving barcodes on the image sensor using the Smart_ROI feature

Embedding this barcode detection function in the sensor can achieve overall cost savings, not only because the processing overhead is significantly reduced, but also because the task is completed in the sensor without the need for other digital signal processing, achieving system performance and stability advantages. The 1D or 2D code detected in the active frame will be in the form of blocks with X/Y coordinates as part of the readable information in the image footer area (invisible). The sensor can detect multiple areas (or barcodes) at the same time, and can also detect other codes, such as reading printed characters in optical character recognition applications (OCR). This function can still work effectively even in applications where the barcode/object/camera is moving.

Significant cost savings and system simplification are the main advantages. 5MP sensors are mainly used in high-end barcode applications because they require larger lenses and more processing power to perform 5MP real-time image processing/decoding, offsetting the main advantages provided by a larger scanning range or area. However, the small pixels of the Snappy series sensors and the processing overhead savings brought by the on-chip smart viewfinder can achieve a smaller system-level cost. Due to its low power consumption and ability to recognize multiple barcode symbols in a single frame, this product is becoming a market driver for high-resolution sensors, and it is a clear advantage in the e-commerce logistics industry.

Fast self-exposure and Smart Framing functions work simultaneously on the Snappy 5MP sensor to ensure fast and powerful operation in environments with changing ambient light.

The Snappy sensor series is designed and optimized to meet the needs of low-cost, low-power systems. As mentioned above, the sensor processes data on-chip, which can significantly improve performance. Although it not only meets the barcode reading application and market, other types of machine vision (MV) applications, including inspection, measurement, optical character recognition, etc., can also benefit from the performance and embedded functions of Snappy sensors. Other application areas include embedded vision systems, IoT edge devices, drones, augmented reality, biometric systems, etc.

Figure 9: The Snappy sensor family is suitable for other machine vision, smart IoT and other industrial computer vision applications

Snappy sensor-integrated lens and advanced micro-motor-based autofocus provide added value

Consumer-grade camera optical modules are not suitable for the harsh and long working environments of B2B industrial applications. In addition, for example, lenses used for barcode reading generally have specialized performance and optical characteristics, requiring maximum working distance, and optical aberrations such as optical modulation transfer function (MTF) need to be minimized in order to effectively decode barcodes with feature sizes smaller than the Nyquist frequency. MIPI optical modules (MOMs) that integrate powerful Snappy image sensors and high-performance lenses can increase the value of the system and save development time and cost.

Figure 10: MIPI Optical Module (MOM) using an integrated fixed focus lens and Snappy sensor

MIPI optical modules are an ideal solution for embedded vision applications, allowing the use of custom lenses to some extent, providing excellent performance and flexibility in a small 20mm x 20mm module. Teledyne-e2v is now sampling the first 2MP MIPI optical modules to end users, and plans to launch a 5MP version, as well as a powerful and lightweight 2MP autofocus version based on micro-electromechanical technology. The biggest advantage of autofocus is that it allows the use of a larger optical aperture to achieve the same or better scanning range or working distance than fixed focus optical systems, but requires significantly less illumination power. Future 2MP MIPI optical modules will provide further performance and/or cost improvements for industrial imaging applications. Details of the new products will be announced in mid-2020. However, an evaluation platform is available now using the new open loop 'multi-focus' function, which provides the maximum working range and maximum frame rate, and has MEMS autofocus components that can be used on the existing Snappy 2MP demonstration kit.

New trends and changes brought about by the explosive growth of e-commerce

The explosive growth of e-commerce has brought about a huge double-digit compound annual growth rate (CAGR) target (greater than 25% per year), which not only brings changes to logistics center operations, but also provides security for traditional physical retail points. The retail market is facing a huge transformation, needing to provide a better customer experience and shorten checkout time by using unmanned automatic 'self-scanning' systems. The key to the success of these systems lies not only in reliable barcode recognition and decoding capabilities, but also in more sophisticated object recognition tasks that require more use of color imaging devices.

Figure 11: The explosive growth of e-commerce has led to new trends and changes in CMOS sensor requirements and functions

To fully realize the high growth potential, the first prerequisite is to achieve higher speed or higher capacity scanners and cameras. Higher resolution sensors with wide images will allow faster reading speeds and enable larger surface areas to be read (including multiple packages and barcodes in the same image), and in the future even cover the entire warehouse area with a single sensor.

Regarding image sensors, we see advantages in the use of flexible sensor technology, which can reduce the complexity of external optical components and alleviate some of the existing limitations on small pixels, such as the diffraction limit of the lens. This development can provide two advantages, one is to simplify and save optics, and the other is to allow small pixels below 2.5µ without reducing MTF and still reach the diffraction limit of the lens optics.

In warehouse applications, the need to scan barcodes on a small package often stems from the growing efficiency goals of e-commerce distribution centers. Every square centimeter of storage and transportation space must be maximized. To achieve three-dimensional monitoring of the entire transportation and supply chain of the goods, so as to read the relevant code/text label through the QR code, two separate cameras are currently required, one for three-dimensional (mostly using structured light or stereoscopic vision based on 3D technology), and the other using a non-interconnected two-dimensional camera.

The current research focus is on developing a 2D and 3D CMOS sensor that can provide both traditional 2D images and 3D point clouds. Teledyne-e2v is committed to providing cutting-edge leadership technology for these new market areas and has developed a future product roadmap and intellectual property that can match the next generation of cameras and imaging systems, so that the relevant technology can focus on application-specific needs. I believe that it will not be long before new products can be foreseen.