CMOS image sensors dominate multiple applications, providing huge market opportunities

Over the past decade, continuous improvements and enhancements in CMOS image sensor technology have enabled it to move from primarily serving the low-end market to some of the most demanding high-performance applications. With this trend, CMOS image sensors have begun to dominate the image sensor market and provide many opportunities for original equipment manufacturers (OEMs) to differentiate themselves.

Market analysis firm TSR estimates that in May 2010, CMOS image sensors already accounted for more than 90% of the market share in high-end digital SLR (DSLR) and mirrorless digital cameras. Similarly, data released by the video sharing site YouTube in 2011 showed that mobile devices based on CMOS image sensors accounted for the vast majority of video uploads, especially high-definition (HD) videos. If the total number of shipments in each market segment is considered across the industry, CMOS is undoubtedly the technology winner in imaging.

Compared with traditional charge-coupled device (CCD) image sensors, CMOS image sensors have several distinct advantages, including manufacturability, low power consumption, ease of integration, low cost, and, more recently, high image quality. Altogether, these inherent advantages have enabled OEMs to develop more competitive imaging systems that better serve end users.

This article will discuss the development trends of CMOS image sensor technology and innovations in four market areas, including mobile phones, automotive, surveillance and digital still cameras (DSC). In each market area, these CMOS technology advances have met or exceeded the market requirements in that area, and the advantages of CMOS have enabled OEMs to improve their camera products.

CMOS Image Sensors Drive Growth and Product Differentiation
in the Mobile Device Market In the mobile space, CMOS image sensors have moved from the low-end mobile phone market to full market dominance. CMOS image sensors are now the natural choice for all mobile devices with camera functions, such as smartphones and tablets. These mobile devices are becoming ubiquitous and represent the most important growth area in the mobile device market.

In March this year, IDC, the world's leading market analysis company, predicted that global smartphone market shipments would grow by about 49.2% in 2011 to about 452 million units. IDC also predicted that global smartphone market shipments would reach 925.7 million units in 2015. Similar to the above analysis, Infinite Research's research report showed that in January 2011, tablet sales will grow at a compound annual growth rate of 56% or more, and according to current estimates, shipments will increase from about 16.1 million units last year to more than 147 million units in 2015.

With so much growth potential in the market, one might think that there must be plenty of opportunities for OEMs to differentiate their products and thus increase their profits. But interestingly, this may not be the case. Two developments in the mobile device space could lead to greater homogeneity in the market.

First, many newly developed smartphones and tablets will use Google's Android operating system. The excellent Android operating system is free in most cases, which removes a large part of the burden associated with developing smartphones and tablets. As a result, OEMs that were previously shut out by software development barriers will be able to compete in the market with almost every other mobile device manufacturer in the world. While this is good for emerging markets and relatively small OEMs in general, making them more competitive, it also means that consumers will begin to see an Android-based tablet or smartphone from one manufacturer as being much different from one from another.

Second, as Arete Research analyst Richard Kramer predicts, low-cost smartphones will have a higher growth rate. These phones, which cost about $150 or less in bill of materials, may further exacerbate the homogeneity of mobile products.

Mobile phone OEMs that wish to gain a competitive advantage in the market and differentiate their products from their competitors cannot do so from an operating system or cost perspective. OEMs should seek to provide consumers with differentiated features and functions, such as a high-performance 5-megapixel to 8-megapixel camera, or the ability to capture high-definition video at a relatively high frame rate.

For mobile device OEMs, CMOS image sensors, which have become the first and best choice for mobile devices, may be the key to achieving higher profits and profitability.

There are clear examples of how important smartphone cameras are to consumers, and how camera performance can have a strong influence on which vendor a consumer ultimately purchases from. For example, in July 2010, a Pew Research study showed that 76% of adults surveyed used their phone’s camera. In contrast, 72% of respondents sent text messages and 38% used their phones to access the Internet. The same survey found that 34% used their phones to record videos. More recently, a March 2011 GSMArena Camera Phone Usage Report showed that 98% of surveyed mobile phone owners used their camera phones at some frequency.

In this market segment, the camera and the applications it enables, such as video calling or augmented reality, could become a key differentiator and contrast from other featureless Android phones.

Advances in CMOS image sensor technology in the 5-megapixel to 8-megapixel range will further help this differentiation. As shown in the example in Figure 1, Aptina uses A-PIX technology to improve quantum efficiency and reduce crosstalk on CMOS image sensors. This pixel technology uses advanced technologies such as light guides, deep photodiodes and 65nm processes to improve pixel performance and overall image quality. By using proven semiconductor manufacturing process technology, CMOS image sensors with A-Pix technology can provide advantages in performance and cost, making them an excellent choice for smartphone and tablet applications.

Figure 1: Image taken at 30 lux using Aptina's A-Pix technology 8-megapixel, 1.4μm pixel image sensor

CMOS Image Sensors Drive Advanced Automotive Applications
As automotive imaging systems become more complex and demanding, CMOS image sensors are also evolving and improving to effectively meet the requirements of the new era of automotive intelligent solutions. Improvements in CMOS technology have benefited from billions of dollars of R&D investment in new technologies over the past decade. The resulting technological advances allow automotive OEMs to differentiate their products and provide advanced and desired features. More importantly, these technological advances also help automotive system OEMs create simpler and safer cars to meet the needs of consumers and government agencies. A typical example is the new requirements proposed by the US government, which will lead to the mandatory installation of rear-view systems, also known as backup camera systems.

Obviously, CMOS image sensors are an important part of such automotive subsystems, and seemingly simple applications such as backup cameras have quite stringent requirements for operating temperature range, sensor dynamic range, spectral range, and even power consumption. In at least some cases, CMOS image sensors are the only imaging solution that can meet all the specifications, which is simply not possible with CCDs.

Other important advanced driver assistance solutions (ADAS) include: lane departure warning systems that use a combination of CMOS imagers, range finding technology and other onboard sensors to help drivers stay in their lanes; park distance control systems and even automatic parallel parking systems that use a combination of CMOS image sensors, ultrasonic sensors and drive-by-wire functions; sign recognition systems that rely on CMOS image sensors to help drivers avoid accidents; and adaptive cruise control systems that use CMOS sensors and range finders to prevent accidents on the highway. The examples mentioned above are just some of the dozens of automotive imaging applications that are already in production or under development.

These various automotive safety and driving convenience systems can be divided into scene observation and scene processing applications.

Scene observation applications convey visual information to the driver. The backup camera system mentioned above is a typical example. In these applications, the key requirements for image sensors include high operating temperature, excellent low-light performance, wide dynamic range (WDR), and low power consumption. Of course, while achieving these performances, they must also meet automotive quality standards and achieve lower unit costs at high volumes.

For example, Aptina's MT9V128 and MT9V129 are VGA system-on-chip (SOC) solutions designed for automotive scene observation applications. Similarly, Aptina offers advanced megapixel (MT9M024) solutions for next-generation scene observation cameras.
Scene processing automotive applications often require information from CMOS image sensors, combined with input from other systems, to make specific decisions such as warning the driver or even adjusting the path or speed of the car. Lane departure warning systems are a good example of scene processing systems.

For scene observation and scene processing applications, CMOS image sensor technology is replacing CCD sensors and becoming the ideal choice for high-volume imaging solutions due to its superior performance in most environments. IMS predicts that the shipment of CMOS image sensors for automotive applications will increase from about 9 million units in 2009 to about 50 million units in 2017. Considering the long life cycle of automotive products, CMOS image sensors are destined to have an unprecedented high growth in the automotive market, which is good news for automotive OEMs, allowing products to further improve performance and cost.

IP Camera Surveillance Systems Require CMOS Imagers
The surveillance industry is going through a major transition period, primarily moving from CCD-based closed-circuit television (CCTV) analog systems to more advanced digital Internet Protocol (IP) surveillance solutions.

In the surveillance market, trends are similar to those in the other areas discussed in this article, where the superior technology of CMOS image sensors is delivering many new features and capabilities that provide key differentiators for surveillance system OEMs.

IP surveillance systems implemented with CMOS image sensors are more economical and easier to install. For example, some forward-looking surveillance system OEMs are launching a new, high-volume, customer-made (DIY) surveillance market segment, where DIY surveillance systems for homes and small offices can cost as little as $100 and could trigger a larger surveillance market than CCD CCTV systems ever expected.

IP camera systems offer several advantages over CCTV systems. They do not have to consider the barriers and limitations of the National Television System Committee (NTSC) and Phase Alternating Line (PAL) resolution standards established decades ago. IP camera systems are also an opportunity to increase the use of megapixel image sensors. In addition, many of today's IP camera systems use wide dynamic range technology, which allows the camera to be placed almost anywhere because it solves the challenge of capturing clear images in very dark and very bright environments. IP camera systems that do not have wide dynamic range technology run the risk of underexposing or overexposing images, which are unacceptable in the image surveillance market.

Many IP camera solutions also offer embedded video processing to analyze live video streams in order to detect or track people or events in a timely manner. Previously, this capability required several servers and considerable processing power. Advances in digital signal processor (DSP), field programmable gate array (FPGA) technology, and lighter heuristic-based video processing algorithms have made it possible to perform most image analysis on the actual camera. This analysis may include background modeling, foreground extraction, binary large object (BLOB) detection, and even optical flow analysis. All of these different components and functions can be integrated into the camera system without adding a lot of cost.

Image sensors used in IP camera systems must provide relatively high resolution, either 720P or 1080P, and also need to achieve a relatively fast frame rate, such as 60 frames per second (fps). Although the standard video frame rate is only 30 frames per second, the rate of 60 frames per second provides camera manufacturers with high flexibility to adapt the width required for the shutter to capture fast-moving objects in the scene. In addition, entertainment venues such as casinos require high frame rate video surveillance in order to capture the evidence needed to convict a crime.

Capturing high-definition video at 60 frames per second with high quality in all lighting conditions is challenging. Megapixel sensors can offer either wide dynamic range or excellent low-light sensitivity, but they typically cannot achieve both at the same time. Fortunately, this is changing. High-definition megapixel sensors designed with Aptina's DR-PIX technology and multi-exposure wide dynamic range imaging can achieve both.

Aptina's DR-Pix technology allows programmable conversion gain adjustment to match all pixels in the field to the overall brightness level of the scene. This technology, combined with true correlated double sampling, enables HD imagers to achieve readout noise below 2E–RMS and quantum efficiency (QE) of over 60%.

CMOS Image Sensors Cannibalize CCD Market Share in Digital Still Camera (DSC) Market
As in other application areas, CMOS image sensors are migrating from low-end products to high-end products in the digital still camera market. At the same time, OEMs have found that the technical advantages provided by CMOS imagers over CCDs often translate into competitive advantages in the market. Specifically, the image quality produced by contemporary CMOS sensors has reached or even exceeded that of CCDs, and in the DSC market, image quality is a key product differentiator.

Here is an example of a 14MP, 1.4μm, 1/2.3" MT9F002 sensor from Aptina integrated into a leading brand digital camera and another leading brand digital camera equipped with a 14MP, 1/2.3" CCD device, both with the same array and pixel size. By analyzing key performance criteria, the 14MP CMOS sensor clearly outperforms the corresponding CCD device, both at low and high ISO conditions.

Figures 2 and 3 show a performance comparison between the two sensors at ISO 100. At this minimum ISO (minimum gain) condition, the CMOS sensor shows a 1.5dB increase in image luminance signal-to-noise ratio (SNR), resulting in lower noise.

Figure 2: Image comparison between Aptina's 14-megapixel CMOS image sensor (MT9F002) and its CCD counterpart at ISO 100

Figure 3 Comparison of enlarged image details at ISO 100

Figures 4 and 5 show that at the other extreme of the ISO range (ISO 1600), CMOS image sensors also outperform CCD devices.

As can be seen from Figure 4, under high ISO conditions, the CMOS sensor has a brightness signal-to-noise ratio increase of more than 2dB (greater than 30%). By zooming in on different areas of the image (see Figure 5), CMOS image sensors are not only comparable to CCD devices, but also have higher image quality under some conditions.

Figure 4: Image comparison between Aptina’s 14MP sensor (MT9F002) and its CCD counterpart at ISO 1600

Figure 5 Comparison of images of different magnified areas at ISO 1600

In addition to being able to capture better photos, some new CMOS image sensors are capable of output rates of 200MP/s, which is exactly the speed required for 60 frames per second full HD video. The next generation of sensors under development use a high-speed parallel readout architecture and are expected to have output rates exceeding 500MP/s. With faster readout speeds, CMOS sensors can capture both high-definition still images and high-quality full HD video at the same time, without requiring designers to compromise between still image and full HD video performance. CMOS sensors provide optimized performance for both still images and video, making it possible for OEMs to develop new and differentiated DSC products.

Conclusion
In various market sectors, CMOS image sensor technology has been continuously improving and evolving, and they have gone far beyond the goal of meeting the specific requirements of the market segments. In each application, CMOS technology not only has higher imaging performance, but also enables OEMs to develop new features and achieve product differentiation in the market. As CMOS image sensors gain more imaging market share, CMOS technology will surely have greater development momentum and more innovation to adapt to the future development trends and requirements of the imaging market.