Synopsys Cloud
Cloud native EDA tools & pre-optimized hardware platforms
Request a Free Trial →
CMOS Image Sensor Pixel Correction with OTP NVM IP
Krishna Balachandran, Product Marketing Manager Senior Staff, Synopsys
Introduction
Remember your astonishment when you first saw a smartphone that included a camera? It was first introduced in Japan by Kyocera in 1999. Called the Kyocera VP-210, it had one front-facing camera with 0.11 Megapixels. The phone supported taking and sending pictures and making video calls and it could store 20 photos before it reached the limit of its storage capacity. Fast forward to 2021 and it is commonplace to find smartphones with multiple cameras (front, rear, ultra-wide angle, and telephoto) with resolutions as high as 108 Megapixels made possible by CMOS image sensors (CIS).
Figure 1: CIS (Source: https://semiengineering.com/cmos-image-sensors-cis-past-present-future/)
Smartphones represent about 70% of the CIS market. Smartphone manufacturers meet consumer demand for increased photo quality by offering higher resolutions and pixel scaling. In turn, this has led to a need for memory solutions that can compensate for sensor variations and correct pixel distortions. One-time programmable (OTP) non-volatile memory (NVM) has become the leading choice to address these needs in smartphone cameras.
Smartphones from two decades ago relied on charge coupled devices (CCDs) to capture light and convert it into electrical signals. CCD-based image sensors perform well across a range of lighting conditions and are less susceptible to noise but are not area efficient and are expensive to mass produce because of the need for custom manufacturing steps. Contrast that to today’s smartphones, which inevitably use CIS because of their lower manufacturing cost, smaller size, and low power consumption. CIS are popular for use in a variety of cameras including surveillance and medical diagnostic equipment, industrial production line monitoring, robotics, autonomous driving, and imaging for military and aerospace applications.
Working Principle of a CMOS Image Sensor
A CIS consists of a micro-lens array layer responsible for focusing the light into the chip; a color-filter layer to separate out the red, green, and blue (RGB) components of light; and an active pixel array, consisting of photodiodes that convert the light into electrical energy and amplification circuitry (Figure 2).
Figure 2: CMOS Image Sensor architecture (Image source: semiengineering.com)
Light absorption and image fidelity depend on the size of the photodiode and the distance they are placed apart. The demand for better picture quality is fulfilled by increasing resolution. An obvious way to increase resolution is to pack as many photodiodes as closely as possible in a chip without exploding the die size and hence the cost of the image sensor. Using a photodiode that is too small reduces its ability to convert light into an electrical signal. If photodiodes are placed in proximity, their currents collide and create crosstalk, destroying the fidelity of the captured image. The distance between two pixels, called pixel pitch, defines the number of pixels that can be packed into a CIS chip, which determines image resolution.
Pixel Scaling Enabled by Innovations
Initial CIS designs used front side illumination (FSI) in which light entered the chip through the micro-lens and made its way to the photodiode. This technique ran out of steam when pixel pitches neared 1.4 microns. Pixel scaling continued with the introduction of design and process innovations with the current state-of-the-art being 0.7 microns. The innovations can be broadly summarized as:
- Design innovation in the form of back side illumination (BSI) in which the image sensor structure is flipped on its back and light enters the silicon substrate holding the photodiode. In this technology, the shorter path to the photodiode increases the efficiency of light absorption.
- Process innovation called deep trench isolation (DTI) to make the photodiodes taller as they shrink with process migration to smaller geometries and better isolation between them to prevent or reduce crosstalk.
Figure 3: Front-side illumination is being replaced by BSI to increase light absorption efficiency (Image source: semiengineering.com)
Figure 4: Small Pixel Scaling, DTI Structures (Image source: techinsights.com)
Once pixel pitches approach the wavelength of visible light, which ranges from 0.4 to 0.7 microns, interference from neighboring pixels becomes more disruptive, making pixel scaling a daunting challenge. However, innovations in packaging and interconnect technology are already pushing pixel scaling to advance to be in within the range of the wavelength of the visible spectrum of light.
However, pixel scaling will continue to meet the demands for increased pixel resolution using new techniques. For example, special layouts that introduce narrow metal grids and narrow dielectric wells can suppress crosstalk and enable pixel pitches approaching or going below the wavelength of the visible spectrum. The special layouts require close collaboration between the CIS designers and semiconductor manufacturing companies, leading to process tweaks and design rules to be able to manufacture CIS using a “standard” CMOS process. In effect, foundries that support CIS designs usually work with CIS designers to create and enforce CIS-specific design rules and process tweaks. In addition to working with the CIS designers, the foundries work with major third-party IP vendors and share the rules via a custom Physical Design Kit (PDK).
Connecting the CIS and ISP
Even with the best CIS design and manufacturing, images captured by the sensor array may suffer from blurring, unnatural colors, or other problems due to aberrations caused by the camera’s lens limitations or the picture taker’s abilities. Some common problems are an unrealistically looking white color in photos, corrected by white balance settings; and the picture looking bleached or dark, compensated by gamma correction.
To fix these issues, designers use image signal processors (ISPs). The ISP is either integrated with the CIS, or a companion chip to the CIS, and does the heavy lifting of processing the information captured by the sensor array and converting it into a good-looking image by correcting for defective pixels, poor lighting conditions, color aberrations, white balance, and providing gamma correction.
Process Technology Scaling for CIS & ISP
Process technology scaling ensures cost-effective improvement of pixel resolution because more pixels can be packed in a smaller area. If the CIS, which performs the sensing function and the ISP, which performs the processing function, are combined on the same chip, process technology scaling is limited by the CIS component of the total solution. That is because while the ISP is predominantly digital and scales well with process technology scaling, the CIS has a lot of analog circuitry that doesn’t scale well making the move to smaller geometries cost ineffective.
Figure 5: Conventional sensor structure (left) vs Stack sensor structure (right)
(Image source: image-sensors-world.blogspot.com)
One notable trend to keep costs low and move the needle on resolution is to use a stacked-die approach, in which the image processing functions are separated from the image sensing. This was pioneered by Sony1 and adopted by other CIS market leaders. In stacked-die systems, the ISP is manufactured in a more advanced CMOS process than the image sensor. With this approach, the industry has been able to address the thirst for higher resolution at lower costs, a hallmark of the smartphone market.
Role of Embedded OTP NVM in CIS
Even the best manufacturing processes will introduce manufacturing variations, which in turn cause inefficient light capture and subsequent image distortion. To correct image distortions, you need a memory to store the locations of defective pixels and trimming information for the sensor array. Other parameters may also be stored to facilitate good image quality. Since the location, trimming, and other values need to be read and used every time the CIS chip is powered up, the information must be stored in persistent embedded memory or embedded NVM. The values need to be programmed only once in the manufacturing flow before the product is shipped, making embedded OTP NVM an ideal choice.
OTP NVM is also an excellent choice to store information in the ISP for exposure control, gain control, white balance, lens correction and defective pixel correction.
Given the sensitive nature of CIS chips, SoC designers must ensure that their OTP NVM was designed carefully with the same considerations for noise suppression, and must follow the special design requirements adhering to the custom PDK.
Military, aerospace, industrial, healthcare, financial, and other security conscious applications require the OTP NVM to be secure because this information as well as other end-user information must be protected from hackers intending to gain control of a camera remotely and hijacking it for their purposes.
Scalable, Reliable, Secure OTP NVM IP for CIS Applications
CIS applications chasing the ever-shifting resolution boundary need to achieve pixel scaling by continuously moving to more advanced process geometries. Cameras used in smartphones, automotive, and high-resolution imaging in space and medical applications can’t compromise on reliability and security. Finally, cameras used in price-sensitive end-applications like smartphones are PPA conscious.
Antifuse OTP NVM, which went into widespread commercial production about 10 years ago, are fundamentally reliable and secure. No special masks or process steps are required to manufacture the NVM device. In a standard CMOS process, the antifuse OTP NVM uses the same rules as logic devices for electrical and layout design, providing scalability at the most advanced nodes, down to 5-nm and beyond.
The underlying technology for antifuse OTP NVM is programming by oxide breakdown, which is achieved by applying a high voltage. Absence of the high voltage on the gate leaves the device unprogrammed. Since no changes to the manufacturing process are required, the antifuse OTP NVM reaps the benefit of the same yield and reliability as a standard CMOS process.
In harsh environments for automotive and industrial applications, reliability is another key requirement. When properly qualified at high operating temperatures of up to 150°C and even 175°C, and tested for early life failure rates, antifuse OTP NVM is the best choice for reliable operation for the life of the product.
Antifuse OTP NVM is not easily susceptible to any passive or invasive security attacks attempted by altering the voltage or temperature. The oxide breakdown is also not visible with a scanning electron microscope (SEM) and it is impossible to tell the difference visually between a programmed and an unprogrammed cell within an antifuse OTP NVM.
Antifuse OTP NVM that is specially designed for CIS applications delivers on all fronts: it follows the layout and design rules specific for CIS manufacturing, is cost-effective, consumes minimal power, and provides fast read times at system boot.
DesignWare OTP NVM IP for CIS Applications
Built on antifuse technology, Synopsys DesignWare® OTP NVM IP offers a secure and flexible embedded memory solution for pixel correction in standard CMOS without requiring additional process or mask steps. Its patented technology offers a small silicon footprint, and its built-in security features protect against active and passive attacks, tampering, hacking, and reverse engineering. The customizable macros allow design flexibility for pixel correction in CIS applications requiring storage of trimming, calibration, and configuration information in a secure fashion. Qualified for automotive AEC-Q100 grade 0 and 1 temperatures, the DesignWare OTP NVM IP accelerates SoC-level development even for harsh automotive environments. It is available across multiple foundries and process nodes. Synopsys’ antifuse OTP NVM IP is in mass production in a broad range of systems including CMOS image sensors.
1“Sony announced the world’s first stacked chip CIS camera system for consumer electronics in 2012, and 8 MP ISX014 stacked chips were found in a tablet computer by early 2013”
https://semiengineering.com/cmos-image-sensors-cis-past-present-future/