IMAGE SIGNAL PROCESSOR (ISP)

Once upon a time, you needed a dedicated camera to take pictures. Now with the help of your smartphone, you can take excellent photos. How? With a little something called the Image Signal Processor.

Nearly every smartphone has a camera today. It is one of the very important parts of a phone.

A lot of phone users, myself included cannot do without a one on a phone. Taking photos today, when compared to twenty years ago, seems so simple today.

You take out your phone and point and shoot. When you want to take a photo, you open the app, aim it at the subject, and click on the shutter button. In a few seconds, the photo appears on your screen. You can choose to either keep it or get rid of it and take another one.

What is an Image Signal Processor?

Within those few seconds between taking the photo and the subsequent picture appearing on your screen, there is a lot of computing and processing that takes place on the phone. These are all handled by the part of the SoC known as the ISP (Image Signal Processor).

An ISP also known as an Image Processor (IP), Digital Signal Processor (DSP) or Image Processing Unit (IPU) is a type of specialized microprocessor used for image processing.

The ISP helps to transform the light image captured by the pixels on the camera sensor into a RAW image format. It then processes the photo further into the compressed final jpg format that you see.

How the ISP works

The ISP has a series of computation processes which are referred to collectively as a Pipeline. When a photo is taken, light bouncing off the subject(s) enters the camera and is captured by the pixels on the image sensor. The pixels then convert the light energy into electrical energy and then send the data to the ISP. The ISP then carries out demosaicing on the photo and assigns each pixel a color value. After this, it then gives out an uncompressed RAW image.

Significant processing is required to convert a RAW image into the final JPG product. Different companies use different software to try to achieve this but the process remains the same. They include:

i. Denoising (noise reduction)
ii. Auto white balance and color correction
iii. Lens shading (improves brightness and color non-uniformity)
iv. Pixel correction (corrects defective pixel output)
v. Tone mapping (for increased contrast) and HDR
vi. Focus correction and image sharpening
vii. Auto exposure adjustment

Post-processing

As said earlier, different companies approach photo processing differently. Some may choose to go with warmer colors and a high level of exposure and saturation, others could go with cooler colors and a moderate level of exposure as well as saturation. Some could go for a sharp image or a dull smooth one.

Types Of Image Signal Processor

There are a lot of ISP products in the market but for this post, we’re only going to be looking at the ISP used in smartphones by SoC manufacturers as well as OEMs (phone manufacturers).

• Kirin ISP: manufactured by HiSilicon and used in Kirin SoCs exclusively for Huawei phones. It is very capable and in combination with the cameras used on such phones has made Huawei a photography giant in the smartphone market.

• Qualcomm Spectra (and Hexagon DSP):

Qualcomm Spectra is the ISP that ships with Snapdragon chipsets. To boost its performance, an extra module for AI and machine learning called Hexagon DSP is added as well making them very capable, fast, and able to handle cameras of very high resolutions.

• Exynos Dual ISP:

is made by Samsung for their Exynos chipsets which are used on Exynos variants of the flagships as well as other Samsung phones shipping with an Exynos SoC.

• Google Visual Core:

Google initially used Qualcomm’s ISP for their phones but with the release of Pixel 2, Google designed the Pixel Visual core as the main ISP for the device and continued to use it till it was replaced by the upgraded Pixel Neural core used in the Google Pixel 4. Both versions of Google’s ISP are very good and combined with Google’s Computational Photography have made Pixel phones one of the best in terms of photography when smartphones are being discussed.

• Mediatek Imagiq:

is MediaTek’s response in the camera arms race among SoC and Smartphone makers. It also offers support for multiple cameras and good pixel correction as well as zoomed-in images with little distortion.

• Sony Bionz X:

BIONZ X for mobile is an image-processing engine based on Sony’s digital imaging technologies. It is used in Alpha and Cyber-shot cameras. BIONZ X includes RAW noise reduction, a technology that offers a better-than-ever low-light shooting performance. It reduces the noise of the image before the compression so that the final picture is smoother and clearer.

• HTC Image Sense:

used on HTC devices and helps with image processing and fine-tuning.

Final note:

The grade of the SoC used (i.e. Flagship, mid-range or budget, etc.) dictates the level of the ISP that comes with it and that translates into the resolution of the camera that it can support.

A low-end SoC can only support poor-resolution camera sensors because it may lack the processing power needed to handle high-resolution photos.

A high-end processor should come with an excellent ISP. Thus, it should have zero problems processing neat photos.

So low-end SoC = weak ISP = low-resolution cameras = average or poor photos. Mid-range SoCs should equal = decent ISP = decent resolution cameras = good photos. Flagship SoCs…well I don’t need to explain that do I?

Please leave a comment if you have any difficulty and remember to:

• Subscribe to our YouTube channel
• Follow on Facebook
• Follow on WhatsApp
• Join our Telegram community
• Participate on Reddit
• Find us on Quora