Description
1 Demosaicing
Demosaicing is a process of interpolating the red, green and blue channels from the raw pixel
image. Here we will try to ll in the missing pixel values from the sampled raw images through
interpolation. Kindly zoom in and view RawImage1.mat to get an idea on how raw image looks
like.
e CFA for ‘RawImage1.mat’ is ‘RGGB’ whose mask is provided in the le ‘bayer1.mat’. In
‘bayer1.mat’ a value of 1 at a particular pixel means that only ‘red’ value is sampled at that pixel.
Similarly values of ‘2’ and ‘3’ mean that only ‘green’ and ‘blue’ values are sampled at those pixels.
e task here is to reconstruct the full colour image from the undersampled raw sensor data. Two
popular ways of lling in missing data on a 2D grid are bilinear and bicubic interpolation. Here,
we will try both the methods and compare the results.
Note that you have to interpolate only the missing values in the grid of pixels. For instance,
at a pixel where the channel ‘blue’ is already sampled you need to interpolate only the ‘red’ and
‘green’ values from the image grid. Use Matlab built-in function griddata to query all missing
pixel values at once, instead of querying it patchwise.
a) Perform bilinear interpolation of the missing pixels in each color channel R, G and B to
reconstruct a full color image from RawImage1 with the CFA provided in ‘b1.mat’ [5 marks]
b) Perform bicubic interpolation of the missing pixels in each color channel R, G and B to
reconstruct a full color image from RawImage1 with the CFA provided in ‘b1.mat’. Compare the reconstructed full colour image with that obtained using bilinear interpolation. [3
marks]
c) Do demosaicing for RawImage1.mat using paern ‘rggb’ with matlab built-in function ‘demosaic’ and compare with previous two reconstructed images [2 marks]
d) What assumptions are you making while interpolating the missing pixel values and what
will happen when the assumptions do not hold? [2 marks]
e) Now demosaic the image ‘kodim19.mat’ which has been sampled with the CFA paern
‘RGGB’ (provided in kodim cfa.mat). For reference the true colour image ‘kodim19.png’ is
given in the data folder. Compare the demosaiced image with the actual image and comment
your observations. (e image ‘kodim19.png’ is obtained from Kodak dataset) [3 marks]
2
2 White Balancing and Tone Mapping
Next process in the pipeline is white balancing. Here you will do white-balancing for all three
raw images RawImage1, RawImage2 and RawImage3 in each subsection. Note that you have
to perform demosaicing for RawImage2 and RawImage3 before white-balancing. e CFA for
RawImage2 is ‘grbg’ (bayer2.mat) and for RawImage3 it is ‘rggb’(bayer3.mat). Use the code from
the previous problem to demosaic the images RawImage2 and RawImage3.
ere are basically three ways to do White-balancing –
a) Assume the average color of the scene to be gray and then do white balancing. [2 marks]
b) Assume that the brightest pixel to be specular highlight and should therefore be white. Use
pixel coordinate (x, y) =
i) (830, 814) for RawImage1 [1 mark]
ii) (1165, 280) for RawImage2. [1 mark]
iii) (175, 675) for RawImage3. [1 mark]
c) Assume that some part of the scene to be neutral. Use pixel coordinate (x, y) =
i) (2000, 435) for RawImage1 [1 mark]
ii) (445, 715) for RawImage2. [1 mark]
iii) (1550, 565) for RawImage3. [1 mark]
d) Perform tone mapping on each of the above images using the following two methods:
i) Histogram equalization [2 marks]
ii) Gamma correction for the gamma values of 0.5, 0.7 and 0.9 [3 marks]
3 Image denoising
Here in this part you will use bilateral ltering operation for denoising. e bilateral lter is
controlled by two parameters σs and σr. As we know, σs controls the spatial weighing whereas
the σr controls the alignment of image intensities. Ideally we would like to locally estimate the
noise and adjust both the parameters accordingly but given the cost here we would go with single
3
set of parameters. To estimate the noise we will choose a constant region in the image (coordinates
are given below) and calculate the standard deviation along all the three channels R, G and B. Now
denoise individual channels with window size as 11, σs as 2.5 and for σr = 1.95×σn, where σn is
noise standard deviation in the corresponding colour channel. For the bilateral ltering, use the
supplied function blter2.m.
For RawImage2 use the rectangular region with coordinates: top-le (705,924) and boomright (765,984) in (row,column) order. For other raw images choose a region on your own and do
the denoising as mentioned above. [5 marks]
3.1 estions:
1. You have to construct a Gaussian blurring kernel with a variance σ for denoising an image.
As you know, Gaussian function has innite support. In order to implement tractably, you
need to truncate the Gaussian function to a nite size. For a given Gaussian kernel with
variance σ, at what point will you choose to truncate the function and why? [2 marks]
2. For a given σn how would you choose the value for σr? What would happen if σr ≤ σn or
σr >> σn? [3 marks]
4
