Description
1. Consider the following motion models: translation, rigid (translation + rotation), rigid and equal scaling in
X,Y directions, rigid and unequal scaling in X,Y directions, affine, non-rigid. Consider each of the following
applications separately.
In each case, identify what is the optimal motion model and justify. Do not needlessly
pick a more complex motion model if it is not needed. For example, in some cases, a rotation is enough, in
which case please do not choose affine as it has additional degrees of freedom. [15 points]
(a) Consider that you have scanned a document twice with the same scanner, when the document was
potentially in slightly different positions. You now want to align these two images. Which motion
model is needed here? Assume there is no stretching or bending of the paper.
(b) In the earlier example, consider that the two images were respectively acquired from two different
scanners with different resolutions. Assume that for both scanners, the X and Y resolutions were the
same. Which motion model is needed here? Assume there is no stretching or bending of the paper.
(c) Consider a document with words written on both sides with ink. When you scan such a document from
one side, some portions from the other side are visible. This is called ‘ink bleeding’. To remove bleeding
artifacts, you need to acquire images of both sides of the document and first align them. Which motion
model is needed here? Assume there is no stretching or bending of the paper.
2. You are viewing the following graph from a research paper. Unfortunately, from the graph, the (x, y) values
at only a few points can be observed. You need to obtain the (x, y) values at many other points. Hence you
can do the following: you extract the image from the paper, and open it through MATLAB which provides a
function called impixelinfo.
This function gives you the (x, y) coordinates of any spatial location pointed by
your mouse. However, the coordinate system of the graph and that of MATLAB will be different. Describe
a procedure to convert from MATLAB’s coordinate system to the coordinate system of the graph. This will
help you obtain the (x, y) coordinates in the coordinate system of the graph. Support your answer with
suitable equations. There is no need to write any code for this. [15 points]
3. Consider two images I and J whose intensity values (in each location) are randomly drawn from the known
probability mass functions (PMFs) pI (i) and pJ (j) respectively. Derive an expression for the PMF of the
image I + J. The expression resembles which operation that we are currently studying in class ? (will be
studying very soon) [15+5=20 points]
4. Read in the images T1.jpg and T2.jpg from the homework folder using the MATLAB function imread and
cast them as a double array. Let us call these images as J1 and J2. These are magnetic resonance images
of a portion of the human brain, acquired with different settings of the MRI machine.
They both represent
the same anatomical structures and are perfectly aligned (i.e. any pixel at location (x, y) in both images
represents the exact same physical entity). We are going to perform a simulation experiment for image
alignment in a setting where the image intensities of physically corresponding pixels are different.
To this
end, do as follows:
(a) Write a piece of MATLAB code to rotate the second image by θ = 28.5 degrees anti-clockwise. You
can use the imrotate function in MATLAB to implement the rotation using any interpolation method.
Note that the rotation is performed implicitly about the centroid of the image. While doing so, assign
a value of 0 to unoccupied pixels. Let us denote the rotated version of J2 as J3.
(b) Our job will now be to align J3 with J1 keeping J1 fixed. To this end, we will do a brute-force search
over θ ranging from -45 to +45 degrees in steps of 1 degree. For each θ, apply the rotation to J3 to
create an intermediate image J4, and compute the following measures of dependence between J1 and
J4:
the normalized cross-correlation (NCC)
the joint entropy (JE)
a measure of dependence called quadratic mutual information (QMI) defined as P
i1
P
i2
(pI1I2
(i1, i2)−
pI1
(i1)pI2
(i2))2
, where pI1I2
(i1, i2) represents the normalized joint histogram (i.e., joint pmf) of I1
and I2 (‘normalized’ means that the entries sum up to one). Here, the random variables I1, I2
denote the pixel intensities from the two images respectively.
For computing the joint histogram,
use a bin-width of 10 in both I1 and I2. For computing the marginal histograms pI1
and pI2
, you
need to integrate the joint histogram along one of the two directions respectively. You should write
your own joint histogram routine in MATLAB – do not use any inbuilt functions for it.
(c) Plot separate graphs of the values of NCC, JE, QMI versus θ and include them in the report PDF.
(d) Determine the optimal rotation between J3 and J1 using each of these three measures. What do you
observe from the plots with regard to estimating the rotation? Explain in the report PDF.
(e) For the optimal rotation using JE, plot the joint histogram between J1 and J4 using the imagesc
function in MATLAB along with colorbar. Include it in the report PDF.
(f) We have studied NCC and JE in class. What is the intuition regarding QMI? Explain in the report PDF. (Hint: When would random variables I1 and I2 be considered statistically independent?)
[2+10+2+3+3+5=25 points]
5. Read in the images ‘goi1.jpg’ and ‘goi2.jpg’ from the homework folder using the MATLAB imread function
and cast them as double. These are images of the Gateway of India acquired from two different viewpoints.
As such, no motion model we have studied in class is really adequate for representing the motion between
these images, but it turns out that an affine model is a reasonably good approximation, and you will see
this.
We will estimate the affine transformation between these two images in the following manner:
(a) Display both images using imshow(im1) and imshow(im2) in MATLAB. Use the ginput function of
MATLAB to manually select (via an easy graphical user interface) and store n = 12 pairs of physically
corresponding salient feature points from both the images.
For this, you can do the following:
for i=1:12, figure(1); imshow(im1/255); [x1(i), y1(i)] = ginput(1);
figure(2); imshow(im2/255); [x2(i), y2(i)] = ginput(1);
Tips: Avoid selecting points which are visible in only one image. Try to select them as accurately as possible, but our procedure is robust to small sub-pixel errors. Make sure x1(i),y1(i) and x2(i),y2(i)
are actually physically corresponding points. Salient feature points are typically points that represent
corners of various structures.
(b) Write MATLAB code to determine the affine transformation which converts the first image (‘goi1’) into
the second one (‘goi2’).
(c) Using nearest neighbor interpolation that you should implement yourself, warp the first image with the
affine transformation matrix determined in the previous step, so that it is now better aligned with the
second image. You are not allowed to use any implementation for this already available in MATLAB.
Display all three images side by side in the report PDF.
(d) Repeat the previous step with bilinear interpolation that you should implement yourself. You are not
allowed to use any implementation for this already available in MATLAB. Display all three images side
by side in the report PDF.
(e) In the first step, suppose that the n points you chose in the first image happened to be collinear. Explain
(in the report PDF) the effect on the estimation of the affine transformation matrix. [5+5+5+5+5=25
points]
