$30.00 $18.00
Introduction
In this homework, we will be exploring the usefulness of homographies. Robots often deal with
planes, whether in the form of walls, ground, or some other flat surface. When two cameras
observe a plane, there exists a relationship between the images captured. This relationship is
defined by a 3 × 3 transformation matrix, called a planar homography.
A planar homography allows us to compute how a planar scene would look from second
camera location, given only the first image. In fact, we can compute how images of the plane
will look from any camera at any location without knowing any internal camera parameters
and without actually taking the pictures, all with the planar homography.
Suppose we have two cameras C1 and C2 looking at a common plane Π in 3D space. Any
3D point P on Π generates a projected 2D point located at p1 ≡ (x1, y1, 1)T on the first
camera C1 and p2 ≡ (x2, y2,1 )
T on the second camera C2. Since P is confined to the plane
Π, we expect that there is a relationship between p1 and p2. In particular, there exists a
common 3 × 3 matrix H, so that for any p1 and p2, the following conditions holds:
p1 ≡ Hp2 (1)
We call this relationship planar homography. Recall that both p1 and p2 are in homogeneous
coordinates and the equality ≡ means p1 is proportional to Hp2.
1 Planar Homographies: Theory (40pts)
The Q1 questions should not take more than a couple of lines each. In particular,
the “proofs” do not require any lengthy calculations. If you are lost in many lines
of complicated algebra you are doing something much too complicated (or wrong).
Question 1.1 (10pts)
Prove that there exists an H that satisfies Equation 1 given two 3 × 4 camera projection
matrices M1 and M2 corresponding to cameras C1, C2 and a plane Π. You do not need to
produce an actual algebraic expression for H. All we are asking for is a proof of the existence
of H. Note: A degenerate case happens when the plane Π contains both cameras centers, in
which case there are infinite choices of H satisfying Equation 1. You can ignore this special
case in your answer.
Question 1.2 (10pts)
Prove that there exists an H that satisfies Equation 1 given two cameras separated by a pure
rotation. That is, for C1, p1 = K1[I 0]P and for C2, p2 = K2[R 0]P . Note that K is not
the same for the two cameras.
Question 1.3 (5pts)
Suppose that a camera is rotating about its center C, keeping the intrinsic parameters K
constant. Let H be the homography that maps the view from one camera orientation to the
view at a second orientation. Let θ be the angle of rotation between the two. Show that H2
is the homography corresponding to a rotation of 2θ.
2
Question 1.4 (5pts)
Why is the planar homography not sufficient to map any arbitrary scene image to another
viewpoint? State your answer concisely in one or two sentences.
Question 1.5 (10pts)
We have a set of points p
i
1
in an image taken by camera C1 and points p
i
2
in an image taken
by C2. Suppose we know there exists an unknown homography H such that
p
i
1 ≡ Hpi
2
(2)
Assume the points are homogeneous coordinates in the form p
i
j = (x
i
j
, yi
j
, 1)T
. For a single
point pair, write a matrix equation of the form
Ah = 0 (3)
Where h is a vector of the elements of H and A is a matrix composed of the point coordinates.
1. How many elements are there in h?
2. How many point pairs are required to solve this system?
3. Show how to estimate the elements in h using the Rayleigh Quotient Theorem mentioned
in class to find a solution to minimize this homogeneous linear least squares system.
2 Planar Homographies: Implementation (30pts)
Question 2.1 (15pts)
Implement a function H2to1=computeH(p1, p2). Inputs: p1 and p2 should be 2×N matrices
of corresponding (x, y)
T
coordinates between two images. Outputs: H2to1 should be a 3 ×
3 matrix encoding the homography that best matches the linear equation derived above in
Equation 2 (in the least squares sense). Hint: Remember that a homography is only determined
up to scale. The Matlab function eig will be useful.
Question 2.2 Mom! I am on TV!! (15pts)
Have you been to a Pirates game at PNC park? Have you ever wondered how cool it would
be to have your face show up on the big screen as shown in Figure 1a1
? With what you have
learned so far, this is no longer a fantasy. Assuming the the big screen is planar (which it most
likely is), there exist a homography between the plane of the big screen and any (planar) image.
Therefore, if we are able to find this homography, then we can map an image on your computer
to the big screen as shown in Figure 1d. Please do the following:
a) Create an image with at least your name on it. You can modify the provided
pnc tomap.jpg as shown in Figure 1b. Please save this image as pnc tomap.jpg.
1Photo taken by Shoou-I Yu, Padres v.s. Pirates, 8/10/2012.
3
b) Find the point correspondences p1 and p2 (2×N matrices) between the big screen of PNC
park and the image with your name. You can use Matlabs cpselect to select matching points
on each image. Hint: What is the minimum amount of point correspondences required to
compute a homography? Which point correspondences should you choose? Please save p1
and p2 in Q4.2.p1p2.mat with this command: save(Q4.2.p1p2.mat, p1, p2).
c) Implement a function:
[img_yourname_warped, img_PNCpark_yourname] =
warp2PNCpark(img_PNCpark, img_yourname, p1, p2)
which takes Figure 1a (img PNCpark) and Figure 1b (img yourname) as input,
and outputs Figure 1c (img yourname warped) and Figure 1d (img PNCpark yourname).
The point correspondences p1 and p2 are the ones you found in the previous step. Apply
your computeH to compute a homography H.
Then use the provided warpH function to warp the image with your name to the coordinates
of the big screen of PNC park as shown in Figure 1c.
warp im=warpH(im, H, out size, fill value)
warps image im using the homography transform H. In this function, the pixels in warp im
are sampled at coordinates in the rectangle (1, 1) to (out size(2), out size(1)). The
coordinates of the pixels in the source image are taken to be (1, 1) to (size(im,2),
size(im,1)) and transformed according to H. The empty regions of warp im are filled
with fill value. Setting an appropriate fill value can help you quickly find empty
regions. The final step will be to combine the output of warpH with img PNCpark to get the
final output.
A common problem: There are different data types to represent images for Matlab.
When you read an image with imread, the data type used is usually uint8. The range of
values for uint8 is 0 to 255, so negative numbers are not available for the uint8 type. If you
want to have negative numbers in your matrix, then you need to use a data type of single or
double. However, if you are using single or double typed images, then for imshow to display
correctly, the values of the image should be between 0 and 1. On the other hand, if you
want to use imshow to view an uint8 image, the values of the image should range from 0 to
255. You can convert between single and uint8 data types with the functions im single =
single(im) and im uint8 = uint8(im).
d) Run the provided q42checker.m. The q42checker.m makes sure that the format of the
functions and files follow the specifications of this homework. Please make sure that your
code can run through the checker without any errors. The checker will generate an image
with a structure very similar to Figure 1d. There will be additional crosses on the images
in the top row showing the point correspondences p1 and p2. Please include this image
in your PDF submission. Make sure that the resolution is HIGH ENOUGH (at
least 1280 x 908) for the TAs to clearly see the image.
3 Panoramas (30pts)
We can also use homographies to create a panorama image from multiple views of the same
scene. This is possible for example when there is no camera translation between the views
(e.g., only rotation about the camera center), as we saw in Question 1.2. You will need to
use the provided warpH function again which you used in Question 2.2(c). Use the provided
q5checker.m, and make sure that your code runs without any error.
4
(a) Big screen showing players name
(b) Image with your name
(c) Image with your name warped (d) Big screen with Shoou’s name
Question 3.1 (15pts)
For this problem, use the provided images taj1r.jpg and taj2r.jpg along with the data in
tajPts.mat. Use Matlabs imread function to load an image and the load function to load a
mat file. tajPts is a 4 × 1048 matrix containing the coordinates of features points in taj1 in
the first two rows and matching feature points for taj2 in the bottom two rows.
You can visualize a sample of these correspondences using plotMatches(img1,img2,pts)
Within a few weeks, we will see how to automatically generate them.
Figure 2: Matched features from tajPts.mat.
a) Apply your computeH using these correspondences to compute H2to1, which is the homography from taj2 onto taj1. To validate your result, try to warp all the feature points in taj2
to the image coordinate of the taj1. And check the error between corresponding points by
calculating the 2D distance between them. Save H2to1 as q5 1.mat using Matlabs save function. Save all the transformed feature points of taj1 by H2to1 as q5 1 warpedFeatures.mat
using Matlabs save function. Note that this should be a 2 × 1048 matrix. Also, write the
Root Mean Squared Error between corresponding points in your PDF submission.
5
Figure 3: Original images and warped
(a) taj1.jpg (b) taj2.jpg (c) taj2 warped into taj1
b) Apply the computed homography H2to1 to taj2 using warpH. Note that since the warped
image will be translated to the right, you will need a larger target image. Use outSize
= [size(taj1,1),3000]; Save this figure as q5 1.jpg. Produce a simple panorama by
laying taj1 on top of the warped taj2. Save this figure as q5 1 pan.jpg. Save your code in
a Matlab function [H2to1,warpedImg,panoImg] = q5 1(img1, img2, pts) which accepts
the images and feature points and returns homography, warped image, and panorama image.
Question 3.2 (15pts)
Suppose we want to display all the visible pixels in the panorama. Write a few lines of Matlab
code to find a matrix M, and out size in terms of H2to1 and size(im1) and size(im2) such
that,
warp_im1=warpH(im1, M, out_size);
warp_im2=warpH(im2, M*H2to1, out_size);
produces images in a common reference frame where all points in im1 and im2 are visible
and out size(2)=1280 (out size(1) should be determined by out size(2)). The matrix M
should perform scaling and translation only, and the resulting image should fit snugly around
the corners of the panorama. Do not select M manually. It should be computed by your code,
considering the size of the panorama image. Save this image as q5 2 pan.jpg. Save the code
used to generate and display the panorama as
[H2to1, panoImg] = q5 2(img1, img2, pts)
where img1 is taj1 and img2 is taj2.
6
Figure 4: Final mosaic view. Border lines are only for reference and need not be present in
your solution.
7
WhatsApp us