Description
1. Introduction
This assignment involves Texture Synthesis and Image Inpainting . The goal is to implement and evaluate the
idea of Texture Synthesis by Non-Parametric Sampling proposed by Efros and Leung, ICCV99, and region
filling by Criminisi et al “Region Filling and Object Removal by Exemplar-Based Image Inpainting”, TIP
2004
1.1. Paper and Code
Efros and Leung paper can be found at:
http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html
A pseudo-code for their work is provided in:
http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.html
The algorithm is also described in Forsyth and Ponce textbook in Chapter 6, and in Szeliski’s textbook in
Section 10.5.
Criminisi’s paper can be found in http://research.microsoft.com/apps/pubs/default.aspx?id=70029
2. Main Task
2.1. Preliminaries
You will be given five images that you are going to use for synthesis.
Download the test images from Sakai resources
2.2. Implementation and Experiments
2.2.1. Texture Synthesis
Write a Python code, implementing Efros and Leung’s approach, to synthesize a 200×200 pixel image for
each of the five example images T1,T2,…, T5.gif. Show synthesized images for T1,T2,…, T5.gif by changing
the WindowSize parameter to 5, 9,11 pixels around the center.
2.2.2. Image Inpainting
Use your code (with modification if necessary) to fill in the gaps (black regions) in the images test_im1.bmp,
test_im2.bmp. Try different parameter settings 5, 9, 11, and others if necessary, to see which one give you the
best results. Comment about it in your report.
2.2.3. Object Removal
A better image inpainting approach was proposed in Criminisi, Perez and Toyama, “Region Filling and
Object Removal by Exemplar-Based Image Inpainting”, Download and read that paper. Write your own
implementation of the algorithm in Python.
The goal of this experiment is to compare Efros and Leung approach, which you implemented in 2.2.1, and
this newer algorithm, for the task of object removal. The task is removing three objects/regions from the
image test_im3 (the man on the left, the sign and pole on the bottom right, and the oversaturated area on the
ground). Use and compare both approaches for this task. This is a color image, so first you need to convert it
to a grayscale image. Run both codes on the image, with one of the regions marked to be filled at a time, and
compare the results.
2.2.4. Image Quilting
In another paper by Efros and Freeman titled “Image Quilting for Texture Synthesis and Transfer”,
SIGGRAPH 2001, another texture synthesis approach was proposed. Read that paper and download an
implementation of that algorithm (or write your own) and compare its performance with Efros and Leung’s
algorithm on the example images.
Write a short essay (up to 1 page) summarizing the Imaging Quilting
approach and describing in what ways it is similar or difference from Efros and Leung’s, and how they are
compared quantitatively and/or qualitatively.
2.2.5. About Efficiency
It is very important to write an efficient code. To achieve that, avoid using for/while loops and try to use array
and matrix operations instead, which are much faster. Also try to write your code using built-in functions,
whenever possible. Report the running time.
2.2.6. Submitting the Code
You will submit all your code to Sakai. There should be at least two files; one for texture synthesis and one
for image inpainting/region filing.
2.3. Report
Write a report with all necessary images properly labeled. Submit only a softcopy of your report to Sakai.
2.3.1. Images
Show synthesized images for T1,T2,…, T5.gif by changing the WindowSize parameter to 5, 9,11 pixels
around the center.
In total you will have 21 synthesized images (7 images x 3 parameter settings). If you find a different better
parameter set for inpainting part, submit result of that as well and comment about it in the report.
Show the output of the two region filling approaches on the three regions in image test_im3
3. Grading
Texture Synthesis – This part will be up to 30% of your grade
Image Inpainting – This part will be up to 30% of your grade
Region Filling – This part will be up to 25% of your grade
Image Quilting – This part will be up to 15% of your grade
Grade Penalties
10% for late submission up to two days.
Up to 10% if the code is not running for some reason and/or giving significantly different results than
the ones reported. Do not forget to submit all the files you have written and any external files that you
have used. Create a ‘how to run’ section in your report if your code needs a special procedure to run.
Up to 50% if group work and/or code sharing is noticed. See Academic Integrity section for details.
Any code downloaded from the internet has to be acknowledged in the report.
3.1. Academic Integrity
This is an individual assignment and it has to be completed by each person independently and not with
groups. Sharing code or/and specific ideas is not allowed. The code, results and report that you have
submitted will be compared with other people’s submissions and your grade will be severely affected if a
clear resemblance exists. Instead of consulting to your classmates, please contact the TA or the Professor to
clear out any confusing points about the assignment. You are expected to write your own code unless
otherwise instructed, in such case you need to specify from where it was downloaded.
Please note that posting your code for the assignment in the future online is a violation of Rutgers policy since
it might facilitate cheating in the future.