Description
Questions
1. (5 points) Is a single point convex? Use the definition of convexity to justify your answer.
2. (10 points) Is the function f(x) = (|2 − 5x| + 2x + 8e
(−4x)
) − 1 convex? Use the principles of composition and the common convex functions shown in the lecture to justify your answer.
3. (10 points) Rewrite the following optimization problem in standard form (x = [x1, x2]
T
):
maximize
x
− 4×2 + 3×1 − 3
subject to x2 ≤ −3,
− x1 − 2 ≥ x1 − 5×2,
x2 + 6.3 = x1,
− x2 + 5 + 4×1 ≤ 4×1.
4. (15 points) Consider f(x) = max{3x
2 − 2, 2x − 1}. At what value(s) of x will the subdifferential
δ f(x) contain more than one subgradient? What is δ f(x) at each such x value?
5. A linear program is defined as:
minimize
x
c
T
x
subject to Gx ≤ h,
Ax = b.
Standard form or Linear programming
a. (10 points) Write down the Lagrange dual function for this problem and define the variables.
b. (5 points) Write down the dual problem for this LP.
c. (5 points) Suppose you solved the dual problem and obtained an optimal value d
∗
. Assuming
the primal is feasible and bounded, how does d
∗
relate to the solution of the primal problem
p
∗
? Explain why.
Software
1. Install Matplotlib, which is a plotting package for python. Open a terminal and run the following
command:
sudo apt-get install python-matplotlib
Documentation is here.
2. Download HW2files.zip. Included in HW2files.zip is a file called plotter.py. This shows a simple example of how to plot a function using matplotlib. Run this script and verify that it produces a
plot. You can use the code in this script to help you debug your code in the implementation section.
3. Install cvxpy. Open a terminal and run the following commands:
sudo apt-get install python-pip
pip install cvxpy
Documentation is here.
Implementation
1. Descent Methods: Here you will implement two descent methods and compare them.
a. (5 points) Implement backtracking line search for functions of the form f(x) : R → R. Set
α = 0.1 and β = 0.6. Submit your code as backtracking.py in your zip file.
b. (10 points) Implement the Gradient Descent algorithm. Use your backtracking line search implementation (with the same α and β as above) to compute the step length. Use ϵ = 0.0001.
Submit your code as gradientdescent.py in your zip file.
c. (15 points) Implement the Newton’s Method algorithm. Use your backtracking line search
implementation (with the same α and β as above) to compute the step length. Use ϵ = 0.0001.
Submit your code as newtonsmethod.py in your zip file.
d. (20 points) Run Gradient Descent and Newton’s method on the following problem, starting at
x
(0) = 5:
minimize
x
f(x) = e
(0.5x+1) + e
(−0.5x−0.5) + 5x
Generate the following plots (this can be done using the matplotlib library). Include these
plots in your pdf and the code to generate them as plot descents.py in your zip. Remember
to label the axes.
2
i. A plot showing the objective function over the interval [−10, 10] (black) and the sequence
of points generated by Gradient Descent (red) and Newton’s Method (magenta).
ii. A plot showing the f(x
(i)
) vs. i for Gradient Descent (red) and Newton’s Method (magenta).
Explain which algorithm performed better in this example in terms of number of iterations and
why.
2. Stochastic Gradient Descent: Open the SGDtest.py file included in HW2files.zip. Here you’ll see
that a function fsum(x) has been defined as a sum of functions fi(x,i) for i = 1, …, nFunctions.
The first and second derivatives of fsum and fi are also given.
a. (10 points) Implement the Stochastic Gradient Descent (SGD) algorithm as a separate file sgd.py
(submit this in your zip file). You can call your algorithm from SGDtest.py to test it. Set the
parameters in the following way:
• Set the step size t = 1. It’s OK to use this large step size for this problem because ∇ fi
is
very small.
• Run the algorithm for 1000 iterations (this is the termination condition).
• Choose one random ∇ fi per iteration.
• Start at x
(0) = −5.
b. (10 points) Create a plot showing fsum(x
(i)
) vs. i and insert it in the pdf and include the code to
generate the plot as plot sgd.py in your zip. Note that you should not evaluate fsum within
the SGD loop because this will be very slow. Instead, store the x
(i)
s you produce and evaluate
fsum after you’re done running the algorithm. Is fsum(x
(i)
) always decreasing? Explain why
or why not.
c. (10 points) Run SGD 30 times and compute the mean and variance of the resulting fsum(x
∗
).
Run SGD with 750 iterations 30 times and compare the resulting mean and variance to what
you got with 1000 iterations. Explain the results.
d. Now we will compare SGD with 1000 iterations to Gradient Descent and Newton’s Method
in terms of computation time. Use x
(0) = −5 and ϵ = 0.0001 for both Gradient Descent and
Newton’s Method. To time how long an algorithm takes, see the timing code in SGDtest.py.
i. (10 points) Put the runtimes you get for the three algorithms in your pdf. Explain your
results, i.e. why are you getting the order of times you get here.
ii. (5 points) Compare the three algorithms in terms of fsum(x
∗
). Which is the best and which
is the worst? Explain why and explain if the difference is significant.
3. Implementing the Barrier Method: Before you start writing code, you will need to figure out some
of the equations you will need. The solutions to the following should be included in the pdf you
submit:
a. (5 points) Write down the objective function used in the centering problem of the log barrier
method (in the general case). Define the variables used in this function.
b. (5 points) Write down the function for (a) in the case of a linear program. Define the variables
used in this function.
c. (5 points) Write down the derivative of the function for (b).
d. (5 points) Write down the second derivative of the function for (b).
e. (5 points) Write down the duality gap for the barrier method with log barrier functions. It
should be in terms of numplanes, the number of hyperplanes, and t, the optimization “force”
increment.
The code: Included in HW2files.zip is a file called barrier.py. This file sets up, draws, and solves
a 2-dimensional LP. The barrier method is heavily commented but some parts are missing. You
will need to fill in those parts and run the code.
You will need to go through the code carefully to
understand what is going on in order to fill in the parts correctly. Please make sure not to change
anything in the script except what is marked as ###YOUR CODE HERE###. When you are finished and
you run your code, it should produce an output that looks like Figure 1.
Figure 1: The red lines are the hyperplanes defining the constraints, blue arrows are the normals of these
hyperplanes. Green xs are the x values computed during the inner loop iterations (i.e. the Newton
iterations). Red xs are the values at every outer-loop iteration.
The script should also print out this message at the end:
The optimal point: (-1.767550, 0.353440)
Total number of outer loop iterations: 28
If you’re not getting this solution (with accuracy up to the 2nd decimal place), something is wrong
with your implementation.
f. (35 points) Include your barrier.py in your zip file. We will test your code on a different
problem to verify that it is correct (i.e. by changing the A,b, and c matrices). Do not assume the
number of hyperplanes is constant. The problem will still be 2-dimensional. If your code does
not run or crashes, you will receive 0 points for this part of the problem.
g. (10 points) Now it’s time to verify your solution using a state-of-the art solver. Write a script
using cvxpy to solve the same problem. You only need to give it the proper A,b, and c matrices
and it will do the rest. Include your code as lptest.py in your zip. When we run this code it
should only print out the solution, which should be the same as what you found in barrier.py
(with accuracy up to the 2nd decimal place). See here for an example LP in cvxpy.
4. Optimal control using a QP: The script carqp.py simulates driving a robot with simplified car-like
dynamics. We will formulate the controller for this car as a quadratic program (QP).
a. (15 points) The car has discrete-time dynamics xt+1 = f(xt
, ut), where xt
is the state at time
t and ut
is the control command applied at time t. The state is the pose of the car [x, y, θ]
T
and the control is [speed, turn]
T
.
The dynamics function is not convex. So, the first step to
controlling the car with a QP is to approximate the true dynamics with a linearization. We will
NOT assume that the dynamics function is known in closed form, so we must use numerical
differentiation to linearize the dynamics. The linearized dynamics are of the form
xt+1 = f(xr
, ur) + A(xr
, ur)[xt − xr
] + B(xr
, ur)[ut − ur
]
where A ∈ R3x3 and B ∈ R3x2
.
A and B are not constant matrices, rather they are computed for
some reference state xr and control ur (they are a local approximation to the dynamics). Since
the dynamics takes as input two vectors (xt and ut), it is convenient to concatenate them to
make the linearization code easier to write:
A(xr
, ur)[xt − xr
] + B(xr
, ur)[ut − ur
] = [A(xr
, ur) B(xr
, ur)] ”
xt − xr
ut − ur
#
In fact, [A(xr
, ur) B(xr
, ur)] is the Jacobian of f evaluated at xr
, ur
. In carqp.py, you will see
the function linearize dynamics numerically, which takes as input xr
, ur
, h, and a function
pointer to the true dynamics. Implement Newton’s difference quotient to compute the Jacobian. Do not compute the derivative analytically, this will receive no credit.
Your code should
go in the section of linearize dynamics numerically denoted by ###YOUR CODE HERE###.
To test your implementation run python3 carqp.py test linearization, which will print
out the A and B matrices and show the prediction error on a test example. A correct implementation will produce an error < 10−10 for this example. We will confirm this when we run
your code.
b. (20 points) Now that you have a method to linearize dynamics, we can use a QP to control the
robot. The function optimize single action in carqp.py should produce the optimal control
command. This function uses the cvxpy optimizer to solve the QP. Your goal is to formulate
the constraints and objective function for this QP. Define these in the ###YOUR CODE HERE###
block.
The objective is to find a control command u
∗
that produces an xt+1 which is as close as possible to the goal state, while obeying the constraints defined in the variables speed limit and
turn limit. To simplify the problem, we will linearize about the current state (xt = xr) and a
control of ur = [0, 0]. We will assume that xt
is an equilibrium point, i.e. xt = f(xt
, [0, 0]).
The
dynamics then become:
5
xt+1 = xt + A(xt
, ur)[xt − xt
] + B(xt
, ur)[ut − ur
] = xt + B(xt
, ur)[ut − ur
]
You will need to use these dynamics as part of your objective function.
cvxpy is very flexible, so your constraints and objective function do not have to be in standard
form. Note that this function uses linearize dynamics numerically, so make sure to complete part (a) before starting this part.
To test your method, run python3 carqp.py run test 0. This will show an animation where
the car drives toward a pre-specified goal state. At each step of the animation, it runs the
optimize single action function and generates a command. That command is then executed
using the true dynamics.
The animation stops when the magnitude of the command is very
small (< 10−10) or a maximum number of steps (40) has been executed. If your function is
implemented correctly the final frame of the animation will show the car being very close to,
but not exactly at, the goal state (see Figure 2 below).
The distance to goal should be less than
0.02. We will test your code by running it on a different example.
Figure 2: Expected output for test 0.
Make sure to include your carqp.py in your zip file.
Note: There are two other tests you can run to test out your method: run test 1 and run test
2. Test 1 should also bring the car very close to the goal, while there will be a large error for
test 2. Remember that the QP approach is a local method that only thinks one step ahead, and
so there are many problems it can’t solve.
6
