Description
Use the Dynamic Mode Decomposition method on the video clips ski drop.mov and monte carlo.mov containing a foreground and background object and separate the video stream to both the foreground video and a
background.
The DMD spectrum of frequencies can be used to subtract background modes. Specifically, assume that ωp,
where p ∈ {1, 2, . . . , `}, satisfies kωpk ≈ 0, and that kωjk ∀ j 6= p is bounded away from zero. Thus,
XDMD = bpϕp
e
ωpt
| {z }
Background Video
+
X
j6=p
bjϕj
e
ωj t
| {z }
Foreground Video
(1)
Assuming that X ∈ R
n×m, then a proper DMD reconstruction should also produce XDMD ∈ R
n×m. However, each
term of the DMD reconstruction is complex: bjϕj
exp (ωjt) ∈ C
n×m ∀j, though they sum to a real-valued matrix.
This poses a problem when separating the DMD terms into approximate low-rank and sparse reconstructions
because real-valued outputs are desired and knowing how to handle the complex elements can make a significant
difference in the accuracy of the results. Consider calculating the DMD’s approximate low-rank reconstruction
according to
XLow-Rank
DMD = bpϕp
e
ωpt
.
Since it should be true that
X = XLow-Rank
DMD + X
Sparse
DMD ,
then the DMD’s approximate sparse reconstruction,
X
Sparse
DMD =
X
j6=p
bjϕj
e
ωj t
,
can be calculated with real-valued elements only as follows. . .
X
Sparse
DMD = X −
XLow-Rank
DMD
,
where | · | yields the modulus of each element within the matrix. However, this may result in X
Sparse
DMD having
negative values in some of its elements, which would not make sense in terms of having negative pixel intensities.
These residual negative values can be put into a n×m matrix R and then be added back into XLow-Rank
DMD as follows:
XLow-Rank
DMD ← R +
XLow-Rank
DMD
X
Sparse
DMD ← X
Sparse
DMD − R
This way the magnitudes of the complex values from the DMD reconstruction are accounted for, while maintaining
the important constraints that
X = XLow-Rank
DMD + X
Sparse
DMD ,
so that none of the pixel intensities are below zero, and ensuring that the approximate low-rank and sparse DMD
reconstructions are real-valued. This method seems to work well empirically.
1
