Multiple tracking segments

Multiple tracking segments are helpful when dealing with video objects that become partially occluded

by the frame boundary, with the masked object moving out of the frame;
by another object, closer to the camera.

Occlusion by the frame boundary

PatchMaker ver. 1.0 lacks capacity to detect occlusions without human intervention. Hence, the tracking accurateness and patching quality might deteriorate as the mask approaches and moves out of the frame bounds.

Consider the following three consecutive frames of a video. The task is to patch the light gray rectangle. The mask (shown in blue color under the corresponding frames) covers the rectangle and an adjacent part of the bridge (see Good masks & bad masks).

The mask in the third frame has partly gone outside the left frame boundary and is distorted (shrinked too much horizontally). The bridge motion, therefore, is not tracked correctly. At the same time the results are good over the larger part of the clip, and the overlay "sticks" well to the target. What should be done to complete the tracking to the end?

Until now, we managed with a single tracking segment. On the Trackline below, the frame is selected where the results become inadequate:

We are going to create another tracking segment that extends from this frame to the end of the clip and assign it a mask that will stay within the frame boundary throughout the segment. Such a mask has to be drawn outside the gray rectangle and, for that matter, the overlay. An example of the new mask in the last frame of the clip is shown below:

The new segment is created by PatchMaker in the top row of the Trackline on top of all other segments. Therefore, inside the interval where it overlaps with the first segment (frames #31 — #38), the transformations from the second segment will be applied to the overlay.

A general rule is: to preserve the "continuity" of transformations between two or more tracking segments, they must overlap over at least one frame. If their common interval consists of more than one frame, the motion data from the lower segments will be ignored in these frames.

In our example, the new mask for frame #49 covers a low-contrast image region. To obtain acceptable tracking quality, we use the Translation motion model for the new segment. This model contains only two parameters and yields a more stable result. (See advice).

Occlusion by an obstacle

The same approach can be taken if the masked region comes to be occluded by an object that is closer to the camera: create multiple segments with masks that are never occluded over their respective intervals.

This technique can be used to accurately determine the motion of an object partly concealed by an obstacle. However, you will not be able to correctly apply an overlay by means of PatchMaker alone — the overlay will lay in front of both the occluded object and the obstacle. Export your transformations to AE, and proceed there with your compositing.