Good masks & bad masks

This section oulines general guidelines for creating masks.

  1. The image region covered by the mask must move exactly as the object to be patched with the overlay. The mask may consist of several connected regions.

  2. The results will be generally better if the object region to be patched is completely covered by the mask or at least surrounded by its parts on all sides.

  3. The mask should not be stretched along a single line, and should contain color or luminosity gradients, even smooth, at least in two different directions. Avoid including uniformly colored regions in your mask (where there is nothing to "lock on"), which is particularly important for mask edges. These requirements actually mean that the masks you create should not be deficient in their image data.

  4. As it moves, the masked region should not be occluded by other objects, or get outside the frame. Otherwise the algorithm is likely to produce erroneous motion parameters (see Multiple tracking segments on how to track occluded objects with several tracking segments).

  5. In should be borne in mind that different faces of a rotating 3D object may give rise to very different 2D motions in the image plane. In such a case, the mask should be limited to the image of only one such face.

  6. Take into consideration the filming conditions for the tracked object. You should distinguish the situations where the perspective of the object varies significantly.

Information density of the mask

As explained below, the masks on the following frames are not equally suited for precise and fast tracking. (Their ratings do not take into account the overlay position.)

A

B

C

D

  1. Very poor: the mask contains no color or luminosity gradients.
  2. Poor: the mask does contain color/luminosity gradients, but they are stretched along two parallel lines.
  3. Satisfactory: there are many luminosity gradients inside the mask, but the tracking speed can be increased by excluding uniform image regions.
  4. Perfect: the mask contains color/luminosity gradients along different directions (the actual mask size is not so important).

Regularity of object motion

PatchMaker assumes that, by drawing the mask, you indicate a region the motion of which can be described with a few parameters (for example, translation, rotation, and scaling along two perpendicular axes). You actually tell PatchMaker that the outlined image region moves regularly. This allows PatchMaker to increase the precision of parameter estimation by using every single pixel of the masked region .

Significant changes in object perspective

Such changes occur usually when:

  • the object rotates, or

  • the camera moves around the object, and its displacement is large in comparison to the distance to the object.

In this case, the mask should cover as much of the plain part of the object surface as possible.

For example, you need to place an image over the car's door, while the car itself makes a turn in the following frames:

A

B

  1. Correct: this mask is good for car door tracking.
  2. Incorrect: the mask covers both the front (one plane) and the side (another plane) parts of the car. The parameter values of their 2D image transformations differ substantially.

Constant object perspective

Stricktly speaking, the perspective does not change if the camera motion is limited to pan, and the object is stationary (does not move or rotate). In this case, the mask may cover unflat portions of the object or even objects different in depth. Even though Patchmaker can handle a much wider variaty of cases, it should be remembered that it does not support the perspective motion model. If the scene is filmed using a wide-angle lens, and the camera pans significantly, the perspective distortions may arise that cannot be described by affine parameters.