GeoTracker

Use GeoTracker to track an object in a video using a 3D model.

With GeoTracker you can track position and rotation of the object in 3D space for every frame and export them altogether as a TransformGeo node. You can also track camera movement based on a large 3D model taking significant space in the frame (scene-based camera tracking). The tracked camera can be then exported on the Results tab.

It’s advisable to use a properly set up Camera node reflecting the settings and properties of the camera used to film the footage. You can also use a tracked camera created with CameraTracker for example, it’s movement and rotation will be taken into account.

Before tracking connect the footage, geometry and a camera nodes to the inputs of GeoTracker and launch pre-analysis. Once the footage is pre-analysed, you can track the object by placing the object in some of the frames: press the ‘Center Geo’ button and then drag the model to the right position.

Inputs

Knobs

Tracking Toolbar

Name Label Hotkey Description
track_to_start z Track backwards until the previous keyframe or the first frame is reached
track_prev x Track one frame backwards
track_next c Track one frame forwards
track_to_end v Track forwards until the next keyframe or the last frame is reached
refine refine r Refine tracking between closest keyframes using tracking information from both keyframes
refine_all refine all Refine tracking between all keyframes
clear_between_keyframes Remove tracking data between closest keyframes
clear_bkwd Remove tracking data and keyframes before the current frame
clear_all Remove all tracking data and keyframes
clear_fwd Remove tracking data and keyframes after the current frame
enable_surface_mask_brush Use the left mouse button to mark polygons and the right button to clear them. Marked polygons will be ignored during tracking. Use this for masking moving or reflective parts of the object.
surface_mask_brush_radius brush radius Size of the brush in pixels. You can also change it draging mouse in the Viewer while holding Shift key.
ignore_surface_mask ignore Turn off the surface mask keeping it unchanged
clear_surface_mask clear Clear the surface mask

Positioning Toolbar

Name Label Description
magic_keyframe Magic Keyframe. Press this button to try automatic model placement. The object is being detected using its contour shape. The quality of detection will improve with every new keyframe, considering the keyframes are precise and consistent.
center_geo center geo Place the object in 3D space so it will be visible in the frame
reset_bones reset bones Load bone transformations from the source (input) in the current frame
focal_length_view Current focal length
remove_pins remove pins Removes selected pins if there are any, or all pins if nothing is selected
pin_unpin pin/unpin Pin or unpin selected pins, applied to all pins if nothing is selected
spring_pins_back spring pins back Pins will be sticking to the surface of the object after dragging
estimate_focal_length estimate focal length Focal length estimation happens when you drag pins. Estimation starts once you have at least 4 pins. It’s automatically turned off after switching frames. It’s also not available if you chose to get the focal length from a connected Camera node.
center_view center view Pan viewer so the geometry is in the middle while the sequence is playing
meta_keyframes Keyframes toolbar

GeoTracker Tab

Analyze Input

Name Label Description
use_gpu use GPU Using GPU may significantly increase analysis performance. At the same time it may slightly decrease the quality of analysis data.
analysis_file_name analysis file The path for the analysis file. The .precalc extension will be added automatically if not found.
analyse analyse Launch analysis and save the result to the analysis file. Tracking is based on the analysis data, so you need to launch analysis before trying to track anything.
show_analysis_info show info Show the analysis file information
tracking_channels tracking channels Channels used for analysis

Focal Length

Name Label Description
focal_length_mode

Focal length settings

  • focal length from Camera node: use the focal length set in the Camera node connected to the ‘cam’ input. Don’t forget to setup the proper film back dimensions in the Camera node. Focal length estimation can’t be used in this mode.
  • fixed focal length: use fixed focal length specified in the field (below or on the Results tab). One focal length is used for all keyframes. The film back dimensions will be taken from a default Camera node, if you want to override them — connect a Camera node with the desired film back dimensions, the focal length will not be taken from that node. Focal length estimation can be used with this mode.
  • varying focal length (zoom): use varying focal length specified in the field below. Focal length is stored within keyframes, it can be different for each keyframe. The film back dimensions will be taken from a default Camera node, if you want to override them — connect a Camera node with the desired film back dimensions, the focal length will not be taken from that node. Focal length estimation can be used with this mode.
track_with_focal_length_estimation estimate during tracking The focal length will be estimated during tracking, which may be useful for tracking objects in footages filmed with zoom lenses. Available only when the varying focal length is selected.

Mask

Name Label Description
ignore_mask mask Exclude frame area from tracking

Render

Name Label Description
lit_wireframe lit wireframe The shape of the lit wireframe will be more apparent due to darkening of the furthest regions.
adaptive_opacity adaptive opacity Opacity will be reduced for dense regions of the wireframe in 2D viewer, so the footage visibility will suffer less.
back_face_culling back-face culling Don’t render back-faced polygons in 2D viewer
textured_wireframe textured wireframe A texture will be used to color the wireframe. The default texture highlights the facial parts. You can connect your own texture to the ‘tex’ input.
draw_bone_weights bone coloring scheme

The way different parts of the 3D model are colored

  • two colors: two colors will be used. The first one will be the main mesh color, the second color can be changed in the appearance settings.
  • depth rainbow: unique color for each bone. Color difference will reflect the depth difference between parts.
  • rainbow: a unique color will be used for each mesh part
  • disabled: default color will be used for all mesh parts
enable_motion_blur motion blur Motion blur simulation. Helps to place the object properly in a frame with motion blur.
samples Sample count for motion blur simulation
shutter How long the shutter should stay open to achieve motion blur. Time is measured in frames, so 0.5 is half a frame.
shutter_offset shutter offset

When the shutter should be opened with respect to current frame number

  • centered: center the shutter around the frame
  • start: open the shutter at the frame start
  • end: close the shutter at the frame end
  • custom: open the shutter at the specified time
shutter_custom_offset Specify custom shutter offset (used only if shutter offset set to ‘custom’).
helpers_color user tracks The color used to draw user track guides in 2D viewer
surface_mask_color surface mask The color of polygons included into the surface mask
pins_color normal pins Normal pin color
residuals_color Color of pin residual lines
unpinned_pins_color unpinned pins Color of unpinned pins
selected_pins_color selected pins Color of selected pins
wireframe_color wireframe Wireframe color
bone_weights_color secondary mesh color The secondary color for mesh parts of rigged models
bone_pivots_color bone pivots The color of bone pivots. You can set alpha to 0 if you don’t want bone pivots to be drawn in viewers.
bone_joining_color skeleton The color of skeleton. You can set alpha to 0 if you don’t want the skeleton to be drawn in viewers.
bone_names_color bone names The bone names font color. You can set alpha to 0 if you don’t want bone names to be drawn in viewers.

Smoothing Tab

All smoothing settings are effective during tracking and refine only.
Once the tracking is done, changing them will not change the track,
you need to launch tracking or refine again to apply new smoothing settings.
You can tweak them differently for different parts of the footage,
the values are not being saved for different parts of the footage.
‘0’ means no smoothing while ‘1’ makes the smoothed parameter(s) almost static.

Name Label Description
z_translation_smoothing Z translation Z-axis (camera view) translation smoothing
xy_translations_smoothing XY translation XY translation smoothing
rotations_smoothing rotation Rotation smoothing
focal_length_smoothing focal length Smoothing of focal length estimation
bones_rotations_smoothing bone rotation Bones rotation smoothing
bones_translations_smoothing bone translations Bones translation smoothing

UserTracks Tab

Name Label Description
helpers User tracks help to track the geometry. Before tracking you can enable or disable user tracks. User tracks can be imported from a Tracker or a CameraTracker node. User tracks affect tracking results only when they’re projected to an object geometry, meaning you can see them over the rendered geometry in 2D-viewer.
import_tracker import tracks Import user tracks from a Tracker or a CameraTracker node
delete_helpers delete tracks Remove selected tracks
set_reference_frame set reference keyframe Use current keyframe as a reference frame for selected helper tracks

Results Tab

Name Label Description
focal_length focal length Current focal length
translate Geometry translation
rotate Geometry rotation
global_fixed_rx rx Lock X-axis rotation
global_fixed_ry ry Lock Y-axis rotation
global_fixed_rz rz Lock Z-axis rotation
global_fixed_tx tx Lock X-axis translation
global_fixed_ty ty Lock Y-axis translation
global_fixed_tz tz Lock Z-axis translation
bone_transformations Rotation and translation for every bone of the imported geometry skeleton. R - rotation, T - translation.
fix_selected fix Lock selected bones so they cannot move and rotate
unfix_selected unfix Unlock selected bones so they can move and rotate
to_default_selected reset Reset bones state to default: translation locked, rotation unlocked
unbreak_rotation unbreak rotation Fix rotation making it continious and removing 360 degrees jumps
scale scale Geometry input scale. All operations are performed with the scaled geometry. The scale is not restored on the output.

Export

Name Label Description
export_mode

Results export mode

  • TransformGeo: create a TransformGeo node containing the movement of the object in 3D relative to the input camera. If you are using the default camera or the custom focal length mode a new Camera node will also be created.
  • Axis: create an Axis node containing the movement of the object in 3D relative to the input camera. If you are using the default camera or the custom focal length mode the new Camera node will also be created.
  • TransformRiggedGeo: create a TransformRiggedGeo node containing the movement of the object in 3D relative to the input camera and the object deformations. If you are using the default camera or the custom focal length mode the new Camera node will also be created.
  • Camera: create a Camera node. Only makes sense if you’re using the default camera or the focal length estimation. Doesn’t contain any movement data.
  • Tracked Camera: create a Camera node containing the movement of the camera as if the object was static and the camera moved
  • Bone as TransformGeo: create a TransformGeo node containing the movement of a selected bone
export export Create a node of the selected type
link_output link output The exported node will be linked to GeoTracker, so when you change someting in GeoTracker the exported node will be updated. Switch it off if you need to embed all data into the exported node, to use in another script for example.
cam_rotate Inverted rotation. When coupled with inverted translation can be used as the rotation of a geometry-tracked camera.
cam_translate Inverted translation. When coupled with inverted rotation can be used as the translation of a geometry-tracked camera.

Last update: Fri Aug 7 09:49:48 2020