Zero-Weighted Trackers

Zero-Weighted Trackers　ゼロウエイトトラッカー

 

あまり自信がない特徴があったと想像してください、それがカメラ（またはオブジェクト）パスと視野に影響を及ぼすことをまったく望みませんでした。しかし、とりあえずそれをトラッキングしたくて、何を得るのか見たかったとします。
木の上の葉の全てかもしれないし、ラフな雲を得ることを望むかもしれません。
トラッカーを取得することができ、解析のウェイトをゼロまで下げてみることができました。
しかし、ウェイトは0.05の下限があるので、それは失敗します。
ウェイトが減少する、そして、トラッカーがだんだん少なく影響があって、若干の好ましくない副作用があるので、SynthEyesはそれを防ぎます。
その代わりに、トラッカー・パネルのゼロウエイトトラッカー（ZWT）ボタンをクリックすることができます。すると、それはウェイトをゼロにセットします（内部的に）。
好ましくない副作用は回避されます、そして、新しい可能性が出てきます。
ZWTsは解析（ビューと通常のトラッカー場所のカメラまたはオブジェクト・パスとフィールド）に影響を及ぼさないで、最初の解析が得られるまで解析されることはありません。

ZWTsは、ノーマル解析の結果において３Ｄの位置をもたらすために解析されます。

 

Tip:別々の選択カラーが、ZWTsのためにあります。
それが通常他のトラッカーと同じ色であるけれども、ZWTsが自動的に目立つことを望むならば、それを変えることができます。
重要なことに、それらの２次元のトラッキング、カメラ（またはオブジェクト）パスまたは視野を変えるときはいつでも、ZWTsは自動的に再解析されます。
ZWT解析が全体的な解析に影響を及ぼさないので、これは可能です。
それは、新しいポストを解析しているワークフローを可能にします。

Solve As You Track

After solving, if you want to add a tracker, create it  and change it to a ZWT (use the W keyboard accelerator if you like). Keep the Quad view open. Begin tracking. Watch as the 3-D point leaps into existence, wanders around as you track, and hopefully converges to a stable location. As you track, you can watch the per-frame and overall error numbers at the bottom of the tracker panel

Hop over to thegraph editor, and take a quick look at the error curve for any spikes—since the position is already calculated, the error is valid.

Once you’ve completed tracking,change the tracker back to normal mode. Repeat for additional new trackers as needed. You can use the same approach modifying existing trackers, temporarily shifting them to ZWTs and back.

When you do your next Refine cycle using the Solver panel, the trackers will be solved normally, and influence the solution in the usual way. But, you were able to use the ZWT capability to help do the tracking better and quicker.

Juicy Details

ZWTs don’t have to be only on a camera, they can be attached to a moving object as well. You can also configure Far ZWTs.

The ZWT calculation respects the coordinate system constraints: you can constrain Z=0 (with On XY Plane) to force a ZWT onto the floor in Z-up mode. A ZWT can be partially linked to another tracker on the same camera or object. It doesn’t make sense to link to a tracker on a different object, since such links are always in all 3 axes, overriding the ZWT calculation. Distance constraints are ignored by ZWT processing.

If you have a long shot and a lot of ZWTs and must recalculate them often (say by interactively editing the camera path), it is conceivable that the ZWT recalculation might bog down the interactive update rate. You can temporarily disable ZWT recalculation by turning offtheTrack/ZWT auto-calculationmenu item. They will all be recalculated when you turn it back on.

Adding Many More Trackers

After you have auto-tracked and solved a shot, you may want to add additional trackers, either to improve accuracy in a particular area of the shot, or to flesh out additional detail, perhaps beforebuilding a mesh from tracker locations.

SynthEyes provides a way to do this efficiently in a controlled manner, with theAdd Many Trackers dialog. This dialog takes advantage of the already-computed blips and the existing camera path to identify suitable trackers: it is the same situation as Zero-Weighted-Trackers (ZWTs), and by default, the newly-created trackers will be ZWTs—they do not have to be solved any further to produce a 3-D position, since the 3-D position is already known.

Important: you must not have already hitClear All Blipson theFeature panel orClean Up Trackersdialog, since it is the blips that are analyzed to produce additional trackers.

The Add many trackers dialog, below, provides a wide range of controls to allow the best and most useful trackers to be created. You can run the dialog repeatedly to address different issues.

You can also use theCoalesce Nearby Trackersdialog to join multiple disjointed tracks together: the sum is greater than the parts!

 

When the dialog is launched from the Track menu, it may spend several seconds busily calculating all the trackers that could be executed, and it saves that list in a temporary store. The number of prospective trackers is listed as the Available number, 2754 above. By adjusting the controls on the dialog, you control which of these prospective trackers are added to the scene when you push the Add button. At most, theDesirednumber of trackers will be added.

Basic Tracker Requirements

The prospective trackers must meet several basic requirements, as described in the requirements section of the panel. These include a minimum length (measured in frames), and an amplitude, plus average and peak errors.

The amplitude is a value between zero and one, describing the change in brightness between the tracker center and background. Larger values will require more pronounced trackers.

The errors numbers measure the distance between the 2-D tracker position and the computed 3-D position of the tracker, mapped back into the image. The average error limits the noisiness and jitter in the trackers, while the peak error limits the largest “glitch” error. Notice that these controls do not change any trackers, but instead select which of the prospective trackers are actually selected for addition.

To a Range of Frames

To add trackers in a specific range of frames in the shot, set up that region in the Frame-Range Controls: from a starting frame to an ending frame. Then, set a minimum overlap: how many frames each prospective tracker must be valid, within this range of frames. For example, if you have only a limited number of trackers between frames 130 and 155, you would set up those two as the limits, and set the minimum overlap to 25 at most, perhaps 20.

To an Area in 3-D Space

To add trackers in a particular 3-D area of the scene, open the camera view, and go to a frame that makes the region needing frames clearly visible. Lasso the region of interest—it does not matter if there are any trackers there already or not. The lassoed region will be saved. (Fine point: the frame number is also saved, so it does not matter if you change frames afterwards.)

Open the Add many trackers dialog, and turn on theOnly within last Lassocheckbox. The only trackers selected will be those where the 3-D point falls within the lassoed area, on the frame at which the lasso occurred.

Zero-Weighted vs Regular Trackers

Once all the criteria have been evaluated, and a suitable set of trackers determined, hitting Add will add them into the scene. There are several options to control this (which should be configuredbeforehitting Add).

The most important decision to make is whether you want a ZWT or a regular tracker. Intrinsically, the Add many trackers dialog produces ZWTs, since it has already computed the XYZ coordinates as part of its sanity-checking process. By using ZWTs, you can add many more trackers without appreciably affecting the re-solve time if you later need to change the shot. So using ZWTs is computationally very efficient, and is an easy way to go if you need more trackers to build a mesh from.

On the other hand, if you need additional trackers to improve the quality of the track, by adding more trackers in an under-populated region of 3-space or range of frames, then adding ZWTs will not help, since they do not affect the overall camera solution. Instead, check the Regular checkbox, and ordinary trackers will be created, still pre-solved with their XYZ coordinates. You can solve again using Refine mode, and the camera path will be updated taking into account the new trackers.

If you add hundreds or thousands of regular trackers, the solve time will increase substantially. Designed for the best camera tracking, SynthEyes is most efficient for long shots, not for thousands of trackers. To see why this choice was made, note that even if all the added trackers are of equal quality, the solution accuracy increases much slower than the rate trackers are added. You can use some of the trackers for the solve, and keep the rest as ZWTs.

Other New Tracker Properties

Normally, you will want the trackers to be selected after they are added, as that makes it easy to change them, see which were added, etc. If you do not want this, you can turn off theSelectedcheckbox.

Finally, you can specify a display color for the trackers being added by selecting it with the color swatch, and turning on theSet colorcheckbox. That will help you identify the newly-added trackers, and you can re-select them all again later using theSelect same coloritem on the Edit menu.

It may take several seconds to add the trackers, depending on the number and length of trackers. Afterwards, you are free to add additional trackers to address other issues if you like—the ones already added will not be duplicated.

Coalescing Nearby Trackers

Now that you know how to create many more trackers, you need a way to combine them together intelligently. Whether you use the Add Many More Trackers panel or not, after an autotrack (or even heavy supervised tracking) you will often find that you have several trackers on the same feature, but covering different ranges of frames. Tracker A may track the Red Rock for frames 0-50, and Tracker B may also track Red Rock from frames 55-82. In frames 51-54, perhaps an actor walked by, or maybe the rock got blurred out by camera motion or image compression.

It is more than a convenience to combine trackers A and B. The combined tracker gives SynthEyes more information than the two separately, and will result in a more stable track, less geometric distortion in the scene, and a more accurate field of view. (Exception: if there is much uncorrected lens distortion, you are better off with consistently short-lived trackers.)

The Coalesce Nearby Trackers dialog, available on the Tracker menu, will automatically identify all sets of trackers that should be coalesced, according to criteria you control.

When you open the dialog, you can adjust the controls (described shortly) and then click the Examine button.

SynthEyes will evaluate the trackers and select those to be coalesced, so that you can see them in the viewports. The text field, reading “(click Examine)” in the screen capture above, will display the number of trackers to be eliminated and coalesced into other trackers.

At this point, you have several main possibilites:

1.   clickCoalesceto perform the operation and close the panel;

2.   adjust the controls further, andExamineagain;

3.   close the dialog box with the close box (X) at top right (circle at top left on Mac), then examine the to-be-coalesced trackers in more detail in the viewports; or

4.   Cancelthe dialog, restoring the previous tracker selection set.

If you are unsure of the best control settings to use, option 3 will let you examine the trackers to be coalesced carefully, zooming into the viewports. You can then open the Coalesce Nearby Trackers dialog again, and either adjust the parameters further, or simply click Coalesce if the settings are satisfactory.

What Does Nearby Mean?

The Distance, Sharpness, and Consistency controls all factor into the decision whether two trackers are close enough to coalesce. It is a fairly complex decision, taking into account both 2-D and 3-D locations, and is not particularly amenable to human second-guessing. The controls are pretty straightforward, though.

As an aside, it might seem that all that is needed is to measure the 3-D distance between the computed tracker points, and coalesce them if the points are within a certain distance measured in 3-D (notin pixels). However, this simplistic approach would perform remarkable poorly, because the depth uncertainty of a tracker is often much larger than the uncertainty in its horizontal image-plane position. If the distance was large enough to coalesce the desired trackers, it would be large enough to incorrectly coalesce other trackers.

Instead, SynthEyes uses a more sophisticated and compute-intensive approach which is evaluated over all the active frames of the trackers.

The first and most important parameter is theDistance, measured in horizontal pixels. It is the maximum distance between two trackers that can be considered for coalescing. If they are further apart than thisin all frames, they will definitely not be coalesced. If they are closer, some of the time, they may be coalesced, increasingly likely the closer they are.

The second most important parameter, theConsistency, controls how much of the time the trackers must be sufficiently close, compared to their overall lifetime. So very roughly, at 0.7 the trackers must be within the given distance on 70% of the frames. If a track is already geometrically accurate, the consistency can be made higher, but if the solution is marginal, the consistency can be reduced to permit matches even if the two trackers slide past one another.

The third parameter,Sharpness, controls the extent to which the exact distance between trackers affects the result, versus the fact that they are within the required Distance at all. If Sharpness is zero, the exact distance will not matter at all, while at a sharpness of one (the maximum), if the trackers are at almost the maximum distance, they might as well be past it.

Sharpness can be used to trade off some computer time versus quality of result: a small distance and low sharpness will give a faster but less precise result. Settings with a larger distance and larger sharpness will take longer to run but product a more carefully-thought-out result—though the two sets of results may be very similar most of the time, because the larger sharpness will make the larger distance nearly equivalent to the smaller distance and low sharpness.

If you are handling a shot with a lot of jitter in the trackers, due to large film grain or severe compression artifacts, you should decrease the sharpness, because those small differences in distance are in fact meaningless.

What Trackers should be Coalesced?

Three checkboxes on the coalesce panel control what types of trackers are eligible to be coalesced.

First, you can request thatOnly selected trackersbe coalesced. This allows you to lasso-select a region where coalescing is required. (Note: if you only need 2 particular trackers coalesced, for sure, use Track/Combine Trackers instead.)

Second, frequently you will only want to coalesce auto-trackers, or trackers created by theAdd Many Trackersdialog. By default, supervised non-zero-weighted trackersare not eligible to be coalesced. This prevents your carefully-constructed supervised trackers from inadvertently being changed. However, you can turn on theInclude supervised non-ZWT trackerscheckbox to make them eligible.

SynthEyes will also generally coalesce only trackers that are not simultaneously active: for example, it might coalesce two trackers that are valid on frames 0-10 and 15-25, respectively, but not two trackers that are valid on frames 0-10 and 5-15. If both are autotrackers, if they are simultaneously active, they are not tracking the same thing. The exception to this is if they are a large autotracker and a small one, or an autotracker and a supervised tracker. To combine overlapping trackers, turn off theOnly with non-overlapping frame rangescheckbox.

A satisfactory approach might be to coalesce once with the checkbox on, as is the default, then open the dialog again, turn the checkbox off, and Examine the results to see if something worth coalescing turns up.

An Overall Strategy

Although we have talked as if SynthEyes only combines two trackers, in fact SynthEyes considers all the trackers simultaneously, and can merge three or more trackers together into a single result in one pass.

It is possible that coalescing immediately a second time may produce additional results, but this is probably sufficiently rare to make it unnecessary in routine use.

However, after you coalesce trackers, it will often be helpful to do aRefine solving cycle, then coalesce again. After the first coalesce, the refine cycle will have an improved geometric accuracy due to the longer tracker lifetimes. With the improved geometry, additional trackers may now be stable enough to be determined to be tracking the same feature, permitting a coalesce operation to combine them together, and the cycle to repeat.

Viewing this pattern in reverse, observe that a broader distance specification will be required initially, when trackers on the same feature may be calculated at different 3-D positions.

This is particularly relevant togreen-screen shots, where the comparatively small number of trackable features and their frequently short lifetime, due to occlusion by the actors, can result in higher-than-usual initial geometric inaccuracy.

Because the green-screen tracking marks are generally widely separated, there is little harm in increasing the allowable coalesce Distance. The features can then be coalesced properly, and the Refine cycle will then rectify the geometry. The process can be repeated as necessary.

If you are using Add Many Trackers and then Coalescing and refining, you should turn on theRegular, not ZWTcheckbox on the Add Many dialog, so that the added trackers will affect the Refine solution.