Many outliers in tracks

[Yuhuoo]

Hi, Thanks for your nice work. I have a question, why are there so many outliers here?
This is inconsistent with the results you reported.

[sriramsk1999]

I found this same issue. I was able to fix it by modifying models/spatracker/predictor.py by copying the code from cotracker/predictor.py. The git diff looks like this:

-                grid_pts_extra = grid_pts[:, point_mask]
-            else:
-                grid_pts_extra = None
-            # queries = torch.cat(
-            #     [torch.ones_like(grid_pts[:, :, :1]) * grid_query_frame, grid_pts],
-            #     dim=2,
-            # )
-            if grid_pts_extra is not None:
-                total_num = int(grid_pts_extra.shape[1])
-                total_num = min(800, total_num)
-                pick_idx = torch.randperm(grid_pts_extra.shape[1])[:total_num]
-                grid_pts_extra = grid_pts_extra[:, pick_idx]
-                queries_extra = torch.cat(
-                    [
-                        torch.ones_like(grid_pts_extra[:, :, :1]) * grid_query_frame,
-                        grid_pts_extra,
-                    ],
-                    dim=2,
-                )
+                grid_pts = grid_pts[:, point_mask]
+
             queries = torch.cat(
-                [torch.randint_like(grid_pts[:, :, :1], T), grid_pts],
+                [torch.ones_like(grid_pts[:, :, :1]) * grid_query_frame, grid_pts],
                 dim=2,
             )
-            queries = torch.cat([queries, queries_extra], dim=1)

After these changes, I was able to remove the extra points.

butterfly_spatracker_pred_track.mp4