question about the non-matching detections

[fatemehazimi990]

Hi @uakfdotb , thanks for sharing the code, I'm very interested in your work and read the paper couple of times, yet I don't understand how the non-matching detections are handled.
At the beginning of the paper, it is mentioned the transition matrix has an additional column for the non-matching class, but I don't see that considered when generating the transition matrix M out of the similarity scores. To my understanding, the score matrix which is generated from the output of the matcher network has a size n_tracks x n_detections, so no "non-match" class here. Then row-wise and column-wise softmax is applied to get the transition matrix M (still no additional column for non-match class).
So in short, my question would be when do you add the column for non-matching class, and how do you infer a detection did not match any of the trackers based on the score/transition matrix? Is there a thresholding operation on the scores/probabilities (lets say p < 0.5 means a non-match)?

Sorry for the long question and many thanks in advance :)

[uakfdotb]

In model.py, the feature list for boxes in each frame (defined at https://github.com/favyen/uns20/blob/main/model.py#L192) includes a zero-vector (appended at https://github.com/favyen/uns20/blob/main/model.py#L203) that is used to represent not matching to any detection in that frame.

This extra box only shows up an an extra column and not an extra row in the transition matrix because it is removed at e.g. https://github.com/favyen/uns20/blob/main/model.py#L429

If no_match_logit option is enabled (default), the logits in this column are overwritten by a constant learned value (https://github.com/favyen/uns20/blob/main/model.py#L304).

Originally we tried to simply use argmax to pick between the detections and the not-match column, but at https://github.com/favyen/uns20/blob/main/infer.py#L270 there is some tuning where we also ignore any values where logit is less than 0.

[fatemehazimi990]

Many thanks for your detailed explanation @uakfdotb !

[fatemehazimi990]

Hi @uakfdotb, hope you are doing well!
I was about to test your method on the car category. Would it be possible to access model weights finetuned on kitti? Many thanks in advance :)

[uakfdotb]

Please try this one https://favyen.com/files/kitti_car_t2_fs.zip

[fatemehazimi990]

Thank you @uakfdotb ! Unfortunately these weights do not match the model and I cannot load the weights ...

[fatemehazimi990]

Is the exact same model used for kitti and mot datasets?