Need of gt_height and gt_pitch for inference: GeoNet3D_ext

[gautamjain1009]

Hey! I have explored your implementation quite nicely. The GeoNet3D_ext model takes as an input the features from the lane segmentation model. Thus those features are then fed to the GeoNet3D_ext model for 3d lane detection. From your implementation, I have realized that you are not predicting camera height and camera pitch instead you are using the gt camera height and gt camera pitch for the same and in your training script you are updating your M_inv using the same gt_camera_height and gt_camera_pitch.

Pytorch_Generalized_3D_Lane_Detection/main_train_GenLaneNet_ext.py

Line 218 in 5bb190e

if not args.fix_cam and not args.pred_cam:

            if not args.fix_cam and not args.pred_cam:
                model2.update_projection(args, gt_hcam, gt_pitch)

I tried to run the train script for GeoNet3D_ext and try to found out that gt_cam_height == pred_cam_height and the same for pitch.

Whereas in the unofficial implementation of 3D lanenet as per the train script, the height and pitch are different from gt's.

**Consider a real-world scenario where I want to use the pre-trained models provided by you for 3D lane Detection: ** in that particular case camera_height and camera_pitch are fixed as not really predicted by the model. Does in that particular case the results for 3d lane detection will differ? Have you tested your approach with any real-world data?

[CompliceU]

@gautamjain1009 Hi, have you tried training with OpenLane?

[gautamjain1009]

WIP. I will train soon and will open source the code for that.

[CompliceU]

@gautamjain1009 That's great too, I'm currently trying this to work and am having a huge problem preventing it from training properly

[CompliceU]

@gautamjain1009 Hello, I don't know if you have tried it. I tried it. I feel that the effect is very different from the results provided by the author of openlane. During the training process, the loss of the verification part has not been effectively reduced.