Visualization with or without post-processing?

[NOTGOOOOD]

Hello, thanks for your great work!

There are some outlier points that are hard to remove, when I use the '.ckpt' file you provided and visualize the result using Open3d in nuscens-0268.

May I ask how you dealt with them?

The photos below have shown the result after using different levels of uniform_down_sample and remove_radius_outlier.

And I found that these outliers appear in car because depth prediction of the overleap part of two camera is not well on a frame.

[zhanghm1995]

@NOTGOOOOD , Hi, I want to know if have you solved this problem? And how do you obtain the result of specific scenes by using the pretrained checkpoints?

[AronDiSc]

Hi @NOTGOOOOD, thank you for your interest in our work. I'm sorry for not getting back to you sooner.

About the consistency between frames: There could be many things causing this. One point that comes to my mind is what I've talked about here "Note about NuScenes training", i.e. the "wrong" camera intrinsic. So if you use the provided checkpoint, you should use those "wrong" intrinsics for inference and reconstruction. You might get better results if you re-train the refinement network with the actual intrinsics.

About removing the outliers: I followed the same approach as you. I don't have the exact parameters at hand though. However, the point above might already improve results.

[NOTGOOOOD]

Hi @NOTGOOOOD, thank you for your interest in our work. I'm sorry for not getting back to you sooner.

About the consistency between frames: There could be many things causing this. One point that comes to my mind is what I've talked about here "Note about NuScenes training", i.e. the "wrong" camera intrinsic. So if you use the provided checkpoint, you should use those "wrong" intrinsics for inference and reconstruction. You might get better results if you re-train the refinement network with the actual intrinsics.

About removing the outliers: I followed the same approach as you. I don't have the exact parameters at hand though. However, the point above might already improve results.

Thank you for your help!

But, I have confused for the "wrong camera intrinsic". I did generate the dataset from scratch, and got the camera intrinsic from L139. It seems that this is the real camera intrinsic, obtained in the "scalabel.label.from_nuscenes.parse_sequence" function.
Then, there was a change to intrinsics in NuScenesDataset.py L138.
i.e. intrinsics[[1, 3]] *= 0.964285

Is that what you refer to "wrong intrinsics" ? I used the provided checkpoint and this intrinsics, now. Should I remove this line of code?

[SeaBird-Go]

@NOTGOOOOD , Hello, could I know how to visualize the inference results like yours?