3D Reconstruction from Road Marker Feature Points

Abstraction

This project performs 3D reconstruction from road marker feature points. Our pipeline consists of several steps, including segmentation, 3D reconstruction, merging views, and refinement.

Details & Links

This is the link to our project’s repository. We use Meta’s SAM model to segment the road marker, remove the distracting street background for the segmentation step, and extract the feature points of those detected road markers.

Fig. 1. The road mark detection after applying SAM model. Left: before. Right: after.

Next, we reconstructed those feature points by projecting them to the ground of world coordinates, Shown in Fig. 1. I designed the algorithm for merging those overlapped projected feature points by comparing the IoU and the size of each overlapping road marker patch. Thus, we can distinguish whether the overlapping happened to two different road markers.

Fig. 2. The birdview of projected feature points on real-world coordinates.

Fig. 3. Left: The same road mark projected by different camera. Right: The filtered road mark.

After getting the surrounded road marker feature points, we use the ICP to calculate the instant location by matching our result with the given feature points of the global environment. Finally, we refine the prediction with a causal filter based on the stability assumption of the car’s movement. For more details, the link to the project is provided.

Fig. 4. Upper left: Raw trajectory of evaluation case 1. Upper Right: Raw trajectory of evaluation case 1. Bottom left: Smoothed trajectory of evaluation case 1. Bottom Right: Smoothed trajectory of evaluation case 1.