To estimate the location and attributes of a sidewalk feature we are using the segmentation label image corresponding to the entire class instead of the specific feature

[himanshunaidu]

This is because in the getWorldPoints method, we do filter out segmentation pixels that fall outside the specific sidewalk feature. Instead, we get all the segmentation pixels that correspond to the sidewalk class, even if a pixel falls outside the specific sidewalk feature.

We need to refine the method to get only the sidewalk pixels corresponding to the object mask.
At the first glance, this can be seemingly done in 2 ways:

1. Create a new segmentation label image out of the contour of the specific sidewalk feature. Then perform the same algorithm for localization and attribute estimation using this new segmentation label image.
   As long as we are able to correctly create an image out of the contour, this should be the more straightforward method. Also, since the method is GPU-accelerated, it should be more efficient as well.
   The only potential issue is that we should be careful with the method we use to create the contour segmentation image. The method should not treat the segmentation as an RGB image, but as ground truth.

But, if the representation itself of the feature were to change (e.g. currently, all the points within a feature contour tend to belong to the feature, but in the future, a contour may just be the convex hull of all the feature points, and there may be huge gaps within the contour), then this method would fail.

2. Get all the segmentation pixels for sidewalk and check if each method is present within the feature contour.
   This will be slightly more tricky to perform, and doesn't have any direct advantage, as long as Method 1 has a proper error-free way to create the segmentation image.
   However, this method is closer to being ground-truth, and would age better if there is a chance that the representation of the feature itself would change (like the example given above)