Meet DeepLSD: A Generic Line Detector that Combines the Robustness of Deep Learning with the Accuracy of Handcrafted Detectors

In surroundings humans have created, line segments are common and efficiently convey the underlying picture structure. They complement feature points nicely because of their spatial range and presence, even in textureless areas. Line characteristics have therefore been employed in various vision tasks, including 3D reconstruction, Structure-from-Motion (SfM), Simultaneous Localization and Mapping, visual localization, tracking, vanishing point estimate, etc. A reliable and accurate detector is needed to extract line characteristics from pictures for all of these applications. The Line Segment Detector (LSD) is one example of a handmade heuristic that is used to extract line segments from a picture gradient. Due to their reliance on the image’s minute features, these techniques are quick and precise.

They may, however, need more resilience under difficult circumstances, such as dim lighting, if the visual gradient is chaotic. Additionally, they don’t account for scene-wide information and will identify any group of pixels with a similar gradient orientation, even if it contains noisy or boring lines. Deep networks have recently opened up new ways to address these problems. The deep wireframe approaches, which seek to infer the line structure of interior images, are responsible for the rebirth of line detection techniques. Since then, more general deep line segment detectors, such as joint line detectors and descriptors, have been developed. These techniques have the potential to learn from difficult pictures and become more robust when more traditional methods fall short.

They can also encode some visual context and differentiate between noisy and meaningful lines since they need a broad receptive field to manage the breadth of line segments in an image. However, most of these techniques are fully supervised, and the Wireframe dataset is the only dataset that contains ground truth lines. This dataset, which was first created for wireframe parsing, is skewed toward structural lines and is only available for indoor scenarios. Therefore, as shown in the figure above, there are better training sets for general line detectors. Additionally, contemporary deep sensors still need to outperform handmade algorithms on simple photos due to their lack of precision, much like feature points.

Due to line fragmentation and partial blockage, it might be challenging to localize line endpoints precisely. Therefore, numerous programs that use lines take limitless lines into account and disregard the endpoints. Based on this evaluation, they suggest in this study to preserve the best of both worlds: analyze the image using deep learning to weed out extraneous features, then detect the line segments using manual techniques. Thus, they maintain the advantages of deep understanding, such as abstracting the picture and increasing resistance to light and noise while preserving the precision of traditional approaches. They succeeded in achieving this aim by adopting the strategies of two earlier techniques that used a dual representation of line segments with attraction fields.

The latter demonstrates continuous representations, ideally suited for deep learning, as input to conventional line detectors. They propose to bootstrap current approaches to produce a high-quality faux ground truth rather than using ground truth lines, as in the case of the prior two methods, to train their line attraction field. As a result, as they demonstrate in their studies, their network can be trained on any dataset and tailored towards certain applications. They also suggest a unique optimization technique to enhance the line segments that have been found. This improvement is based on the vanishing points that were optimized along with the details, as well as the attraction field that was produced by the proposed network.

This adjustment may be utilized to significantly increase their prediction’s precision and enhance the performance of other deep line detectors. In conclusion, they suggest making the following contributions: 

• They describe an optimization strategy that can improve line segments and vanishing points concurrently. They suggest a technique for bootstrapping current detectors to construct ground truth line attraction fields on any picture. 

• By combining the robustness of deep learning approaches with the precision of handcrafted methods in a single pipeline

• They set a new record in several downstream tasks requiring line segments. This optimization can be used as a stand-alone refinement to improve the accuracy of any existing deep line detector. Code implementation is available on GitHub.

Check out the Paper and Github. All Credit For This Research Goes To Researchers on This Project. Also, don’t forget to join our Reddit page and discord channel, where we share the latest AI research news, cool AI projects, and more.