Take a look at the photographs above. Are you able to tell the difference? It’s as if trying to distinguish between twins. Within the realm of computer vision systems, the same issue occurs. This research focuses on geometric vision tasks, resembling 3D reconstruction, wherein these methods often encounter the challenge of discerning whether two images portray equivalent 3D surfaces in the actual world or two distinct 3D surfaces that bear a striking resemblance. Incorrect determinations on this regard may end up in erroneous 3D models. This task known as “visual disambiguation”. 

The proposed solution by researchers at Cornell entails the creation of a novel dataset called “Doppelgangers,” which comprises pairs of images that either represent the identical surface (positives) or two distinct yet visually similar surfaces (negatives). Constructing the Doppelgangers dataset was a difficult task, as even humans can struggle to distinguish between equivalent and similar images. The approach leverages existing image annotations from the Wikimedia Commons image database to routinely generate a considerable set of labelled image pairs.

We will summarise the contributions within the above image as follows:  

(a) When presented with a pair of images, key points, and matches are extracted through the applying of feature-matching methods. It’s essential to spotlight that on this specific scenario, the photographs represent a negative pair (doppelganger) showcasing opposing sides of the Arc de Triomphe. Notably, the feature matches are primarily concentrated within the upper segment of the structure, characterised by repetitive elements, in contrast to the lower section featuring sculptures.

(b) Binary masks for key points and matches are subsequently created. Following this, each the image pair and the masks undergo alignment using an affine transformation, which is decided based on the identified matches.

(c) The classifier utilized on this context takes the concatenation of the photographs and binary masks as input and produces an output probability. This probability serves as a sign of the likelihood that the given pair constitutes a positive match.

Nonetheless, it was observed that training a deep network model directly on these raw image pairs yielded unsatisfactory results. To deal with this issue, a specialized network architecture was designed. This network incorporates invaluable information in the shape of local features and 2D correspondence to boost the performance of the visual disambiguation task.

Within the evaluation using the test set, this proposed method demonstrates impressive performance in tackling intricate disambiguation tasks. It outperforms each baseline approaches and alternative network designs by a major margin. Moreover, the study investigates the utility of the learned classifier as a simple pre-processing filter in scene graph computations inside structure-from-motion pipelines, resembling COLMAP. 

Overall, these findings highlight the potential of this approach to enhance the reliability and precision of computer vision systems in tasks related to 3D reconstruction and visual disambiguation. This research contributes invaluable insights and tools to the sector of computer vision, with promising applications in real-world scenarios requiring accurate surface recognition and reconstruction.

Try the Paper and Project. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to hitch our 30k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the newest AI research news, cool AI projects, and more.

For those who like our work, you’ll love our newsletter..