In this work, we present a Deep Learning based approach for visual correspondence estimation, by deriving a Deep spectral graph matching network. We formulate the state-of-the-art unsupervised Spectral Graph Matching (SGM) approach, as part of an end-to-end supervised deep learning network. Thus allowing to utilize backpropagation to learn optimal image features, as well as algorithm parameters. For that, we present a transformation layer that converts the learned image feature, within a pair of images, to an affinity matrix used to solve the matching problem via a new metric loss function. The proposed scheme is shown to compare favorably with contemporary state-of-the-art matching schemes when applied to annotated data obtained from the PASCAL, ILSVRC, KITTI and CUB-2011 datasets.

Joint work with Yoav Liberman.