Project Page

Multi-modal Metric Learning With Local CCA

Abstract
We address the problem of multimodal signal processing from a kernel-based manifold learning standpoint. We propose a data-driven method for extracting the common hidden variables from two multimodal sets of nonlinear high-dimensional observations. To this end, we present a metric based on local canonical correlation analysis (CCA). Our approach can be viewed both as an extension of CCA to a nonlinear setting as well as an extension of manifold learning to multiple data sets. We test our method in simulations, where we show that it indeed discovers the common variables hidden in high-dimensional nonlinear observations without assuming prior rigid model assumptions.
  Source Code (Github)
The Matlab code for [1] can be downloaded from the following Github repository: Source Code
The code contains the two experiments described in [1].
The code in the repository is applied to the following example: Rotating Pieces
  References
[1] O. Yair, R. Talmon, “Multimodal metric learning with local CCA”, Proc. SSP 2016.
[2] O. Yair, R. Talmon, “Local canonical correlation analysis for nonlinear common variables discovery”, submitted, 2016 (arXiv version).