Multimodal Vision Research Laboratory

MVRL

Research Area: Representation Learning

We develop novel representation learning methods for computer vision and multimodal understanding. Recent work includes Frobenius norm minimization for self-supervised learning (FroSSL), dynamic feature alignment for semi-supervised domain adaptation, and learning geo-temporal image features from webcam networks. We also explore hierarchical probabilistic embeddings for multi-view image classification, covariance-based PCA for multi-size data, and representation learning approaches that leverage temporal variations in outdoor scenes. Our research addresses fundamental challenges in learning robust, transferable representations from diverse and often limited data sources.

View All Publications →

Publications

  1. Thumbnail for FroSSL: Frobenius Norm Minimization for Self-Supervised Learning
    Skean O, Dhakal A, Jacobs N, Giraldo LGS. 2024. FroSSL: Frobenius Norm Minimization for Self-Supervised Learning. In: European Conference on Computer Vision (ECCV).
    paper bibtex Representation Learning
  2. Khanal S, Sastry S, Dhakal A, Jacobs N. 2023. Learning Tri-modal Embeddings for Zero-Shot Soundscape Mapping. In: British Machine Vision Conference (BMVC).
    code paper bibtex Remote Sensing Social Media Representation Learning GeoAI
  3. Thumbnail for Dynamic Feature Alignment for Semi-supervised Domain Adaptation
    Zhang Y, Liang G, Jacobs N. 2021. Dynamic Feature Alignment for Semi-supervised Domain Adaptation. In: British Machine Vision Conference (BMVC).
    paper bibtex Representation Learning
  4. Thumbnail for Learning Geo-Temporal Image Features
    Zhai M, Salem T, Greenwell C, Workman S, Pless R, Jacobs N. 2018. Learning Geo-Temporal Image Features. In: British Machine Vision Conference (BMVC).
    paper bibtex Localization Social Media Representation Learning
  5. Thumbnail for MPCA: EM-Based PCA For Mixed-Size Image Datasets
    Shi F, Zhai M, Duncan D, Jacobs N. 2014. MPCA: EM-Based PCA For Mixed-Size Image Datasets. In: IEEE International Conference on Image Processing (ICIP). 1807–1811. DOI: 10.1109/ICIP.2014.7025362.
    paper doi bibtex Representation Learning
  6. Zhai M, Shi F, Duncan D, Jacobs N. 2014. Covariance-Based PCA for Multi-Size Data. In: International Conference on Pattern Recognition (ICPR). 1603–1608. DOI: 10.1109/ICPR.2014.284.
    paper doi bibtex Representation Learning
  7. Thumbnail for Consistent Temporal Variations in Many Outdoor Scenes
    Jacobs N, Roman N, Pless R. 2007. Consistent Temporal Variations in Many Outdoor Scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1–6. DOI: 10.1109/CVPR.2007.383258.
    website paper doi bibtex Representation Learning