multi-modality

🏄 Embed/reason/rank images and sentences with CLIP models

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Algorithms and Publications on 3D Object Tracking

[ICCV2019] Robust Multi-Modality Multi-Object Tracking

Unifying Voxel-based Representation with Transformer for 3D Object Detection (NeurIPS 2022)

An official PyTorch implementation of the CRIS paper

This repo contains the official code of our work SAM-SLR which won the CVPR 2021 Challenge on Large Scale Signer Independent Isolated Sign Language Recognition.

[CVPR'23] MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

[ICCV 2021] TRAR: Routing the Attention Spans in Transformers for Visual Question Answering

Official code for WACV 2021 paper - Compositional Learning of Image-Text Query for Image Retrieval

CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representations, ICCV 2021

(NeurIPS 2022 CellSeg Challenge - 1st Winner) Open source code for "MEDIAR: Harmony of Data-Centric and Model-Centric for Multi-Modality Microscopy"

Multi-modal Graph learning for Disease Prediction (IEEE Trans. on Medical imaging, TMI2022)

Fusion ICA Toolbox (MATLAB)

The code of CorrI2P

The implementation of MoCA

Cross-Modality Mutual Learning for Smart Contract Vulnerability Detection

Official Implementation for Pre-CoFactv2 (AAAI-23 DeFactify2.0 Workshop 1st Place)

[IJHCS] An assistant prototype for breast cancer diagnosis prepared with a multimodality strategy. The work was published in the International Journal of Human-Computer Studies.