Build software better, together

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!

python deep-learning jupyter pytorch attention attention-mechanism graph-attention-networks self-attention pytorch-implementation gat graph-attention-network pytorch-gat gat-tutorial

Updated Feb 12, 2021
Jupyter Notebook

speedinghzl / CCNet

Star

CCNet: Criss-Cross Attention for Semantic Segmentation (TPAMI 2020 & ICCV 2019).

pytorch segmentation semantic-segmentation cityscape self-attention scene-parsing

Updated Mar 19, 2021
Python

brightmart / bert_language_understanding

Star

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

nlp text-classification question-answering document-classification transfer-learning fasttext language-model textcnn attention-is-all-you-need self-attention transformer-encoder bert-model pre-training language-understanding

Updated Jan 1, 2019
Python

microsoft / DeBERTa

Star

The implementation of DeBERTa

deeplearning representation-learning language-model bert natural-language-understanding self-attention roberta transformer-encoder

Updated Mar 22, 2022
Python

Separius / awesome-fast-attention

Star

list of efficient attention modules

awesome transformer attention attention-is-all-you-need multihead-attention reformer self-attention transformer-network longformer linformer

Updated Aug 23, 2021
Python

The-AI-Summer / self-attention-cv

Star

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

machine-learning deep-learning machine-learning-algorithms transformers artificial-intelligence transformer attention attention-mechanism self-attention

Updated Sep 14, 2021
Python

DirtyHarryLYL / Transformer-in-Vision

Star

Recent Transformer-based CV and related works.

computer-vision deep-learning paper transformer visual-language multi-modal self-attention vision-transformers

Updated Mar 22, 2022

prakashpandey9 / Text-Classification-Pytorch

Star

Text classification using deep learning models in Pytorch

text-classification pytorch lstm-model glove rnn-model sentiment-classification attention-model self-attention

Updated Nov 17, 2018
Python

kaituoxu / Speech-Transformer

Star

A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.

end-to-end pytorch transformer attention asr attention-is-all-you-need self-attention

Updated May 7, 2020
Python

kaushalshetty / Structured-Self-Attention

Star

A Structured Self-attentive Sentence Embedding

visualization deep-learning python3 pytorch classification attention attention-mechanism attention-model sentence-embeddings self-attentive-rnn self-attention attention-weights

Updated Sep 22, 2019
Python

xxxnell / how-do-vits-work

Star

(ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"

pytorch transformer self-attention loss-landscape vision-transformer

Updated Mar 20, 2022
Python

Tixierae / deep_learning_NLP

Star

Keras, PyTorch, and NumPy Implementations of Deep Learning Architectures for NLP

nlp deep-learning numpy word2vec word-embeddings keras python3 pytorch recurrent-neural-networks attention nmt skipgram doc2vec cnn-keras saliency-map seq2seq-pytorch self-attention document-embeddings hierarchical-attention-network

Updated Mar 11, 2020
Jupyter Notebook

leaderj1001 / Stand-Alone-Self-Attention

Star

Implementing Stand-Alone Self-Attention in Vision Models using Pytorch

pytorch self-attention stand-alone-self-attention

Updated Feb 13, 2020
Python

jayparks / transformer

Star

A Pytorch Implementation of "Attention is All You Need" and "Weighted Transformer Network for Machine Translation"

machine-translation attention-mechanism attention-is-all-you-need self-attention

Updated Oct 1, 2020
Python

daiquocnguyen / Graph-Transformer

Star

Universal Graph Transformer Self-Attention Networks (TheWebConf WWW 2022) (Pytorch and Tensorflow)

transformer graph-machine-learning graph-embeddings graph-classification self-attention graph-neural-networks graph-representation-learning node-embeddings graph-transformer

Updated Mar 10, 2022
Python

L0SG / relational-rnn-pytorch

Star

An implementation of DeepMind's Relational Recurrent Neural Networks in PyTorch.

deep-learning language-modeling pytorch recurrent-neural-networks transformer deepmind language-model word-language-model self-attention

Updated Dec 27, 2018
Python

alohays / awesome-visual-representation-learning-with-transformers

Star

Awesome Transformers (self-attention) in Computer Vision

computer-vision transformers awesome-list representation-learning self-attention vision-transformer

Updated Jul 31, 2021

cbaziotis / neat-vision

Star

Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Processing (NLP) tasks. (framework-agnostic)