The Wayback Machine - https://web.archive.org/web/20221206141246/https://github.com/topics/linear-attention

#

linear-attention

Here are 5 public repositories matching this topic...

BlinkDL / RWKV-LM

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

deep-learning transformers pytorch transformer lstm rnn gpt language-model attention-mechanism gpt-2 gpt-3 linear-attention rwkv

Updated Dec 6, 2022
Python

lironui / Multi-Attention-Network

The semantic segmentation of remote sensing images

remote-sensing segmentation attention-mechanism semantic-segmentation linear-attention

Updated Jul 29, 2022
Python

lironui / MAResU-Net

The semantic segmentation of remote sensing images

remote-sensing attention segmentation attention-mechanism semantic-segmentation linear-attention

Updated Jul 29, 2022
Python

mtanghu / LEAP

LEAP: Linear Explainable Attention in Parallel for causal language modeling with O(1) path length, and O(1) inference

deep-learning parallel transformers pytorch transformer rnn attention-mechanism softmax local-attention dot-product-attention additive-attention linear-attention

Updated Oct 5, 2022
Jupyter Notebook

Rushi314 / Transformers-for-high-resolution-image-synthesis

Taming Transformers for High-Resolution Image Synthesis

computer-vision gans linear-attention transformers-gpt2 sflickr

Updated May 2, 2022
Jupyter Notebook

Improve this page

Add a description, image, and links to the linear-attention topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the linear-attention topic, visit your repo's landing page and select "manage topics."