Sasha Rush (@srush

You use GPUs everyday, but do you (actually) know how they work? GPU-Puzzles (v0.1) - 14 short puzzles in Python with a visual debugger. No background required. Do puzzles, learn CUDA. Link: github.com/srush/GPU-Puzz…

Sasha Rush

@srush_nlp

Mar 20, 2025

Some personal news: I recently joined Cursor. Cursor is a small, ambitious team, and they’ve created my favorite AI systems. We’re now building frontier RL models at scale in real-world coding environments. Excited for how good coding is going to be.

00:00

Sasha Rush

@srush_nlp

Oct 8, 2024

Updating all my NeurIPS papers.

Sasha Rush

@srush_nlp

Oct 17, 2023

I'm advising a PhD student at Mistral and, really not sure how I feel about their LateX template.

Sasha Rush

@srush_nlp

Apr 24, 2020

1/ Spent the last couple weeks in quarantine obsessively coding a website for Virtual ICLR with @hen_str. We wanted to build something that was fun to browse, async first, and feels alive.

GIF

Sasha Rush

@srush_nlp

Mar 8, 2023

For some reason, I needed xkcd formatting to convince my brain what Large means.

I originally had the "Gettysburg Address", but after reading 10^4 tokens about Taylor Swift, I couldn't get Blank Space out of my head.

Sasha Rush

@srush_nlp

Oct 16, 2023

Introducing COLM (colmweb.org) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models. Submissions: March 15 (it's pronounced "collum" 🕊️)

Sasha Rush

@srush_nlp

Nov 19, 2020

These tutorial slides on "High Perf NLP" are really impressive. Every slide is current to the minute. Amazing set of diagrams. gabrielilharco.com/publications/E… (@gabriel_ilharco @Tim_Dettmers @IuliaTurc @kentonctlee Felipe Ferreira Cesar Ilharco)

Sasha Rush

@srush_nlp

Jan 25, 2024

Mamba apparently was rejected !? (openreview.net/forum?id=AL1fq…) Honestly I don't even understand. If this gets rejected, what chance do us 🤡 s have.

Sasha Rush

@srush_nlp

Sep 8, 2025

Cursor NY office in Soho.

Sasha Rush

@srush_nlp

Apr 18, 2022

Do you (really) know PyTorch 🔥? Try out my Tensor Puzzles 🧩. 16 mini-puzzles for those ready to take off the stackoverflow training wheels. github.com/srush/Tensor-P…

Sasha Rush

@srush_nlp

Dec 8, 2023

This talk by Angela Fan on Llama2 is so good. 30 min, she just tells you all the things.

Developing Llama 2 | Angela Fan

If you know Torch, I think you can code for GPU now with OpenAI's Triton language. We made some puzzles to help you rewire your brain. Starts easy, but gets quickly to fun modern models like FlashAttention and GPT-Q. Good luck! github.com/srush/Triton-P…

GIF

Sasha Rush

@srush_nlp

Feb 9, 2023

There are like 200 people in the world who were in charge of training 50B+ param LLMs. They all talk about it like they survived a tornado. "And then the door flew off, but the roof held!"