Yu Zhang's picture

Yu Zhang

yzhangcs

·

https://yzhang.site

AI & ML interests

None yet

Organizations

yzhangcs's activity

upvoted a paper 5 days ago

Gated Linear Attention Transformers with Hardware-Efficient Training

Paper • 2312.06635 • Published Dec 11, 2023 • 5

upvoted 2 collections 7 days ago

GLA

5 items • Updated 5 days ago • 1

GSA

3 items • Updated 8 days ago • 1

upvoted a paper 8 days ago

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

Paper • 2409.07146 • Published 9 days ago • 18

upvoted a paper 22 days ago

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published 27 days ago • 21

upvoted an article 29 days ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 58

upvoted an article about 1 month ago

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 242

upvoted a paper about 1 month ago

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

Paper • 2407.10960 • Published Jul 15 • 10

upvoted 4 papers 3 months ago

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Paper • 2406.06484 • Published Jun 10 • 3

Scalable MatMul-free Language Modeling

Paper • 2406.02528 • Published Jun 4 • 8

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31 • 63

Block Transformer: Global-to-Local Language Modeling for Fast Inference

Paper • 2406.02657 • Published Jun 4 • 36

upvoted a paper 4 months ago

Simple linear attention language models balance the recall-throughput tradeoff

Paper • 2402.18668 • Published Feb 28 • 18

upvoted a paper 5 months ago

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 24

upvoted an article 5 months ago

Article

Introduction to State Space Models (SSM)

By

•

Jul 19

• 77

upvoted 7 papers 5 months ago

HGRN2: Gated Linear RNNs with State Expansion

Paper • 2404.07904 • Published Apr 11 • 17

Pre-training Small Base LMs with Fewer Tokens

Paper • 2404.08634 • Published Apr 12 • 33

TransformerFAM: Feedback attention is working memory

Paper • 2404.09173 • Published Apr 14 • 43

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 62

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Paper • 2404.07143 • Published Apr 10 • 103

Linear Transformers with Learnable Kernel Functions are Better In-Context Models

Paper • 2402.10644 • Published Feb 16 • 78

Textbooks Are All You Need II: phi-1.5 technical report

Paper • 2309.05463 • Published Sep 11, 2023 • 86

upvoted a paper 6 months ago

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper • 2402.17764 • Published Feb 27 • 590

upvoted a collection 6 months ago

based

These language model checkpoints are trained at the 360M and 1.3Bn parameter scales for up to 50Bn tokens on the Pile corpus, for research purposes. • 14 items • Updated May 14 • 8

upvoted 2 papers 6 months ago

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Paper • 2403.03234 • Published Mar 5 • 11

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

Paper • 2402.04347 • Published Feb 6 • 13

upvoted 2 papers 7 months ago

Scaling TransNormer to 175 Billion Parameters

Paper • 2307.14995 • Published Jul 27, 2023 • 21

BitDelta: Your Fine-Tune May Only Be Worth One Bit

Paper • 2402.10193 • Published Feb 15 • 17

upvoted a paper 8 months ago

TinyLlama: An Open-Source Small Language Model

Paper • 2401.02385 • Published Jan 4 • 88

upvoted a collection 10 months ago

Training Datasets

A collection of pseudo-labelled datasets used to train the Distil-Whisper model. • 9 items • Updated Mar 21 • 14

upvoted a paper 11 months ago

LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery

Paper • 2310.18356 • Published Oct 24, 2023 • 22

upvoted a paper 12 months ago

Language Modeling Is Compression

Paper • 2309.10668 • Published Sep 19, 2023 • 82