Percy Liang

@percyliang

professor of computer science @Stanford @stanfordnlp, co-founder of @togethercompute, marin.community, @simile_ai, pianist

Stanford, CA

cs.stanford.edu/~pliang/

Replies Reposts Media

Pinned
Percy Liang
@percyliang
May 19, 2025
What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision:
73
Percy Liang
@percyliang
Sep 29
Given how many people have tried their hand at NanoGPT, I really didn't expect a 40% reduction in a single go!
Larry Dial
@classiclarryd
Sep 28
New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with
15
Percy Liang
@percyliang
Sep 24
Marin believes strongly in the power of the open community. There are so many datasets that have been produced, living on Hugging Face. We have built scalable infrastructure that ingests this data, performs deduplication, decontamination, mixing, producing 25T tokens.
Will Held @ COLM 6-8
@WilliamBarrHeld
Sep 23
There’s an enormous amount of open training data on Hugging Face. What does it take to make it work together 🤗? For Marin’s 535B run, we built on 25T tokens from 152 datasets with licenses permitting training. Here’s the work between downloading those and training a model 🧵
13
Percy Liang
@percyliang
Sep 17
Glad to see another live training run! Here's how Marin 535B-A23B is doing today: wandb.ai/marin-communit… Anyone else want to share?
Fuli Luo
@_LuoFuli
Sep 16
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task
32
Percy Liang
@percyliang
Sep 3
A year ago, David was the only FTE on Marin. Today, thanks to Open Athena, Marin has 10 FTE. Read his post to better understand the context of Marin. And David tokens are always a pleasure to read.
David Hall
@dlwh
Sep 3
It’s been a few weeks since Marin kicked off our 535B/A23B MoE hero run. So far it’s gone almost boringly well. It’s also been just over a year since Marin joined Open Athena. I wrote up some thoughts about how far we’ve come, and how we got there. openathena.ai/blog/marin-535…
11