About

Hi 👋 I work on AI infrastructure at Moonshot AI — large-model inference, RL infrastructure, deep-learning compilers, and GPU performance engineering. Most of what I do sits between the model and the hardware: making large models cheaper to serve, and making the cost of serving them knowable before anything is deployed.

Before Moonshot I led the LLM inference and AI compiler team at Kuaishou, where the work spanned inference systems for large models and a compiler for recommendation workloads. Earlier I led the AI framework team at VirtAI Tech, working on GPU virtualization; before that, computer vision at Horizon Robotics, and networking software at Cisco Systems in San Jose. I hold an M.S. in Computer Science and a B.S. in Electronic Engineering, both from Tsinghua University.

Lately I have been building HuggingArch, a harness that makes the arithmetic of inference cost automatic and verifiable.

Latest News

• 2026.08 — Released HuggingArch: give it any model open on HuggingFace and it derives a validated architecture spec, then computes KV cache, parallel sharding and inference throughput on top of it.
• 2026.07 — Kimi K3 released — tech report.
• 2026.06 — Joined Moonshot AI as a Member of Technical Staff.
• 2025.09 — Launched PaperCache, a paper-reading blog built on LLM close reads rather than summaries — the design principles behind it.
• 2025.03 — Spoke on AI compilers at GTC 2025 (S72642), and published the DeepSeek inference-efficiency series — the first public work to establish DeepSeek's deployment ceilings on H800 and H20.
• 2024.12 — Wrote up GPU clock throttling and GPU-to-GPU copy over PCIe.
• 2024 — Spoke at DataFun Summit 2024 on compute-engine optimization for recommendation, search and advertising.

Selected Writing

• DeepSeek V3/R1 inference efficiency, in three parts — a throughput ceiling from the paper alone, a layer-by-layer reverse-engineering of the published EP144 deployment, and a simulator generalizing it across device counts and hardware
• GPU clock throttling — why a large GEMM never reaches a card's rated TFLOPS, measured across T4, A10, A800 and H800
• GPU-to-GPU copy over PCIe — from cudaMemcpyAsync to a custom vectorized kernel, against NCCL

All posts are on the blog.

Projects

• HuggingArch — a harness making LLM inference cost analysis automatic, verifiable and reusable for any model open on HuggingFace.
• PaperCache — a paper-reading blog built on LLM close reads, covering machine learning systems, large models and AI accelerators.
• DeepSeek_Simulator — a decode-configuration simulator over DeepGEMM, FlashMLA and torch.

Talks

• Unlocking the Potential of the AI Compiler in Recommendation Systems [S72642] — GTC 2025
• Compute-engine optimization for recommendation, search and advertising at Kuaishou — DataFun Summit 2024 中文

---

Feel free to drop me a line — [email protected]