Vincent Zhong — Writing https://vincentzed.com/writing 2026-10-13T00:00:00Z Vincent Zhong The Training and RL Infrastructure Behind MAI-Thinking-1 https://vincentzed.com/writing/mai-thinking-1-infrastructure 2026-10-13T00:00:00Z Hardware, numerics, the YOLO training framework and the Rocket RL stack behind Microsoft AI's 35B-active, 1T-total MoE, read from its technical report. Porting Block-Scaled GEMM Kernels from Blackwell to Rubin https://vincentzed.com/writing/rubin-sm107-kernels 2026-10-13T00:00:00Z What changes for a tcgen05 GEMM on sm_107a, read from FlashInfer's Rubin kernels: wider K, packed FP4, bigger SMEM and TMEM, new scale formats, and a mixed-cluster scheduler. How Tokens Reach Their Experts During LLM Inference https://vincentzed.com/writing/a2a-backends 2026-10-11T00:00:00Z How SGLang moves each token to the GPUs that hold its experts and back: the twelve values of --moe-a2a-backend, from a plain all-gather to DeepEP and FlashInfer, and when each one pays off. How GPUs Divide Up the Requests During LLM Inference https://vincentzed.com/writing/dp-attention 2026-10-11T00:00:00Z On GLM-5.3, DP attention lets 4 GPUs hold 3.3 times more context, and makes a lone request almost twice as slow. Where that time goes, and at which load the trade is worth it. Serving Kimi K3 End to End: 8 GPUs, 32 GPUs, and a Rack https://vincentzed.com/writing/kimi-k3-deployment 2026-10-11T00:00:00Z Serving Kimi K3 with SGLang at three scales: one 8-GPU server measured, 32 GPUs as one wide expert-parallel domain, and an NVL72 rack with prefill and decode split. How GPUs Fuse Communication and Computation During LLM Inference https://vincentzed.com/writing/megamoe 2026-10-11T00:00:00Z DeepGEMM's Mega MoE runs dispatch, both expert GEMMs, and combine as one kernel per GPU. Where the saved time comes from, and what its multi-gigabyte buffer costs. What Matters in a ModelOpt Quantization Recipe https://vincentzed.com/writing/modelopt-quantization 2026-10-11T00:00:00Z How FP8 and NVFP4 work, and what each field of a ModelOpt quantization recipe changed on a model served with SGLang. How a Row Meets Its Experts https://vincentzed.com/writing/moe-expert-parallelism 2026-10-11T00:00:00Z Mixture-of-Experts parallelism in SGLang, from first principles: five ways for a row to reach its experts across GPUs, measured for decoding and prefill on 8 GPUs. How an MoE Model Picks Its Experts During LLM Inference https://vincentzed.com/writing/moe-router-gemm 2026-10-11T00:00:00Z The small matrix product and top-k that route every token in a Mixture-of-Experts layer: what it costs in SGLang, and how far a fused kernel takes it. How GPUs Avoid Waiting for the CPU During LLM Inference https://vincentzed.com/writing/speculative-decoding 2026-10-11T00:00:00Z SGLang's overlap scheduler, from first principles: how the next step is queued before the CPU has seen the current one, and what speculative decoding adds. How GPUs Share a Long Context During LLM Inference https://vincentzed.com/writing/context-parallelism 2026-10-08T00:00:00Z Prefill and decode context parallelism in SGLang, from first principles: why splitting a long context by token takes two separate features, and what each one costs. Keeping Speculative Decoding Fast During RL Training https://vincentzed.com/writing/verl-speco 2026-10-02T00:00:00Z Why a drafter goes stale while RL changes the model it drafts for, and how verl-SpeCo, slime, Miles, and SpecForge keep it up to date.