Blog
Notes on technical exploration.
- HuggingArch: Automating Model Architecture Analysis
An agent writes an open model up as a checked spec; one evaluation of that spec gives its architecture, cost and capacity under any deployment — no weights, optional GPU.
- PaperCache: Close Reading of Papers with an LLM
A paper-reading blog built on LLM close reads rather than summaries — why existing tools did not fit, and the seven design principles behind it.
- DeepSeek V3/R1 Inference Efficiency (3): Generalizing the Decode Configuration
A simulator built on DeepGEMM, FlashMLA and torch, sweeping DP-EP and TP-DP-EP configurations on H800 and H20 — what device count, batch size, KV cache dtype and overlap scheme actually maximize throughput under a latency SLO.
- DeepSeek V3/R1 Inference Efficiency (2): Reverse-Engineering the Production Deployment
Layer-by-layer profiling of DeepSeek's published EP144 setup — prefill and decode FLOPs, per-operator timings, MFU, and the overlap schedule that follows from them.
- DeepSeek V3/R1 Inference Efficiency (1): A Back-of-the-Envelope Decoding Throughput Ceiling
Before DeepSeek published its inference-system numbers, how close could you get from the V3 paper alone? A memory-and-compute budget for Attention DP + MoE EP, and what it says about the achievable tokens/s per H800.
- GPU Clock Throttling: Why You Never Reach Peak FLOPS
When a large GEMM falls short of a GPU's rated TFLOPS, the vendor library is usually not the problem — the power budget is. Measured on T4, A10, A800 SXM/PCIe and H800 SXM.
- GPU-to-GPU Copy over PCIe: From cudaMemcpyAsync to a Custom Kernel
How much bandwidth can a device-to-device copy actually reach between two GPUs on a PCIe topology? Four implementations measured on A800 PCIe and RTX 4090, and compared against NCCL.