How an LLM generates a response
Follow one prompt through request routing, prefill, and the decode loop. Watch the KV cache grow as the server streams the answer.
xiaonan.co
Applied AI · ML Systems · Product
Follow one prompt through request routing, prefill, and the decode loop. Watch the KV cache grow as the server streams the answer.
An overview of NLP’s evolution from feature engineering to prompt-based learning, with applications at Agoda. Published in Agoda Engineering & Design.
Exploring practical applications of AI in technology and finance.
How to make AI agents and the workflows around them more reliable.
What it takes for AI systems to work dependably as their use grows.