@ycombinator built AI versions of its partners to help more people work through their startup ideas. For its Office Hour Simulator, the team needed useful answers delivered quickly enough for a spoken conversation. They had been testing lightweight Gemini and OpenAI models before moving to GLM-5.2 on a dedicated Wafer endpoint. YC then compared that deployment with GPT-4.1 mini on OpenAI and Gemma 4 31B on Cerebras, evaluating answer quality, latency, and conversation duration. The Wafer configuration delivered 31% lower average LLM latency than OpenAI and 44% lower than Cerebras. Users spent 2.5 minutes longer talking to YC’s AI partners when using Wafer. Read how YC built the experience and landed on Wafer 🧵 link in thread

Look at that latency 😍 As proven by 25+ years of internet services, speed is a feature. For any app where there is a consumer (or agent!) on the other side waiting for the result, being even a bit faster results in higher conversion (and lower drop-off / churn) @wafer_ai is the choice for low latency inference 📈🚀

*overheard at 2nd street blue bottle* larper 1: "bro Attention is All You Need is one of the most important papers in ai." larper 2: "yes bro it's so good" larper 1: "have you read it?" larper 2: "no, have you?" larper 1: "no" don't be like these guys. pls read

who has actually read this paper? prob the most consequential paper in modern ai btw stop larping and read it. this is only the second resource in the ai performance engineering repo imagine the ball knowledge in the other ones

you'll know more about inference than 90% of people if you fully understand this article this is only the first resource in the ai performance engineering repo btw. imagine the ball knowledge in the other ones

"The biggest bottleneck in the GPU industry at the moment is not chips, CoWoS, or even power. It's credit." our head of compute @gpugene on how uncertain GPU residual values shape lending terms and the capital required to bring new compute online. link to article in thread 🧵

we launched the most comprehensive ai performance engineering repo in the world last week now we'll be doing a deep dive on every single resource this is Wafer's AI Performance Engineering Series save this as your starting point. links in thread 🧵 Part 1: Prefill vs Decode a causal autoregressive Transformer computes next-token logits from the supplied prefix. a decoding policy selects one token and appends it before the model predicts the next. in a conventional causal decoder, each position attends only to itself and earlier positions. appending a token adds no allowed input to old positions. in evaluation mode, with the prefix, weights, mask and positional computation fixed, the old representations remain reusable. the KV cache stores earlier positions' per-layer keys and values for reuse in later forwards. prefill processes the known prompt under a causal mask and saves each layer's keys and values. the final prompt position produces logits for choosing the first output token. prompt positions can run together within a layer; the layers still depend on one another. the decoding policy selects a token. greedy decoding takes a highest-scoring token; sampling draws from a distribution. feeding the selected token back through the model creates its K/V and produces the logits for the next output. emitting a token and processing it are separate steps. to emit N ≥ 1 tokens, an ordinary loop needs one unchunked prefill and N−1 incremental forwards. emitting the final token does not require processing it through the model. the new token still runs through the layers, including its projections and MLP. its query computes new attention scores and a weighted sum over cached values. storing K/V saves repeated prefix computation; it leaves new attention work over the growing context. for fixed prompt length and model dimensions, caching changes total projection and MLP work from quadratic to linear in output length. full causal attention across generation remains quadratic. these work counts do not establish a measured latency improvement. prefill has many known token rows from one prompt. ordinary decode contributes one new row per active request, so batching it combines different requests. each request also needs its own logical KV state. more concurrent requests and longer retained contexts increase that state. choose the performance target around the workload. offline generation may prioritize completed work per dollar within a deadline. streaming chat also needs low time to first token and responsive delivery afterward. a single-user device may favor latency within its memory and compute limits. for a serving comparison, fix prompt/output lengths and concurrency. measure client time to first token, the intervals between later tokens, and total output tokens over a common measurement window. keep model-only prefill timing separate from queueing and delivery. a higher aggregate token rate alone cannot tell you whether one user's answer arrives sooner. chunked prefill and speculative generation change the execution pattern and are outside this ordinary-loop example.

wafer is growing exponentially and there's orders of magnitude more customer demand than we can attend to. we're 8 people supporting millions of ARR with triple digit month over month growth. we just announced our $40m series A to build the team that can capture that demand and make Wafer the leading inference company in the world. four roles that matter the most right now: member of technical staff work across kernels, inference engines, serving infrastructure, and heterogeneous clusters. most of your time is engineering, and you own the customer workload from benchmark through production. $250k base + plus equity. founding growth build Wafer’s marketing function with the founders and gtm lead. own positioning, founder-led social, launches, events, customer stories, the website, and distribution. $140k–$180k. chief of staff work with the ceo on whatever matters most across customers, recruiting, finance, legal, operations, and strategy. this role is for someone who can take an incomplete thought and return with a finished outcome. gtm become our first dedicated top-of-funnel hire. create qualified meetings with ai-native companies, turn them into benchmarks, and grow into full-cycle technical sales. $100k base, $140k ote, uncapped commission, plus equity. all four roles are full-time and on-site in San Francisco, five days a week. if you've ever done anything exceptional, reach out. we'd love to chat. links in comments.