How we made DeepSeek V4.1 Flash faster: five serving setups and the one we kept
The setup that did the most total work made people wait up to 80 seconds for a first token. We kept a newer vLLM build with our existing settings, learned where the time inside each decoding step goes, and then found that the slowest part of a request was not on the GPU at all.

Last weekend, DeepSeek V4.1 Flash 1017 was unlimited for every Pro and Ultra user on Perch, and on Sunday evening, for everyone on every plan. It ran on GPUs we serve ourselves, as we described in what a weekend on our own GPUs taught us.
Before the weekend started we wanted it faster for the moments when many people use it at once. So early Saturday morning we tested five serving setups with the same load test, picked one, and had it serving everyone by 4:52 AM ET.
Then, with the new setup live, we found that the slowest part of a request was not on the GPU at all.
Key findings
- A newer vLLM build with our existing settings won. It was the only setup that was fast for one person and stayed fast for a crowd. Per-user generation speed went up as much as 38 percent in our load test, and the median production call streamed about 15 percent faster.
- More total work is not more speed. One ATOM setup did 7 to 13 percent more total work than our pick, but people waited 22 to 80 seconds for a first token and a single user got half the speed.
- Data-parallel attention cut the worst waits from 71 seconds to 5.2 seconds, but only if every turn of a conversation lands on the same attention copy.
- One scheduler setting decided ATOM's results. The recipe default held new requests back, and only about 3 people at a time were being served.
- The slowest first tokens came from the network. Agent prompts close to 1 MB sometimes stalled in an HTTPS proxy for 4 to 12 seconds. Compressing them and re-sending stuck requests halved the share of first tokens slower than 5 seconds.
The test
The load test replays what an agent session looks like on Perch. Each simulated user has a conversation history of about 146,000 tokens and takes three turns, adding 6,000 to 8,000 new tokens each time and getting a reply of about 1,200 tokens. The first turn reads the whole history. The next two find it in the prompt cache, the way a real follow-up does. We ran it for 1, 8, 16, 32, 48 and 64 simultaneous users.
For every setup we looked at two numbers, because they answer different questions:
- Generation speed per user: how fast the answer streams for each person, in tokens per second.
- Wait for a returning user: how long the slowest 5 percent of follow-up turns waited for the first token, when the conversation was already in the prompt cache.
A server can do more total work and still feel slower to every individual person. That turned out to be the main story.
The five setups
| Setup | What we changed | Result |
|---|---|---|
| Newer vLLM build | Moved to vLLM's November 14 nightly build, settings unchanged | Kept |
| New prefill schedule | Changed how much new context is read between decoding steps, and how often | No speed gain, waits of 70 s at 32 users. Dropped |
| Data-parallel attention | Each GPU runs its own copy of attention, with a router keeping each user on one copy | Shortest peak waits. Kept as an option |
| ATOM, tuned for first token | AMD's ATOM engine, prefill coalescing off | Shortest waits overall, 4 to 14% less total work. Dropped |
| ATOM, data-parallel attention | ATOM with each GPU running its own attention | 7 to 13% more total work, waits of 22 to 80 s. Dropped |
The winner: a newer build, same settings
| Simultaneous users | 1 | 8 | 16 | 32 | 48 | 64 |
|---|---|---|---|---|---|---|
| Generation speed per user (tokens/s) | 893 | 308 | 186 | 136 | 101 | 88 |
At 32 users, the slowest 5 percent of returning turns waited 1.1 seconds for a first token. There were no errors.
Our earlier test of the previous build gave 729 tokens per second for one user, 102 to 109 at 32 and 64 to 68 at 64. That test used an older version of our load generator, whose prompts our draft model predicts a little less well, so treat the 22 to 38 percent gap as an upper bound. In production, at similar load, the median call streamed about 15 percent faster after the switch.
The weak spot is a burst at heavy load. At 48 and 64 users, when many long conversations arrive at once, the slowest returning turns waited 23 and 71 seconds.
Data-parallel attention: shorter waits, one catch
In the default setup, every GPU works on every request together. With data-parallel attention, each GPU runs attention for its own share of conversations and keeps its own prompt cache, while the expert layers are still shared across GPUs.
| Simultaneous users | 1 | 8 | 16 | 32 | 48 | 64 |
|---|---|---|---|---|---|---|
| Generation speed per user (tokens/s) | 536 | 246 | 183 | 124 | 103 | 83 |
It is 40 percent slower for a single user and within 10 percent of the winner from 16 users up. Where it shines is the burst: the slowest returning turns waited 1.27 seconds at 48 users and 5.2 seconds at 64, against 23 and 71 seconds for the winner.
The catch is the cache. Because each attention copy has its own prompt cache, every turn of a conversation has to go to the same copy. The router decides that from a user ID sent with each request. On our first run that ID was missing, turns scattered across copies, the cache hit rate fell to 46 percent, and the results were useless. In production it would mean sending a stable ID with every call. It is our option if peak waits become the main problem.
ATOM: one setting decided everything
ATOM is AMD's own inference engine, built on its AITER kernels. We ran it in its own environment next to the same vLLM build (ATOM's latest code needed a newer AITER than that build ships, so we used the most recent commit that worked with it).
With the recipe's settings, ATOM looked great per user, and that was misleading. The recipe sets a prefill interval of 40, and any value above 0 turns on ATOM's prefill coalescing, which batches up new requests before reading them. At 32 users, only about 3 conversations were generating at any moment while the rest waited. Total output was 39 percent lower than vLLM's, and the median call took 44 seconds instead of 10.6. Per-user speed only looked good because so few people were being served.
There were real strengths underneath. At 1 and 8 users, each decoding step was as fast as vLLM's, and with one user ATOM read new context twice as fast (49,000 tokens per second against 23,000).
Then we tuned it:
- Coalescing off (the setting at 0). The shortest waits of any setup: under 1 second for the slowest 5 percent of returning turns, from 32 all the way to 64 users. But 4 to 14 percent less total work than vLLM.
- Data-parallel attention. 7 to 13 percent more total work than vLLM from 16 to 64 users, and new context read at 68,000 to 77,000 tokens per second. But returning users waited 22 to 80 seconds for a first token, and a single user got about half of vLLM's speed. This is the setup that did the most work and felt the slowest.
Two things ruled ATOM out regardless of speed. It ignored request priority and served everything first come, first served, and our scheduling depends on priority when the server is busy. And it refused images whenever speculative decoding or data-parallel attention was on, and agents read screenshots all the time.
Where the time goes inside a step
We profiled the winner at 32 users. One decoding step took about 34 milliseconds:
| Part of the step | Share of time |
|---|---|
| Expert layers | 43% |
| Block-scaled FP8 matrix math | 17% |
| Combining results across GPUs | 11% |
| Attention | 8.5% |
| Everything else | about 20% |
vLLM also has a fused, single-kernel decode path for this model. It only helps at 8 simultaneous users or fewer. Speculative decoding does a lot of the work at every load: the draft model proposes 5 tokens per step, and in production each step produced 3.6 to 4.2 tokens on average.
So scheduling changes were never going to be the big lever. The next real gain is in the expert-layer kernels.
The slowest part was not the GPU
With the new build live, the slowest first tokens in production still bothered us. On Saturday afternoon the slowest 5 percent of turns waited 10 to 11.6 seconds.
Inside the node, a large agent prompt got its response headers in 0.31 seconds and its first token in about 0.45 seconds. From our servers, the same prompt sometimes waited 4 to 12 seconds before anything came back.
The cause was the size of the request. An agent re-sends its whole conversation as JSON on every turn, and that body can be close to 1 MB. Through the HTTPS proxy in front of our GPUs, 52 KB and 208 KB requests never stalled. Requests of about 980 KB stalled in 1 to 4 of every 10 tries.
We fixed it in two steps, without restarting the model server:
- Re-send stuck requests. If a large streamed request has no response headers after 2 seconds, we send it once more. Whichever copy answers first wins, and the other is cancelled.
- Make the request smaller. Our edge relay now compresses the request on its last hop, and a small process on the node decompresses it before vLLM sees it. Requests travel about 10 times smaller.
For prompts of 150,000 tokens or more:
| First tokens over 5 s | Median | Slowest 5% | |
|---|---|---|---|
| Before | 9.2% | 1.47 s | 7.7 s |
| Re-send only | 8.1% | 1.66 s | 8.4 s |
| Re-send and compression | 4.6% | 1.48 s | 4.8 s |
Re-sending alone barely helped, because the second copy was just as big as the first. Making the request smaller is what fixed it. The median did not move, because most requests never stalled. The tail is what changed.
What we would tell anyone tuning a model server
- Measure what one person feels, not just total work. The setup with the most total work in our test was the one that made people wait the longest.
- Try the newest build of your engine before tuning anything else. It was our biggest win, and it took no new settings.
- Read the scheduler defaults. One setting made ATOM serve 3 people at a time and look fast doing it.
- Data-parallel attention needs sticky routing. Each copy has its own prompt cache, so a conversation has to stay on one copy.
- Profile the whole request path. Last time our bottleneck was a CPU process in front of the GPU. This time it was the size of the request on the network. Both times the GPU was fine.
Frequently asked questions
- How do you make DeepSeek V4.1 Flash faster on vLLM?
- In our tests, the biggest gain came from moving to a newer vLLM nightly build and keeping our existing settings. Each user's generation speed went up as much as 38 percent in our load test, and the median production call streamed about 15 percent faster at similar load. Changing how new context is batched did not help. Most of each decoding step is spent in the expert layers, so the next real gains are in those kernels.
- Is AMD's ATOM faster than vLLM for DeepSeek V4.1 Flash?
- It depends on what you measure. With data-parallel attention, ATOM did 7 to 13 percent more total work than our vLLM setup, but returning users waited 22 to 80 seconds for a first token and a single user got about half the speed. Tuned for first token instead, ATOM had the shortest waits of any setup but did 4 to 14 percent less total work. No ATOM setup beat vLLM on both. ATOM also ignored request priority and refused images whenever speculative decoding or data-parallel attention was on.
- What is data-parallel attention and when does it help?
- Each GPU runs its own copy of attention with its own prompt cache, while the expert layers stay shared across GPUs. In our test it cut the slowest first-token waits at our heaviest load from 71 seconds to 5.2 seconds, at the cost of slower speed for a single user. It only works if every turn of a conversation goes to the same attention copy. Without that, the prompt cache hit rate fell to 46 percent.
- Why are first tokens slow for long AI agent prompts?
- Sometimes it is not the model. An agent re-sends its whole conversation every turn, so a request can be close to 1 MB. In our case, large requests sometimes stalled for 4 to 12 seconds in an HTTPS proxy in front of the GPU, while the model itself started answering in under half a second. Compressing the request on the last hop and re-sending any large request that had no answer after 2 seconds cut the share of first tokens slower than 5 seconds from 9.2 percent to 4.6 percent.