Skip to documentation

Reference · Streaming physics

Available

Last verified

Test the stream, not just the final text

A client can parse a complete JSON response correctly and still fail on slow first tokens, partial chunks, uneven pacing, or a stream that stops early. Fixture streaming settings make each of those behaviors explicit and repeatable.

In this guide

  • Every streaming field and default
  • How delays are computed
  • Truncation and non-streamed latency
  • Scripted WebSocket events

Prerequisites

  • A hosted LLM DotMock workspace
  • The workspace ID in $API_ID and its runtime URL in $MOCK_URL (SDK base URL $MOCK_URL/v1 for OpenAI-compatible clients)
On this page
  1. When a response streams
  2. Streaming fields
  3. How each delay is chosen
  4. Truncated and broken streams
  5. Describe WebSocket events directly

When a response streams

A response streams when the client asks for it the provider's way: stream: true for OpenAI, Anthropic, Azure, and Cohere; :streamGenerateContent for Gemini; converse-stream or invoke-with-response-stream for Bedrock; and Ollama's default stream behavior. The same fixture serves both streamed and non-streamed requests, and each provider gets its own envelope (see Providers and endpoints).

bash
curl -N "$MOCK_URL/v1/chat/completions" \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"hello"}]}'

Streaming fields

Set these under a fixture's streaming object. Content, reasoning, and tool-call arguments are split into chunks of chunkSize characters (runes), and each chunk becomes one provider event.

  • chunkSize — characters per chunk, default 20
  • ttftMs — delay before the first chunk (ttft is accepted as an alias)
  • tps — target chunks per second; the inter-chunk delay is 1000 / tps milliseconds
  • latencyMs — fixed delay between chunks when tps is not set, default 0
  • jitter — fraction of random variation applied to each delay; 0.2 means ±20%
  • interChunkDelaysMs — explicit list of delays; entry i is used before chunk i + 1
  • totalDurationMs — spread the remaining time evenly over all chunks after the first
  • replaySpeed — divide every delay; 2 plays a recorded rhythm twice as fast
  • truncateAfterChunks — stop cleanly after N content chunks without the normal terminal event
Streaming settings
"streaming": {
  "chunkSize": 8,
  "ttftMs": 900,
  "tps": 25,
  "jitter": 0.2
}

Expected result: About 900 ms before the first chunk, then a chunk roughly every 40 ms (±20%).

How each delay is chosen

For the first chunk DotMock waits ttftMs (or ttft). For every later chunk it uses the first rule that applies: the matching interChunkDelaysMs entry, then an even share of totalDurationMs, then 1000 / tps, then latencyMs. Jitter is applied to the chosen delay and the result is divided by replaySpeed.

  • Use ttftMs to test loading indicators and first-byte timeouts
  • Use tps and a small chunkSize to test incremental rendering and back-pressure
  • Use jitter to catch consumers that assume uniform pacing; pin X-Dotmock-Seed so the jitter is identical on every run
  • Use interChunkDelaysMs to reproduce a specific stall, such as a long pause mid-answer

Truncated and broken streams

truncateAfterChunks ends the stream early and cleanly: the connection closes without [DONE], message_stop, response.completed, or the provider's equivalent terminal event. Assert that your client clears its loading state, keeps or discards the partial text as intended, and offers a retry. For malformed frames and hard disconnects, use chaos settings, which are probabilistic and seedable.

Truncation
"streaming": { "chunkSize": 10, "latencyMs": 30, "truncateAfterChunks": 3 }

Describe WebSocket events directly

A fixture with protocol ws or both can script WebSocket output: ordered typed events with per-event delays, then close normally, close with a chosen code, or disconnect after a duration. WebSocket fixtures replay these events as written; they do not apply HTTP streaming physics, chaos, sequences, or templates.

json
"ws": {
  "events": [
    { "type": "response.created", "data": { "id": "resp_test" } },
    { "type": "response.output_text.delta", "data": { "delta": "Ready" }, "delayMs": 40 },
    { "type": "response.completed", "data": { "id": "resp_test" } }
  ],
  "closeAfter": true,
  "closeCode": 1000
}