Skip to main content

read_completion_stream

Function read_completion_stream 

Source
pub async fn read_completion_stream<S: ChunkSource>(
    source: &mut S,
    request_body: Value,
    max_bytes: u64,
    on_delta: impl Fn(StreamDelta),
    started: Instant,
    now: impl Fn() -> Instant,
) -> Result<Completion, CompletionError>
Expand description

Reads a streamed model reply from source and assembles it into a [Completion].

The reply arrives as server-sent events (SSE). The function reads events until the [DONE] sentinel and fails if the stream exceeds max_bytes bytes. It passes each StreamDelta to on_delta as soon as it is decoded, so a Host can show the reply as it arrives. The returned completion holds the whole turn either way.

request_body is the body the transport sent, as build_request_body returned it. The completion carries it back, so a run’s debug capture records exactly what was sent. The completion is labeled with the model that request_body names.

started is the transport’s clock reading from just before it sent the request, and now reads that same clock. The completion’s [ClientTiming] holds three figures measured against them: time to first token, mean inter-token latency, and end-to-end time. It takes every clock reading from started and now.

§Errors

Returns a MalformedResponse-kind CompletionError when the stream exceeds max_bytes or ends before the sentinel, and the source’s own error when a read fails. Also returns the error that reassembling the reply raises for a malformed chunk, a mid-stream error envelope, a truncated tool-call batch, or an empty turn.