Struct GatewayChat 

#[non_exhaustive]
pub struct GatewayChat { /* private fields */ }

A client that sends chat completion requests to one gateway URL.

The client usually presents the gateway’s shared bearer key on every request. The key is optional: by default, a gateway on the same machine admits keyless loopback callers. A keyless client (GatewayChat::keyless) omits the Authorization header entirely.

Implementations

impl GatewayChat

pub fn new(endpoint: GatewayEndpoint, key: SecretString) -> GatewayChat

Builds a client that sends requests to endpoint with key as the bearer key.

The endpoint is a validated GatewayEndpoint, and the key is a SecretString, which keeps it redacted. GatewayChat::from_env builds its client this way when a key is set.

pub fn keyless(endpoint: GatewayEndpoint) -> GatewayChat

Builds a client whose requests omit the Authorization header that carries the bearer key.

This client suits a gateway on the same machine, because such a gateway trusts keyless loopback callers, unless its operator set trust_loopback = false. On a shared machine, that trust also covers every other OS account there. Against any other gateway, the requests fail with an Unavailable-kind error when the gateway answers 401.

This constructor accepts an endpoint with any host, so the caller decides when a keyless client is appropriate. GatewayChat::from_env decides by GatewayEndpoint::is_loopback.

pub fn disabled() -> GatewayChat

Builds an offline client, for execution paths that must stay hermetic.

Any attempted model call fails with an Unavailable-kind CompletionError. The client reads no gateway configuration and sends no HTTP request.

pub fn with_request_limits( self, request_timeout: Duration, max_response_bytes: NonZeroU64, ) -> GatewayChat

Applies the run’s HTTP limits to this client: a timeout for each receive and a cap on the response size.

request_timeout is the longest a completion request waits for its response headers, and then for each next body chunk. Every chunk that arrives restarts the wait. A long stream that keeps arriving completes, and a stream that stalls fails as a timeout.

The client refuses the response body as soon as it would exceed max_response_bytes, before any UTF-8 or JSON decoding runs.

pub fn from_env() -> Result<GatewayChat, GatewayConfigError>

Builds a client from environment variables.

• PROMPTFORGE_GATEWAY_URL holds the gateway URL. It is required.
• PROMPTFORGE_GATEWAY_API_KEY holds the gateway’s shared bearer key. An empty value counts as missing. The key is optional when the URL’s host is loopback (127.0.0.1, ::1, localhost) and required for every other host. By default, a loopback gateway trusts keyless callers on the same machine, so for a loopback URL a missing key yields a keyless client.

That trust also admits every other OS account on a shared machine, so the gateway’s operator there sets trust_loopback = false. Then set the key, or a keyless client’s requests fail with an Unavailable-kind error when the gateway answers 401.

Errors

Returns a GatewayConfigError when PROMPTFORGE_GATEWAY_URL is missing or invalid, when either variable is set to a value that is not valid Unicode, or when the URL’s host is not loopback (a LAN or remote gateway) and PROMPTFORGE_GATEWAY_API_KEY is missing or empty.

pub async fn complete( &self, messages: &[Message], tools: Option<&[ToolSchema]>, options: &CompletionOptions, on_delta: impl Fn(StreamDelta), ) -> Result<Completion, CompletionError>

Sends a list of messages to the gateway and returns the model’s reply.

The request always streams. It asks for server-sent events (SSE) and sets stream_options.include_usage, so the stream ends with a summary chunk that reports token usage. The client reassembles the streamed fragments into the body a buffered chat completion response would carry. It calls on_delta with each text or reasoning fragment, as a StreamDelta, as soon as the fragment arrives. A caller that ignores the fragments passes an empty closure.

The returned [Completion] holds the reassembled turn, the metadata parsed from the stream’s summary chunk, and a ClientTiming measured on this client’s own clock: time to first token, mean inter-token latency, and end-to-end time.

When tools is Some and holds at least one schema, the request carries a tools array with one OpenAI function tool per schema: an object whose type is function and whose function holds the schema’s name, description, and parameters. The request also sets tool_choice to auto. Passing None or an empty slice omits the tools field, so the request is a plain chat completion.

The request names the model that options names, and the returned completion is labeled with that name, whatever name the response gave. The optional temperature, max_tokens, and thinking settings in options extend the request when present.

Errors

Returns a CompletionError whose kind is always one of the following:

• Unavailable when this client was built with GatewayChat::disabled, or when the gateway answers 401 or 403.
• Timeout when the request timeout elapses before the response headers or the next body chunk arrive.
• Transport on any other transport failure, such as a failed connection, or when the stream carries a mid-stream error envelope that names no known cause.
• ContextOverflow, RateLimited, QuotaExhausted, Overloaded, Refused, ServerError, or Rejected when the gateway answers with a status outside the 2xx range, as classify_http_failure reads it.
• MalformedResponse when the stream exceeds the response size cap, a chunk fails to parse as usable JSON (a JSON decode error is kept as the error’s source), or the stream ends before the [DONE] sentinel. It is also the kind when a length or content_filter finish reason cuts off a batch of tool calls, because partial arguments must not run.
• EmptyReply when the turn has zero tool calls and blank text.

Trait Implementations

impl Clone for GatewayChat

fn clone(&self) -> GatewayChat

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GatewayChat

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.