harness_gateway_client/
lib.rs

1//! The standard client a Host uses to run model rounds, list models, and
2//! search the web through the PromptForge Gateway.
3//!
4//! [`GatewayBroker`] is the [`harness::InferenceBroker`] that a Host using
5//! the Gateway passes to `Harness::new`. It runs every model round on a
6//! [`GatewayChat`] under the Engine's default run limits, and it lists the
7//! Gateway's models through [`fetch_model_catalog`]. A Host that shows a
8//! reply as it forms runs the round through
9//! [`GatewayBroker::chat_streaming`], which hands each [`StreamDelta`] to
10//! the Host's callback as it arrives.
11//!
12//! [`GatewayChat`] is the HTTP client that sends a `Chat` effect's round
13//! to one Gateway URL. It presents the Gateway's shared bearer key when
14//! built with one. Every request streams: the Gateway's
15//! `/chat/completions` endpoint answers with server-sent events (SSE).
16//! [`GatewayChat::complete`] sends the request body and reads the stream
17//! within the client's byte cap and per-receive timeout. It calls the
18//! caller's delta callback as each piece arrives and returns the one
19//! completion the round produced. [`fetch_model_catalog`] fetches the
20//! Gateway's model list as a typed `ModelCatalog`. The client holds no
21//! credential except the Gateway's shared key. The model vendor's
22//! credential stays in the Gateway.
23//!
24//! [`GatewaySearch`] is the [`harness_web::SearchProvider`] that a Host
25//! using the Gateway supplies for web search. It sends each search through
26//! the Gateway's `/tools/web_search` relay with a 30-second deadline and
27//! maps the reply into the provider's results. The search vendor's
28//! credential also stays in the Gateway.
29//!
30//! Under the client sits the OpenAI chat-completions wire code. It turns a
31//! round into a request body, a streamed reply into a
32//! [`Completion`](promptforge::model::Completion), and a failed response
33//! into the [`CompletionError`] the round fails with.
34//!
35//! - [`build_request_body`] builds the JSON body that every round sends.
36//! - [`read_completion_stream`] reads a streamed reply from a
37//!   [`ChunkSource`] the caller supplies, up to its `[DONE]` sentinel and
38//!   within a byte cap. It forwards each live delta, assembles the stream
39//!   into one completion, and checks that turn against one strict set of
40//!   rules.
41//! - [`read_body_capped`] reads, within a byte cap, a body that the caller
42//!   decodes whole.
43//! - [`escape_controls`] cuts a backend error body to a length limit and
44//!   escapes its control characters.
45//! - [`classify_http_failure`] turns a failure status and its body into a
46//!   `CompletionError` of the matching kind. [`classify_stream_error`] does
47//!   the same for an error envelope that arrives inside the stream.
48//!
49//! The client, or another broker, owns the connection and supplies the
50//! wire code's chunks and clock.
51//!
52//! ## Invariants
53//!
54//! - Every `Completion` and `ToolCall` is built through the Engine's public
55//!   validating constructors, so the Engine's model-independent reply
56//!   checks run on every decoded turn.
57//! - A Gateway bearer key is never written to logs, `Debug`, `Display`, or
58//!   error text.
59//! - A backend error body is bounded and control-escaped before it is
60//!   kept. A chat round keeps it only in the opt-in
61//!   `CompletionError::detail`, and a search keeps it in the
62//!   `GatewaySearchError` message.
63//! - A client sends requests without a key only when the caller builds it
64//!   with `GatewayChat::keyless`, which does not check the endpoint's
65//!   address, or when `GatewayChat::from_env` finds no key for a loopback
66//!   URL. A client built with `GatewayChat::disabled` also holds no key,
67//!   but it sends no requests.
68
69mod broker;
70mod catalog;
71mod config;
72mod failure;
73mod search;
74mod transport;
75mod wire;
76
77pub use broker::GatewayBroker;
78pub use catalog::fetch_model_catalog;
79pub use config::GatewayConfigError;
80pub use config::GatewayEndpoint;
81pub use config::SecretError;
82pub use config::SecretString;
83pub use promptforge::model::CompletionError;
84pub use promptforge::model::CompletionErrorKind;
85pub use search::GatewaySearch;
86pub use search::GatewaySearchError;
87pub use search::GatewaySearchErrorKind;
88pub use transport::GatewayChat;
89pub use wire::classify::classify_http_failure;
90pub use wire::classify::classify_stream_error;
91pub use wire::delta::StreamDelta;
92pub use wire::read::ChunkSource;
93pub use wire::read::read_body_capped;
94pub use wire::read::read_completion_stream;
95pub use wire::request::build_request_body;
96pub use wire::stream::escape_controls;
harness_gateway_client/lib.rs

harness_gateway_client/
lib.rs
