Struct GatewayBroker 

pub struct GatewayBroker { /* private fields */ }

A model broker that sends the Harness’s model rounds to the PromptForge Gateway and lists the Gateway’s models.

A Host passes this broker to the Harness as its [InferenceBroker]. The broker performs each Chat effect from the Engine as one model round on a GatewayChat. It lists the Gateway’s models through fetch_model_catalog. Both kinds of request go to the same API root and authenticate with the same bearer key.

Every round uses the Engine’s default run limits from [RunLimits::new]: the wait limit for each part of the response and the cap on response size in bytes.

The broker’s futures, models included, need a tokio runtime with its reactor and timer. A Host that uses this broker must therefore await Harness::run inside such a runtime. The Harness’s own futures run on any executor.

Implementations

impl GatewayBroker

pub fn new(endpoint: GatewayEndpoint, key: SecretString) -> GatewayBroker

Creates a broker that reaches the Gateway at endpoint with the bearer key key.

endpoint is the Gateway’s OpenAI-compatible API root, the /v1 path. Model rounds and model-list requests both go to it.

pub fn chat_streaming( &self, _binding: ModelBinding, messages: Vec<Message>, tools: Vec<ToolSchema>, options: CompletionOptions, on_piece: Arc<dyn Fn(StreamDelta) + Send + Sync>, ) -> BoxFuture<Result<Box<Completion>, CompletionError>>

Runs one model round and passes each piece of the reply to on_piece as it arrives.

The round is the same one InferenceBroker::chat runs. on_piece receives the pieces in the order the stream delivers them. The returned completion holds the whole reply.

on_piece is called inline as each piece is read, so it must not block.

Trait Implementations

impl Clone for GatewayBroker

fn clone(&self) -> GatewayBroker

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GatewayBroker

fn fmt(&self, formatter: &mut Formatter<'_>) -> Result

The bearer key is never written to logs or Debug output.

impl InferenceBroker for GatewayBroker

fn models(&self) -> BoxFuture<Result<ModelCatalog, CompletionError>>

Lists the models the broker serves.

fn chat( &self, _binding: ModelBinding, messages: Vec<Message>, tools: Vec<ToolSchema>, options: CompletionOptions, _round: Round, ) -> BoxFuture<Result<Box<Completion>, CompletionError>>

Runs one model round and returns the finished reply.