Introduction

PromptForge turns Markdown files into executable AI prompt pipelines. You write a prompt as a document - YAML frontmatter for metadata, embedded Lua for logic, prose blocks for model instructions - and the system parses, validates, and executes it against any OpenAI-compatible endpoint.

A PromptForge prompt is a single file. Sections run top to bottom. Lua blocks control flow, bind models, declare tools, and write to a virtual filesystem. The model sees the prose. The tool loop dispatches calls and feeds results back. Fanout maps a worker section over a list in parallel. The result is a string.

Components

PromptForge is a workspace of cooperating crates:

How to read this guide

This guide follows the user journey. Start with Getting Started to run your first prompt. Then read Prompt Files and Lua Scripting to understand the format. The remaining chapters cover each subsystem in depth.

If you are integrating PromptForge as a library, the Execution chapter explains the programmatic API. If you are deploying a model backend, start with Gateway.

Getting Started

This chapter walks you through your first PromptForge CLI invocation, explains how input and output work, and shows how to configure a gateway for remote capabilities.

Running Your First Prompt

The CLI binary is named promptforge. It has one command:

promptforge run <file.md> [input]

The file must be a PromptForge prompt. That means its YAML frontmatter must declare a promptforge: version. If it does not, the CLI refuses the file before attempting to parse it:

error: prompt.md is not a promptforge prompt: its frontmatter declares no `promptforge:` version

A valid prompt file is read from disk, parsed, and executed in-process. The binary links the PromptForge executor directly rather than connecting to an MCP server or any other service. This makes the CLI a development tool for the edit-run loop: you edit a prompt file, run it, and see the result immediately.

The simplest invocation takes just a file path:

promptforge run prompts/hello.md

Prompts are addressed by file path, not by name from a catalog. There is no configuration file, no resolution rule, and no catalog lookup. Shell completion, relative paths, and .. work as they do with any file argument.

Input and Output

Passing Input

The optional second argument is a raw input string that becomes the prompt's args value in its entirety:

promptforge run prompts/staker.md "Bloomberg"

The prompt body decides what that text means. The binary does not inspect, split, or coerce it. An input containing spaces must be quoted as a single shell argument.

Capturing Output

When the prompt completes, its returned value goes to stdout. Errors go to stderr. Nothing is mixed. On success, stdout contains exactly the returned value and nothing else. On failure, nothing appears on stdout.

This clean separation means shell substitution works:

report=$(promptforge run prompts/digest.md "2026-08")

The variable report captures exactly what the prompt returned.

Gateway Configuration

Gateway credentials come from two environment variables:

• PROMPTFORGE_GATEWAY_URL - the gateway base URL
• PROMPTFORGE_GATEWAY_API_KEY - the bearer token

There are no CLI flags for credentials. This is deliberate: secrets never appear in argv, where ps and shell history can expose them.

Local-Only Mode

Local-only mode is the default. With neither variable set (or with empty values), the CLI runs without a gateway. The web_fetch tool is available, but there is no web_search and no remote model catalog. A prompt that makes no model calls works entirely self-contained in this mode.

Remote Mode

Remote mode activates when both variables are set:

export PROMPTFORGE_GATEWAY_URL="https://gateway.example.com/v1"
export PROMPTFORGE_GATEWAY_API_KEY="your-bearer-token"
promptforge run prompts/search-demo.md "latest Rust news"

This enables the web_search tool and fetches the remote model catalog, so prompts can perform inference through the gateway.

Setting a key without a URL is rejected explicitly:

error: PROMPTFORGE_GATEWAY_API_KEY is set but PROMPTFORGE_GATEWAY_URL is missing or empty; both are required to reach the gateway

File IO

A prompt can declare input and output files. This lets callers pass file content into the prompt's store before execution and collect results from it afterward.

A minimal prompt with file IO:

---
name: summarize_paper
description: Summarize a paper in three bullet points
promptforge: 1
input:
  path: paper.md
  description: The paper to summarize
---

# Summarize

## Run

```lua
local content = store.read("paper.md")
```

Summarize the following paper in exactly three bullet points:

{= content =}

Call it via the MCP server:

{
  "prompt": "summarize_paper",
  "input_file": "/home/user/papers/p2996r7.md"
}

The server reads the file, seeds paper.md in the store, and the prompt accesses it through store.read() without knowing where the content came from.

Next, see Prompt Files for a detailed look at how prompt files are structured and parsed.

Prompt Files

A prompt file is a Markdown document with YAML frontmatter. The frontmatter must declare name and description. A promptforge: key identifies the file as a promptforge prompt - the runtime refuses files that lack a supported version number.

---
name: summarizer
description: Summarize a document into bullet points
promptforge: 1
---

Below the frontmatter, the document has one H1 title and zero or more H2 sections. A prompt with H2 sections walks them top to bottom in fall-through order. A prompt with no H2 sections executes the H1 blocks and returns the model reply. The H1 region always runs first, resolving tools and models before any section begins.

The walk is level-independent and never descends on its own: a section's children (H3 under H2, and so on) do not run in fall-through order. Control reaches a child level only when a section jumps to one of its own children - that starts a child-level walk within the jumper's children, beginning at the addressed child and falling through to its following siblings under the same rules (off-walk skips included). When the child level exhausts, the parent walk resumes at the section after the jumper, carrying the child walk's last reply. The rule recurses to deeper levels.

execute(target, input) starts a contained chain at the target: a walk with every normal rule (fall-through, off-walk skips, jumps, child chains) that never moves the outer walk. When the chain ends - its level exhausts or a return fires - the chain's final reply is the call's return value, and the caller continues. A return ends only the chain it fires in; the top-level walk's return ends the run.

Minimal Prompt File

---
name: hello
description: A greeting prompt
promptforge: 1
---

# Hello

## Greet

Say hello to the user in a friendly tone.

The parser compiles Lua code at parse time. A successfully parsed prompt is syntactically executable without any runtime compilation step - Lua syntax errors surface before any network call is made.

Structural Rules

The parser enforces strict structure:

• When H2 sections are present, the first and every root heading must be exactly H2.
• Sibling section names must be unique; duplicates produce a diagnostic naming both heading locations.
• Orphan deep headings (H4 under H2 with no H3) are rejected rather than silently reparented.
• Unknown frontmatter fields are rejected so misspelled keys fail loudly.
• Sections nest recursively using heading levels H2 through H6.
• Executable Lua fences must use exact unindented triple-backtick lua openers. Longer markers, indentation, or extra info-string words remain inert prose.

Parse errors report stable kind discriminants and optional byte spans for editor diagnostics. Lua compilation errors include absolute source-line numbers that map back to the original prompt file.

The --- Marker

A --- thematic break inside a section carries one of two roles, decided by position.

As a section's first content (only blank lines before it), the marker takes the section off the walk: the top-to-bottom walk skips it entirely, and it runs only when addressed directly by execute, jump, or fanout. Content below the marker executes normally. This is how a shared worker or a list section lives as a top-level section without running in the walk:

## Main

```lua
local reply = execute("## Helper")
```

## Helper

---

```lua
return "helper reply"
```

Anywhere else, the rule is a comment boundary: everything below it until the next heading is reader-only. No Lua below it compiles or runs, no prose below it reaches the model, and no list items parse from it. The two roles compose - a section may carry the off-walk marker at the top and a later rule starting a comment region. On the H1 only the comment role applies.

An off-walk list section is the natural home for a shared item list. list_from_section("## List") returns a list section's pre-parsed items (bullets or numbered lines) as a Lua array of strings without running the section. It resolves a heading string against the caller's sibling sections plus its own direct children; anything else - the parent, nieces and nephews, grandchildren, the caller itself - is not found, and the error lists only the visible sections. Naming a section with no pre-parsed items is an error. The array feeds fanout directly - fanout("## Worker", list_from_section("## List")) - because fanout's second parameter is always a collection, never a section name.

Authoring note: the marker is recognized only as a genuine Markdown thematic break. After a prose line it needs a blank line before it - a text line immediately followed by --- is a setext heading underline, not a marker. After a heading or a fence it stands alone.

Optional Frontmatter Fields

• max_tool_iterations - integer between 1 and 1000 (default: 24)

H1-Only Prompts

A prompt with no H2 sections is valid. The H1 blocks execute, the model reply becomes the run result, and no section walk occurs:

---
name: summarize
description: Summarize the input
promptforge: 1
---

# Summarize

```lua
models.default("m", "A model suited for careful analysis")
```

Summarize this text in one paragraph.

{{ args }}

Shared Libraries

A lua shared fence in the H1 region defines a library compiled once at parse time and replayed into every section VM as its first chunk - before any of the section's own Lua blocks run. The replay runs with the full section environment installed (args, sys, var, reply, store, log, the tools/models tables, and the control globals), so top-level shared code may use them at load. Two exclusions apply: the captured tool/model alias globals install only after the replay (a declared alias wins over a same-named shared global), and jump during the load is a hard error. A scalar top-level return is discarded - the replay loads a library, it does not produce the section's result.

Input and Output Declarations

A prompt can declare a file it expects to find in the store and a file it will leave there:

---
name: gate_paper
description: Produce a gating report for a paper
promptforge: 1
input:
  path: paper.md
  description: The paper markdown to analyze
output:
  path: report.md
  description: The gating report produced by analysis
---

Both input and output are optional. A prompt may declare one, both, or neither.

• path is the store-internal filename the prompt uses in Lua (store.read('paper.md'))
• description documents the file's purpose and appears in MCP tool listings via list_prompts

The declarations are metadata only. The runtime does not enforce that the prompt actually reads the input or writes the output - they tell callers what the prompt expects and produces.

Execution

A prompt file is a Markdown document with YAML frontmatter that promptforge compiles into an executable pipeline. The frontmatter declares identity (name, description) and a version tag (promptforge: 1). Below the frontmatter, an H1 title heads the prompt and zero or more H2 sections supply the instructions, Lua logic, and prose that the runtime walks at execution time.

The Run Function

Execution is a free function call over caller-owned resources. There is no process-global state. The caller owns the prompt, the execution id, the tool picker, the tool catalog, the model catalog, the store, and the observer.

#![allow(unused)]
fn main() {
use promptforge_core::{run, Prompt, RunConfig, StoreRef, ResolutionContext};
use promptforge_core::tools::ToolCatalog;

let prompt = Prompt::parse(source, "my-execution", &observer)?;
let tool_catalog = ToolCatalog::new(&tools)?;

let result = run(
    &prompt,
    "user input here",
    ResolutionContext::new(&picker, &models, &tool_catalog),
    &StoreRef::memory(),
    RunConfig::new("my-execution"),
).await?;
}

The run resolves the H1 block once, then walks H2 sections top to bottom. A section falls through to the next when its Lua does not return a value. An explicit return stops fall-through. When execution falls off the last section, the result is the last model reply, then the generic string "done".

H1-Only Execution

A prompt with no H2 sections executes its H1 blocks (including any prose), and the model reply becomes the run result:

---
name: summarize
description: Summarize the input
promptforge: 1
---

# Summarize

```lua
models.default("m", "A model suited for careful analysis")
```

Summarize this text in one paragraph.

{{ args }}

Run Configuration

RunConfig uses a builder pattern:

#![allow(unused)]
fn main() {
RunConfig::new("execution-id")
    .observer(my_observer)
    .debug(my_debug_capture)
    .client(gateway_client)
    .cancel(cancel_handle)
    .limits(run_limits)
}

All builder methods are optional. Without .client(), the runtime lazily constructs one from environment variables.

Run Limits

Configurable limits cap resource consumption:

#![allow(unused)]
fn main() {
RunLimits::new()
    .max_tool_iterations(NonZeroU32::new(24).unwrap())        // model round-trips per section
    .max_fanout_concurrency(NonZeroUsize::new(8).unwrap())    // parallel arms
    .max_response_bytes(NonZeroU64::new(16 * 1024 * 1024).unwrap())
    .lua_memory_bytes(NonZeroUsize::new(64 * 1024 * 1024).unwrap())
    .lua_log_events(NonZeroU32::new(1024).unwrap())
    .request_timeout(Duration::from_secs(120))
}

Lua Scripting

A prompt is built from alternating Lua and prose blocks. Each section can contain any number of Lua blocks interleaved with prose segments. The last prose block in a section runs a full tool-call loop; earlier prose blocks run single-shot (one model round, then control continues to the next Lua block).

Preamble, prologue, and epilog are positions, not phases: the preamble is the H1 region, the prologue is a section's Lua before its first prose, and the epilog is Lua after the last prose. The behavior is emergent - an epilog runs simply because it is the next block.

A tool-call loop may end silently: when the model finishes with finish_reason: "stop" and an empty reply after completing at least one tool call, the loop accepts that as a clean exit and the section's reply is "". This supports "record everything via tools, output nothing" prompts. Any other empty reply - no prior tool calls, or a missing or non-"stop" finish reason - fails the run with an empty-model-reply error.

The H1 Preamble

Lua blocks in the H1 region execute once in source order before any H2 section. The preamble declares tools and models, sets variables, and can short-circuit the entire run:

# My Prompt

```lua
models.default("writer", "a capable writing model")
tools.bind("search", "web search capability")
tools.always("search")
var.topic = "Rust async patterns"
```

## Write

Write an article about {{ var.topic }}.

Returning a scalar value (string, integer, number, or boolean) from H1 skips all H2 sections and becomes the run result.

Shared Libraries

A lua shared fence in the H1 defines a reusable library compiled once and replayed into every section VM as its first chunk, with the full section environment already installed:

```lua shared
function summarize(text)
    return "Summary: " .. text
end
```

The replay sees everything a later chunk sees - args, sys, var, reply, store, log, the tools/models tables, and the control globals - so top-level shared code may read args or write store files at load. Only the captured tool/model alias globals (the bare search, analyst handles) install after the replay, so a declared alias always wins over a same-named shared global. A scalar top-level return is discarded: the replay is a library load, not a result. jump is the one exclusion - calling it during the load fails the run with "jump is not available during shared library load".

Section Environment

Each section VM provides these globals:

The sys table includes when, now, id, section_name, execution, section_count, model (after first model interaction), and reply_finish_reason (after inference). It is sealed - writes raise errors and the metatable cannot be replaced. sys.id is the run-global execution-unit counter: H1 keeps id 0, and every section entry and every fanout arm takes the next value, so entering the same section twice yields two ids. A fanout arm also carries sys.index, its 1-based position within the current fanout (a nested fanout restarts at 1); sys.index is absent outside fanout, and reading it there raises.

var is the walk-local clipboard. Writes to it persist across sections on the same walk, H1 included, so H1 var writes are visible to every H2 section on the walk. execute and fanout start contained walks that clone the caller's var in and discard it out - child writes never reach the caller. var holds JSON data only: assigning a function, userdata, or a table containing one fails at the assigning line. Bare globals (x = 42 without local) are section-local scratch instead, and prose reads them as {{ x }}.

Template Substitution

Prose blocks support {{ path }} template substitutions. The sources are args, reply, var, sys, item (fanout arms only), and bare globals: an unknown first segment resolves as a section-local Lua global, with dotted paths indexing into its JSON form. Scalars render naturally, tables as JSON; a missing global, or one holding a function or userdata, is an error:

## Research

```lua
var.query = "latest Rust async runtimes"
```

Search for {{ var.query }} and summarize the results for {{ args }}.
The previous section said: {{ reply }}
Current item: {{ item }}
Run id: {{ sys.id }}

Escape literal delimiters with backslash: \{{ emits {{.

Control Flow

jump(target) transfers control to another section by heading name, clearing conversation context. The current reply value is preserved across the jump, so the target section can reference it in prose ({{ reply }}) or Lua. Clear it explicitly with reply = nil before jumping when the target should not inherit the previous reply. execute(target, input) starts a contained chain at the target with a fresh VM and conversation, returning the chain's final reply:

## Router

```lua
local result = execute("## Research", "find Rust crates for HTTP")
var.research = result
jump("## Synthesize")
```

## Research

Research the topic: {{ args }}

## Synthesize

Using this research: {{ var.research }}

Write a summary.

Both jump and execute address any section in the caller's visible set: its sibling sections at its own nesting level (for a top-level section, the other H2 sections) plus its direct children, disambiguated by heading level - ## Peer matches only a sibling, ### Child only a direct child. The parent, nieces and nephews, grandchildren, and the caller itself are not visible and resolve as not-found, with the error listing only the visible sections.

A jump to a child heading starts a child-level walk within the jumper's children: the walk begins at the target (which runs even when marked off-walk) and falls through to its following siblings under the same rules as the top-level walk. When the level exhausts, the parent walk resumes at the section after the jumper, and the sub-walk's last reply becomes the reply the next section sees. The rule recurses to deeper levels - a child can jump to its own children. A walk never descends on its own, so a section's children run only when addressed.

execute() runs a contained chain starting at its target: a walk with every normal rule - fall-through, off-walk skips, jumps, child chains - that never moves the outer walk. When the chain ends (its level exhausts or a return fires), the chain's final reply is the call's return value and the caller continues. A return ends only the chain it fires in; the top-level walk's return ends the run. Because a chain falls through like any walk, a multi-section subroutine is best expressed as a child walk (the children need no off-walk marker, since no walk descends on its own) or placed after the run-ending section.

Reply preservation across jump() enables routing patterns where one section's analysis determines the next section's context:

## Analyze

Analyze this input for severity. End with exactly CRITICAL or NORMAL.

{{ args }}

```lua
if reply:find("CRITICAL") then
    jump("## Alert")
else
    jump("## Summary")
end
```

## Alert

The analysis found a critical issue:

{{ reply }}

Escalate this with recommended actions.

execute() nests up to 8 levels deep, and the count accumulates across fanout boundaries - each arm runs one level deeper than the section or arm that spawned it. A chain starts with reply set to nil - pass context through the input parameter instead. A jump() inside a chain moves within the chain, and a return inside a chain ends the chain, not the run. Sections are referenced by heading string.

fanout(worker, collection) maps the worker over any Lua table, resolved against the same visible set. The collection is always a table, never a section name - a non-table second parameter is an error that points at list_from_section. The array part (1..#t) iterates in order first, then the hash part in undefined order. An array member arrives as the arm's item unchanged - a string stays a string, a number a number, a table a table - while a hash member arrives as a pair table with item.key and item.value. Keys must be strings, numbers, or booleans; a function or userdata member is an error naming its index. Each arm result's .item carries the member value back, so the caller can correlate results with rich items. An empty collection is an error - no work is likely a bug. To fanout over a list section's pre-parsed items, pass list_from_section("### List") as the collection.

Lua API Summary

Both infer forms share one shape: a single tool-free round on a fresh conversation that never sets reply and never touches sys. models.infer(prompt) uses the section's current model; models.get(alias):infer(prompt) uses any declared model. A Lua block that needs tools uses execute on a section.

Local Tools

tools.add_local(alias, description, params, handler) declares a tool backed by a Lua function. When the model calls it during the tool loop, the handler runs synchronously in the section's VM instead of reaching an external service:

tools.add_local("extract_section", "Extract a range of lines from the paper", {
    name = {"string", "Section heading text"},
    start_line = {"integer", "1-based line number where the section begins"},
    end_line = {"integer", "1-based line number where the section ends"},
}, function(args)
    local lines = store.read_numbered("paper.md")
    return "extracted " .. args.name
end)

The alias must be unique within the section. It cannot reuse an alias declared by tools.bind or tools.always, and a second tools.add_local call with the same alias is an error.

The params table maps each parameter name to either a bare type string or a {type, description} array. Supported types are "string", "integer", "number", and "boolean". All declared parameters are required. The engine converts the table into the JSON Schema the model sees.

The handler receives the arguments as a Lua table with the named fields and returns a string; Lua errors surface as tool-call failures. The handler shares the section's VM, so it can use store, var, and section globals, and it may call execute(), fanout, and the infer forms (models.infer(prompt), handle:infer(prompt)). It cannot call jump() - jump is disabled for the duration of the call. Local tool output is trusted (no nonce envelope), since the prompt author wrote the handler. A local tool becomes visible to the model starting from the next prose block.

Sandbox Constraints

The Lua sandbox provides only string, table, and math standard libraries. Dangerous globals (load, dofile, require, print, rawget, rawset, collectgarbage) are removed. A runaway Lua block is automatically aborted after exceeding the instruction budget (approximately 10 million instructions). Per-VM memory ceiling defaults to 64 MiB. The log() function accepts messages limited to 256 Unicode scalars with no newlines or control characters.

Tool and model aliases must match [A-Za-z][A-Za-z0-9_-]{0,63}.

Models

Models are declared by capability description and resolved semantically against a model catalog at runtime.

Declaring and Binding

-- Declare a model by what you need it to do
models.bind("writer", "a creative writing model", {
    thinking = true,
    temperature = 0.7,
    context = 128000,
    max_tokens = 4096
})

-- Set it as the prompt-wide baseline
models.default("writer")

The models.default(alias, description, opts) form declares and designates in one atomic call; the single-alias form designates a model already declared with models.bind. Within sections, models.use(alias) selects a specific model and returns its handle:

local analyst = models.use("analyst")

Sections without models.use inherit the models.default baseline. A prompt can carry both - the baseline applies everywhere a section does not override it. Sections with non-empty prose but no model binding receive a clear error.

Hard Constraints

The opts table filters the catalog before semantic resolution:

• thinking - boolean, required or forbidden
• context - minimum context window (positive integer)
• temperature - float in range 0.0 to 2.0
• max_tokens - positive integer

Duplicate model aliases or duplicate models.default calls are rejected atomically. models.use may be called at most once per section.

Model Inference from Lua

infer has one shape: a single tool-free inference round on a fresh conversation. It never sets reply and never touches sys. Two forms exist:

-- The section's current model (the models.use selection, else the models.default baseline)
local tag = models.infer("One-word sentiment of: " .. args)

-- Any declared model, via its handle
local critic = models.get("critic")
local review = critic:infer("Critique this draft: " .. reply)

models.get(alias) returns the handle for a declared model without changing the section's model selection, so handle:infer is the way to consult a different model inside a section. A Lua block that needs tools uses execute on a section instead.

Inspecting Model Properties

After binding, a model handle's frozen properties are accessible from Lua: name, model_id, description, context, thinking, temperature, and max_tokens.

Model Catalog

The library fetches a live model catalog from a gateway's GET /v1/models endpoint with bearer authentication. The caller provides a model catalog built from descriptors with identity, description, context window, and thinking mode (Always, Switchable, or Never).

Tools

Tools are declared by capability description and resolved semantically at runtime via a picker. You control exactly what the model sees - declaring a tool does not automatically expose it.

Declaring Tools

-- Declare a tool binding
local search = tools.bind("search", "web search capability")

-- Promote to prompt-wide availability (available in all sections)
tools.always("search")

A tool declared with tools.bind is not exposed to the model unless tools.always or tools.add is called.

-- Section-local scoping
tools.add("search")            -- by alias string
tools.add(search)              -- by handle object
tools.add({"a", "b", tool_c}) -- arrays of strings or handles

tools.add calls are atomic: a failure rolls back all entries. An empty add is a no-op.

Tool Properties

After tools.bind, the returned handle exposes: name, description, parameters (JSON schema), wire_name, and untrusted flag. Tool objects are frozen - assigning a field errors. The model-facing description is overridden positionally at declaration or scoping time:

tools.bind("search", "web search capability", "Search the web for current information")
tools.always("search", "Search the web for current information")
tools.add("search", "Search the web for current information")

Precedence is add over bind/always over the catalog description.

Tool Dispatch Loop

The tool loop runs the model in a cycle: dispatch tool calls, feed results back, re-prompt until the model produces a final text reply or the iteration cap is reached (default 24 rounds, configurable via max_tool_iterations in frontmatter).

Tool Safety

Untrusted tool output is wrapped with a CSPRNG nonce envelope before reaching the model, preventing prompt injection. One nonce per run; envelopes are deterministic within a run. Trusted tool output passes verbatim. Trust marking is mandatory at construction time.

Near-duplicate tools available to the same section are detected and rejected before any model call, with similarity diagnostics. Tool calls for tools not available to the section produce a clear error distinguishing globally-declared-but-unavailable tools from truly unknown ones.

Tool Call Counts

Per-alias call counts are tracked during execution. Read them from Lua to measure or assert model behavior:

tools.add("search")

After the prose block runs with the tool loop:

if tools.calls.search == 0 then
    log("model never searched")
end

Counts increment even when a tool call fails. Mistyped aliases produce a hard error with the available tools listed.

Local Tools

tools.add_local(alias, description, params, handler) declares a tool backed by a Lua function, available from any H2 Lua block. When the model calls the tool, the handler runs synchronously in the declaring section's VM rather than reaching an external service:

tools.add_local("grab", "Grab a value from the store", {
    key = {"string", "Store path to read"},
}, function(args)
    return store.read(args.key)
end)

The alias must be unique within the section. It cannot reuse an alias declared by tools.bind or tools.always, and a second tools.add_local call with the same alias is an error.

The four positional arguments:

• alias - tool name, same rules as tools.bind ([A-Za-z][A-Za-z0-9_-]{0,63})
• description - one-sentence description the model sees
• params - flat table of parameter declarations (see below)
• handler - function receiving an args table, returning a string

Each params value is either a bare type string or a {type, description} array:

-- Bare types
{ name = "string", count = "integer" }

-- Type plus per-parameter description (helps small models)
{ name = {"string", "Section heading text"},
  start_line = {"integer", "1-based first line"} }

-- Mixed
{ name = {"string", "Section heading text"}, count = "integer" }

Supported types are "string", "integer", "number", and "boolean". All declared parameters are required; there are no optional parameters. The engine converts the table into a JSON Schema parameters object for the model.

Handler rules:

• Receives the arguments as a Lua table with the named fields; returns a string
• Runs in the section's VM with access to store, var, and section globals (accumulator patterns work)
• May call execute(), fanout, and the infer forms (models.infer(prompt), handle:infer(prompt))
• Cannot call jump() - it is disabled for the duration of the call
• Lua errors propagate as tool-call failures
• Output is trusted: no nonce envelope, since the prompt author wrote the handler

A local tool becomes visible to the model starting from the next prose block. Local tools are H2-only; declaring one in H1 is not supported.

Implementing Custom Tools

A custom tool requires:

• A stable ToolId (server + name pair)
• A wire name matching [A-Za-z0-9_.-]
• A description string
• A JSON-Schema parameters definition
• An async call method returning ToolOutput (marked trusted or untrusted)

Tools can run locally in-process or proxy through a remote gateway, both dispatched uniformly through the Tool trait:

#![allow(unused)]
fn main() {
use promptforge_core::{Tool, ToolId, ToolOutput};

#[async_trait]
impl Tool for MyTool {
    fn id(&self) -> &ToolId;
    fn wire_name(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters(&self) -> &serde_json::Value;
    async fn call(&self, arguments: &str) -> Result<ToolOutput, ToolError>;
}
}

Built-in Web Search

The web search tool sends queries through a gateway proxy so the search provider credential never leaves the server. Results are automatically marked as untrusted output.

Parameters

• count - number of results (1-20)
• freshness - time filter: pd (past day), pw (past week), pm (past month), py (past year)
• safe_search - level: off, moderate, strict
• domains_include - allowlist (up to 20 domains)
• domains_exclude - blocklist (up to 20 domains)
• country - country code
• language - language code

Fanout

fanout(worker, collection) maps a worker section over a collection in parallel. Each member is processed by its own isolated execution arm with a fresh Lua VM.

## Process

```lua
local results = fanout("### Worker", list_from_section("### URLs"))
var.output = table.concat(results, "\n\n")
```

### Worker

Fetch and summarize: {{ item }}

### URLs

- https://example.com/page1
- https://example.com/page2
- https://example.com/page3

The worker is referenced by markdown heading address (level + name) and resolves against the caller's visible sections - its siblings plus its direct children. The second parameter is always a collection, never a section name: any Lua table works, and list_from_section("### List") feeds a list section's pre-parsed bullet or numbered items straight in.

The array part (1..#t) iterates in order first, then the hash part in undefined order. An array member arrives as the arm's item unchanged - a string stays a string, a number a number, a table a table. A hash member arrives as a pair table: item.key and item.value. Keys must be strings, numbers, or booleans; a function or userdata member is an error naming its index. An empty collection is an error - no work is likely a bug.

Arm Execution

Each arm receives the current member as the item variable, a sys.index giving its 1-based position within the current fanout (a nested fanout restarts at 1, and sys.index is absent outside fanout), and a unique run-global sys.id. Each arm also starts with a fresh clone of the caller's var - arm writes to var never reach the caller. An arm's reply starts at the fanout caller's incoming reply as captured when the fanout call was set up (the caller's section-start reply), not the caller's latest reply at the call site: a caller whose own prose produced a reply before calling fanout hands every arm that earlier reply. The arm can:

• Run a Lua prologue that short-circuits (enabling pure-Lua map operations)
• Substitute {{ item }} in prose (strings verbatim, numbers and booleans in their natural string form, tables as compact JSON)
• Run the full model tool loop
• Execute an epilog for post-processing
• Call execute, fanout, and list_from_section, resolved against the worker's visible sections (the set the worker was resolved from, minus the worker, plus its children), and transfer control with jump - the arm's remaining blocks are skipped and the arm's text becomes the jumped-to walk's reply. Recursion depth accumulates across the fanout boundary: each arm runs one execute level deeper than its caller, so the 8-level cap bounds mixed execute/fanout nesting uniformly

Results are returned in collection order (array part first, then the hash part), not finish order. Each result has .text, .ok, .item, and .exhausted fields; .item carries the member value back - a pair table for hash members - so the caller can correlate results with rich items. An arm that falls through without producing a reply yields .text == "" with .ok == true. The result array supports table.concat since objects coerce via __tostring.

All arms share the run's store. Two arms of one fanout calling store.write on the same path is a hard error (a write-write race) that aborts the sibling arms; store.append from concurrent arms stays legal, with unspecified order.

Resilience

An exhausted arm (tool loop budget exceeded) soft-degrades into an incomplete stub rather than failing the entire fanout. A fatal error in any arm aborts all sibling arms, preventing wasted work. Cancellation propagates from the parent into each spawned arm cooperatively.

Default concurrency is 8 parallel arms, configurable via RunLimits.

Store

The store is a run-scoped virtual filesystem shared across all sections. Data persists within a single run and the handle is thread-safe across concurrent tasks.

store.write("notes/summary.md", "# Summary\n" .. reply)
store.append("log.txt", "processed: " .. args .. "\n")

local content = store.read("notes/summary.md")
local slice = store.read("notes/summary.md", 20, 40) -- lines 20-40 only
local numbered = store.read_numbered("notes/summary.md") -- whole file, numbered from 1
local numbered_slice = store.read_numbered("notes/summary.md", 20, 40) -- "20| ..." to "40| ..."

store.str_replace("notes/summary.md", "old text", "new text")

local files = store.glob("notes/*.md")
local exists = store.exists("notes/summary.md")

store.delete("notes/summary.md")

store.delete on a missing path is silent - delete is idempotent. Within a single fanout, two arms calling store.write on the same path is a hard error (a write-write race); store.append from concurrent arms stays legal.

Bounded Reads

store.read takes optional 1-based inclusive line bounds: store.read("log.txt", 20, 40) returns lines 20 through 40 joined by newlines, with no trailing newline. store.read("log.txt", 20) reads from line 20 to the end of the file. An end past the last line clamps to it, and a start past the last line returns an empty string. A start below 1 or an end before start raises an error.

store.read_numbered takes the same optional bounds and returns the lines with absolute line numbers: each line is prefixed with its number, right-aligned to the width of the largest number emitted, followed by "| ". With no bounds the whole file is numbered from 1. A bounded slice keeps its absolute numbers, so store.read_numbered("log.txt", 20, 40) yields 20| ... through 40| ... - cite a line by its number and the reader can verify it against the whole-file numbering.

Safe Injection

Wrap stored content in the untrusted-input guard envelope with the untrusted(s) global before re-injecting it into model prompts: untrusted(store.read(path)). Forged close-tags in stored content are escaped, so injected data cannot break out of the envelope:

store.write("user-data.txt", user_provided_content)
-- Later, safely inject into a prompt context:
local safe = untrusted(store.read("user-data.txt"))

Path Validation

All store paths are validated:

• Forward-slash only (backslash rejected)
• No path traversal (. and .. segments rejected)
• No Windows reserved device names (CON, NUL, COM1-9, LPT1-9)
• No trailing dots or spaces
• Maximum 1024 bytes

Glob Matching

• * matches within a single path segment
• ** matches across path separators
• Unsupported syntax (backslash escapes, triple-star, misplaced **) is rejected
• Matching uses a bounded, non-backtracking algorithm

The str_replace operation requires the old text to be unique in the file; ambiguous matches are refused with a count of occurrences.

The default in-memory backend (StoreRef::memory()) requires no filesystem or network and drops cleanly with the run. Custom backends implement the Store trait.

Pre-populated Stores

Callers can seed the store with files before a prompt runs. The MCP server does this when input_file or input_text is provided - it writes the content into the store at the path declared by the prompt's input: frontmatter before execution begins.

From the prompt's perspective nothing is different. It calls store.read('paper.md') and gets content. It does not know whether that content was placed there by an earlier section, by a caller, or by test scaffolding. This keeps prompts decoupled from their invocation context.

Gateway

promptforge-gateway is the one process in PromptForge that talks to LLM backends. Point it at a TOML file, and it serves an OpenAI-compatible HTTP API that routes chat completions to configured backends, holds every credential, manages a model catalog, runs a built-in web search tool, and optionally spawns local llama-server processes for GGUF models. Nothing above it holds a vendor key. Nothing above it knows which machine answers. A key rotation touches one file on one host.

After reading this chapter you will be able to configure, start, and operate the gateway for remote endpoints, local models, multiple profiles, and built-in tools.

What the Gateway Does

The gateway accepts POST /v1/chat/completions requests in the OpenAI chat completions format. It resolves the model name the caller asked for, substitutes the backend's own model string into the outgoing request, forwards it, and restores the caller's model name on the response. Everything else in the request body - sampling parameters, tool definitions, template arguments - passes through untouched in a flattened map, so a parameter the gateway has never heard of reaches the backend without a gateway release. Models configured with kind = "embedding" are served instead at POST /v1/embeddings in the OpenAI embeddings format, with the same routing and passthrough discipline. Models configured with kind = "classifier" serve rerank requests at POST /v1/rerank - a query and a document set in, ranked relevance scores out - under the same discipline.

Credentials live here and nowhere else. Each [[endpoint]] carries an api_key, each [tools.web_search] carries a search provider key, and each [[local_model]] is reached over a loopback connection with a generated bearer. The Secret type ensures no credential can be serialized, logged, or printed: it redacts in both Debug and Display, and expose() is the single plaintext accessor.

Model resolution is one exact string lookup. A miss is a 404. There is no prefix matching, no regex, no alias chain, and no default model. A typo is a clear error rather than a silent charge against the wrong backend.

Configuration

The gateway boots from two TOML files: the boot file named on the command line, which is the catalog and infrastructure, and a named profile from the boot file's sibling profiles/ directory, which is the initial loaded set. Every configuration struct uses deny_unknown_fields, so a misspelled key is a boot failure rather than a setting silently ignored.

A minimal configuration defines a server (bind address and bearer key), one endpoint, and one model:

[server]
bind = "127.0.0.1:8080"
api_key = "${PROMPTFORGE_GATEWAY_API_KEY}"

[[endpoint]]
id = "anthropic"
protocol = "openai"
base_url = "https://api.anthropic.com/v1"
api_key = "${ANTHROPIC_API_KEY}"

[[model]]
name = "reasoning-large"
description = "Anthropic's best reasoning model"
context = 200000
upstream = "claude-sonnet-4-6"
endpoints = ["anthropic"]

The name is what callers request. The upstream is what the backend knows the model by. Name your models by capability (reasoning-large, fast-draft) when you want the same prompt to work across environments where the backend changes.

Environment Variable Interpolation

Any string value can use ${VAR} to reference an environment variable. Interpolation runs after the TOML is parsed, so it applies only to string values. An unresolved variable fails the load, so a deployment that forgot to export a credential never starts serving with a blank one. Use $$ for a literal dollar sign.

There is no implicit pickup of ANTHROPIC_API_KEY or OPENAI_API_KEY from the ambient environment. Every credential must appear in the configuration as an explicit ${VAR} reference.

At boot the process environment is populated from at most two env files: the profile's own file first (profiles/main.env for --profile main), then the boot file's sibling env file (gateway.env beside gateway.toml). Neither file overrides a variable that is already set, so precedence is the process environment, then the profile's file, then the boot file's. Included files' env files are never loaded. On a profile switch only the new profile's env file is loaded; the boot file's is already in the process.

Model Fields

embedding and classifier models reject chat-only fields at load: thinking, default_max_tokens, tool_dialect, effort_levels, default_effort, and adaptive_thinking are chat-only, while context applies to every kind. The catalog entry carries the kind so clients can filter before building a request. The effort knobs (effort_levels, default_effort) are also rejected when thinking = "never", and max_output must not exceed context.

Endpoint Fields

Starting the Gateway

promptforge-gateway serve gateway.toml --profile main

Boot requires two things: a config path and a profile name. The config path comes from the positional argument or the PROMPTFORGE_GATEWAY_CONFIG environment variable; the CLI argument wins, and with neither set the boot fails with a usage error naming both sources. The profile name comes from --profile only (no env var). It is required - there is no anonymous boot. Every gateway has at least one profile; the initial loaded set always has a name.

The profiles directory is always the profiles/ directory beside the config file - never independently configurable, and there is no ~/.promptforge/profiles default. Booting with an unknown profile name fails with a startup error listing the available profiles; a missing profiles/ directory or a missing profile file is likewise a startup error.

The boot file is the catalog and infrastructure; it is not loaded as the runtime config directly. The named profile is loaded with include resolution and becomes the initial config. A profile may declare a top-level models = ["name", ...] allowlist selecting a subset of the catalog's [[model]] and [[local_model]] entries; the loaded set is exactly the selection, so GET /v1/models shows nothing else. An allowlist entry naming a model the catalog does not define is a validation error at load. With no models key, the profile loads the full catalog. The single-file setup needs one minimal profile, profiles/main.toml beside gateway.toml:

include = ["../gateway.toml"]

Startup order: load the two env files, resolve the profile's include chain, start local model runtime (when [[local_model]] is present), build the routing table, bind, serve. A broken config never reaches a listening socket.

Model Catalog and Routing

Clients discover available models by calling GET /v1/models with a bearer token:

{
  "object": "list",
  "data": [
    {
      "id": "reasoning-large",
      "object": "model",
      "kind": "chat",
      "description": "Anthropic's best reasoning model",
      "context": 200000,
      "thinking": "switchable",
      "max_output": 64000,
      "images": true,
      "parallel_tool_calls": true,
      "effort_levels": ["low", "high"],
      "default_effort": "low",
      "adaptive_thinking": false
    }
  ]
}

Each entry includes the model's kind, context window, thinking mode, and capability metadata so clients can make binding decisions before sending a request. The flags (images, parallel_tool_calls, adaptive_thinking) and effort_levels are always present; the optional knobs (max_output, default_temperature, default_effort) appear only when configured.

A chat completion request names a model and provides messages:

{
  "model": "reasoning-large",
  "messages": [{"role": "user", "content": "Explain monads"}],
  "temperature": 0.7
}

Send this as POST /v1/chat/completions with Authorization: Bearer <token>.

The gateway validates: model must be non-empty, messages must be non-empty, each message must be a JSON object with a supported role (system, user, assistant, tool, function, developer) and either content or a tool/function call. Everything else passes through verbatim.

The response carries the caller's model name, not the backend's.

Emulated Tool Calling

A chat model whose backend has no native tool array can still serve tool calls by setting tool_dialect = "gemma3_tool_code" on its [[model]] entry:

[[model]]
name = "gemma-remote"
description = "A Gemma 3 instruct model behind an OpenAI-shaped server"
context = 8192
tool_dialect = "gemma3_tool_code"
upstream = "gemma3-it"
endpoints = ["local-llama"]

For a non-streaming chat completion to such a model the gateway emulates the protocol. Outbound, the OpenAI tools array is translated into a plain-language system guide teaching the tool_code fence protocol (one name(key=value) call per line) and prepended as a system message; tools and tool_choice are stripped from the forwarded request. Inbound, the reply's content is scanned for a leading tool_code fence (or an interim json fence carrying a tool_calls blob); a well-formed fence becomes OpenAI tool_calls objects with a null content and a tool_calls finish reason.

Recovery is warn-and-continue, never silent and never fatal: a recognized-but-malformed fence (unterminated, empty, an invalid call line, or trailing prose after the fence) yields an empty content plus a gateway_warning field on the message carrying the reason, and the recovery is logged at warn. gateway_warning is a gateway-specific extension on the OpenAI response shape; downstream clients should ignore unknown fields. The dialect uses content-fence parsing only - the gateway never sets response_format: json_object.

Emulation applies to non-streaming completions only; a stream: true request is forwarded unchanged.

Streaming

Pass "stream": true on a chat completion to get a server-sent events stream instead of a single JSON response:

{
  "model": "reasoning-large",
  "messages": [{"role": "user", "content": "Explain monads"}],
  "stream": true
}

The response is Content-Type: text/event-stream with one data: line per chunk, terminated by data: [DONE]. The relay is typed: the gateway parses and validates each upstream chunk, rewrites the model name back to the caller's, and re-serializes it - it never splices raw upstream bytes through. A failure before the stream starts (an upstream 4xx/5xx, an unreachable backend) comes back as the usual JSON error envelope, never as an SSE stream that dies mid-flight; a failure mid-stream is emitted as an error-envelope data: event before the stream ends. The dominion queue slot is held for the stream's whole lifetime, so a long stream counts against concurrency until it ends. Streaming requests use a connect-timeout-only HTTP client: the usual whole-request deadline is not applied, since it would kill any stream that outlived it.

Two caveats. A client that disconnects mid-stream cancels the upstream stream: dropping the response drops the upstream connection and frees the queue slot in the same unwind, so a hung client never holds a backend stream open. And a malformed upstream chunk - one that is not JSON, or that lacks the minimal choice shape (at least one choice with an index and a delta) - is logged and skipped rather than relayed or fatal, so one bad chunk never ends an otherwise healthy stream. The terminal data: [DONE] is a sentinel, not JSON; it is recognized before parsing and never logged as malformed.

Embeddings

Models configured with kind = "embedding" serve OpenAI-shaped embedding requests at POST /v1/embeddings:

{
  "model": "embed-large",
  "input": ["first document", "second document"],
  "encoding_format": "float"
}

The input is a single string or an array of strings; encoding_format (float or base64) is optional, and every other OpenAI embeddings field passes through verbatim. The route applies the same bearer auth, model resolution and rewrite, and dominion queue admission as chat completions, and the response restores the caller's model name with data and usage passed through from the backend. Naming a chat or classifier model here is a 400 kind_mismatch. A model whose upstream cannot serve embeddings at all - a local chat server, for example - fails with 400 and code: "model_unavailable".

Rerank

Models configured with kind = "classifier" serve rerank requests at POST /v1/rerank:

{
  "model": "rerank-large",
  "query": "what is rust",
  "documents": ["a card game", "a systems language"],
  "top_n": 2
}

The request names a query and the documents to rank against it; top_n is optional, and every other field passes through verbatim. The route applies the same bearer auth, model resolution and rewrite, and dominion queue admission as chat completions, and the response restores the caller's model name with results and usage passed through from the backend. Naming a chat or embedding model here is a 400 kind_mismatch. A model whose upstream cannot serve rerank at all - a local chat server, for example - fails with 400 and code: "model_unavailable".

Authentication and Errors

Every route except GET /health checks Authorization: Bearer <token> against server.api_key. The comparison is constant-time: both values are SHA-256 hashed to fixed-length digests, then compared with the subtle crate's ConstantTimeEq. A missing or wrong token returns 401 with no detail.

GET /health is unauthenticated and always returns {"status": "serving"} while the process is up.

All errors use the OpenAI error envelope:

{
  "error": {
    "message": "unknown model reasoning-large",
    "type": "invalid_request_error",
    "code": "model_not_found"
  }
}

An unmodified OpenAI SDK surfaces these as its own error types rather than unparseable blobs.

Concurrency and Queuing

Bind an endpoint to a dominion to cap how many requests are in flight at once. The limit lives on the dominion, so everything bound to it shares one pool of slots:

[[dominion]]
id = "anthropic-pool"
kind = "remote"
max_concurrency = 10
max_queue = 100         # waiting requests (not counting in-flight); default 100
policy = "queue"        # "queue" | "reject" (fail-fast); default "queue"
fair_scheduling = true  # round-robin by client key; default true

[[endpoint]]
id = "anthropic"
protocol = "openai"
base_url = "https://api.anthropic.com/v1"
api_key = "${ANTHROPIC_API_KEY}"
dominion = "anthropic-pool"

Requests beyond the limit wait in the dominion's bounded queue. What a full queue does comes from policy: with queue (the default), up to max_queue requests wait and further arrivals are rejected with 503 and code: "queue_full"; with reject, a request that finds no free concurrency slot is turned away immediately - fail-fast, mapped to 429 with code: "queue_rejected".

When fair_scheduling is true, callers identify themselves via the X-PromptForge-Client header. Each client gets turns in round-robin order, so one fast client cannot monopolize slots. Missing or invalid headers map to the "default" bucket. The scheduler tracks up to 32 distinct client labels; additional labels fold into "default". The header is self-asserted: a scheduling hint for trusted-host callers, not an authenticated identity.

An endpoint without a dominion is unlimited.

Dominions

A dominion is a named pool of compute - a remote provider pool or a local GPU - carrying one concurrency limit and one bounded waiting queue shared by everything bound to it: two endpoints bound to the same dominion compete for the same slots. A dominion binding is the only way to cap concurrency.

[[dominion]]
id = "runpod-pool"
kind = "remote"
max_concurrency = 4
max_queue = 50            # bounded wait, then rejection; default 100
policy = "queue"          # "queue" | "reject" (fail-fast); default "queue"
fair_scheduling = true    # per-client round-robin; default true

[[dominion]]
id = "gpu0"
kind = "local"
vram_gb = 24              # local kind only; co-residency budget

[[endpoint]]
id = "runpod-a"
protocol = "openai"
base_url = "https://..."
api_key = "${RUNPOD_KEY}"
dominion = "runpod-pool"  # optional; absent = unlimited pass-through

[[local_model]]
name = "qwen-local"
description = "..."
source = "..."
context = 65536
dominion = "gpu0"         # optional; must name a local dominion
parallel = 4              # child --parallel; the queue limit when no dominion is bound
vram_gb = 14              # footprint estimate for the co-residency check

Binding is by explicit id and is kind-checked: an endpoint's dominion must name a remote dominion, a local model's dominion must name a local one, and an unknown id is a boot failure. vram_gb on a remote dominion is rejected. Dominion ids must be unique and non-empty, and max_concurrency and max_queue must be at least 1 when set.

An endpoint bound to a remote dominion shares that dominion's queue with every other bound endpoint, and a local model bound to a local dominion shares that dominion's queue with every other bound local model.

When a local dominion sets vram_gb, every local model bound to it must set its own vram_gb footprint estimate, and the estimates must sum to no more than the budget: an over-booked or incomplete budget fails validation at boot and at profile switch, before any child process starts and surfaces as an OOM. A local dominion without vram_gb imposes no co-residency obligation.

Web Search Tool

Enable the built-in web search tool by adding a [tools.web_search] section:

[tools.web_search]
provider = "brave"
api_key = "${BRAVE_API_KEY}"

The gateway proxies search requests to the Brave Search API with its own credential. The executor never sees the search key.

Send a search request:

{
  "query": "Rust async runtime comparison",
  "count": 5
}

Send this as POST /v1/tools/web_search with Authorization: Bearer <token>.

The response contains trimmed results:

{
  "query": "Rust async runtime comparison",
  "results": [
    {
      "title": "Comparing Tokio, async-std, and smol",
      "url": "https://example.com/article",
      "description": "A detailed comparison of...",
      "age": "2 days ago",
      "site_name": "example.com"
    }
  ]
}

Provider extras like thumbnails and ranking metadata are dropped - every byte would land in a model's context window.

Request Fields

Domain filters must be bare hostnames (no scheme, path, or port). A hostname matches when it equals the domain or ends with .<domain>.

Configuration Defaults

Results are post-processed in fixed order: sanitize text, strip tracking parameters, set site_name, apply include/exclude domain filters, diversify by hostname (max 2 per host by default), then cap at count. Over-length URLs are dropped whole rather than truncated into broken links.

When [tools.web_search] is absent, the route returns 404 - an absent resource, not a broken capability.

Named Profiles

Organize configurations for different environments as TOML files in the profiles/ directory beside the boot file:

<config-parent>/
  gateway.toml
  profiles/
    main.toml
    analytical.toml
    dev.toml

Start with a named profile:

promptforge-gateway serve gateway.toml --profile analytical

Every gateway boots into a profile, so the initial loaded set always has a name. A profile typically contains include = ["../gateway.toml"] plus its own overrides, keeping the boot file as the shared catalog.

Selecting a Subset of the Catalog

A profile selects its loaded set with a top-level models allowlist:

# analytical.toml
include = ["../gateway.toml"]
models = ["reasoning-large", "qwen3.8-local"]

After the include chain merges, the catalog's [[model]] and [[local_model]] arrays filter to the listed names: the loaded set is the selection, and GET /v1/models shows exactly it. The filter runs before validation, so reference checks and the VRAM co-residency check apply to the loaded set only - a catalog whose local models over-book a GPU in total still boots a profile whose selection fits the dominion's vram_gb budget, and endpoints or dominions referenced only by filtered-out models may stay defined. An allowlist entry naming a model the catalog does not define fails the load. With no models key the profile loads the full catalog, and when several files in one include chain declare models, the later file's list replaces the earlier one. GET /admin/status reports the active selection as model_allowlist (null when the full catalog is loaded).

Profile Inheritance

A profile can include parent files:

# analytical.toml
include = ["base.toml"]

[[model]]
name = "analysis"
description = "Deep analysis model"
context = 200000
upstream = "claude-sonnet-4-6"
endpoints = ["anthropic"]

Includes resolve depth-first relative to the including file. Max nesting depth is 16. Cycles are detected and rejected.

Merge rules:

• Arrays ([[endpoint]], [[model]], [[local_model]], [[dominion]]): merged by append. An entry with the same id or name replaces the earlier definition.
• Scalars (server.*, [local].cache_dir): later wins.
• The models allowlist: later wins - one list replaces the other, never unioned.

The Boot File Owns [server]

After include resolution, the profile's merged [server] section must equal the boot file's [server] exactly - bind address and api_key, compared as values after ${VAR} interpolation. A mismatch fails the boot (or the profile switch): a bind mismatch names both addresses, while an api_key mismatch names only the profile and the field, with both keys redacted. The conventional setup passes by construction because profiles include the boot file. The consequence: the socket and the gateway bearer key are fixed for the process lifetime, and a profile switch never rotates the admin credential.

Includes remain free-form: a profile may include a different file than the boot path, or be self-contained - a self-contained profile must replicate the boot file's [server] verbatim to boot. At startup the gateway logs the resolved include chain, plus a warning when the boot file is not in it: the likely-mistake case, where edits to the boot file have no effect.

Admin Routes

All admin routes use the same bearer token as /v1:

Switch with:

{"name": "analytical"}

Send this as POST /admin/switch-profile with Authorization: Bearer <token>.

Profile switches are serialized by a mutex. The old local children are stopped (freeing VRAM) before new ones start. The new configuration is built and validated before touching live state. On success, the routing, web-search settings, and local runtime are atomically swapped. On failure, the previous state stays intact with a stable admin credential.

The [server] section does not change on switch: the boot file owns it, and a profile whose merged [server] differs from the boot file's is rejected. Moving the socket or rotating the gateway key requires a restart.

Profile names must be a single path component - no separators, no . or .., no empty string. This confinement prevents directory traversal through the admin API.

Local Inference

Run local generative models by declaring [[local_model]] entries. The gateway provisions a pinned llama-server binary (GPU builds: Vulkan on Windows/Linux, Metal on macOS), downloads each GGUF, and spawns one child process per model.

[local]
# cache_dir = "~/.promptforge"  # default

[[local_model]]
name = "qwen-local"
description = "A careful analysis model suited to structured reasoning"
source = "https://huggingface.co/Qwen/Qwen3.5-9B-GGUF/resolve/main/qwen3.5-9b-q4_k_m.gguf"
sha256 = "abcdef..."
context = 65536
gpu_layers = 99
flash_attention = true

Each local model becomes a normal catalog entry. Clients reach it through the same POST /v1/chat/completions as remote models (POST /v1/embeddings for kind = "embedding", POST /v1/rerank for kind = "classifier") - the fact that it runs locally is invisible to callers.

Configuration Fields

A local model with kind = "embedding" or kind = "classifier" rejects the chat-only fields thinking, chat_template_file, effort_levels, default_effort, and adaptive_thinking at load; context and the launch knobs (gpu_layers, flash_attention, cache types, parallel, vram_gb) apply to every kind. The effort knobs are also rejected when thinking = "never", and max_output must not exceed context.

Local Embeddings

A local model with kind = "embedding" launches its child as llama-server --embeddings and serves POST /v1/embeddings exactly like a remote embedding model. Artifact download, digest pinning, dominion binding, and child supervision (respawn of a dead child) are unchanged from a chat child.

[[local_model]]
name = "bge-local"
kind = "embedding"
description = "A compact English embedding model for retrieval and similarity"
source = "https://huggingface.co/CompendiumLabs/bge-small-en-v1.5-gguf/resolve/main/bge-small-en-v1.5-q8_0.gguf"
sha256 = "ec38e8da142596baa913124ae50550de284b6916bf59577ef2f0cb9660c2f514"
context = 512

Local Classifiers

A local model with kind = "classifier" launches its child as llama-server --reranking and serves POST /v1/rerank exactly like a remote classifier model. Artifact download, digest pinning, dominion binding, and child supervision (respawn of a dead child) are unchanged from a chat child.

[[local_model]]
name = "jina-local"
kind = "classifier"
description = "A tiny English reranker for scoring query-document relevance"
source = "https://huggingface.co/gpustack/jina-reranker-v1-tiny-en-GGUF/resolve/main/jina-reranker-v1-tiny-en-Q8_0.gguf"
sha256 = "0defc1f8a1f4dd22183124a2a25a97765603e5a9e42258046c9b2c8a26d1f553"
context = 512

Cache and Provisioning

The cache directory defaults to ~/.promptforge (set [local].cache_dir to override). Models land in <cache>/models/, the llama.cpp binary in <cache>/llama.cpp/.

First-time downloads show an indicatif progress bar on interactive TTY stderr - percent, bytes, rate, and ETA. On non-TTY stderr, periodic tracing progress lines are emitted instead.

When sha256 is set, the downloaded file is verified against the digest.

Tool-Calling Dialect Detection

After a local child reports ready, the gateway queries its /props endpoint and resolves a tool-calling dialect from that evidence, hard-failing on ambiguous or absent evidence so a local model never silently defaults to an incorrect dialect. A sidecar .md file beside the GGUF (with frontmatter and a Jinja chat template) provides fallback evidence when /props omits chat_template; live props always win. The probe runs for kind = "chat" children only: a non-chat child has no chat template to evidence a dialect, so it carries the default openai dialect, same as a remote model. The resolved dialect is gateway-internal and selects how tool calls are emulated; it is not advertised in the catalog.

Child Supervision

If a transport failure occurs against a dead llama-server child, the gateway respawns it once on the same port and alias, then retries the request. There is no background watchdog. GET /health remains process-level liveness only.

Local Concurrency

parallel on a [[local_model]] (default 1) is both the child's --parallel argument and, when the model has no dominion, its gateway queue limit:

[[local_model]]
name = "qwen-local"
description = "..."
source = "..."
context = 65536
parallel = 4

Bind the model to a local dominion to share one concurrency limit and one VRAM budget with every other bound model; admission is then governed by the dominion's shared queue instead of a per-model limit.

Dropping the LocalRuntime (on process exit or profile switch) kills all llama-server children.

MCP Server

promptforge-mcp-server runs PromptForge prompts for agentic harnesses like Cursor and Claude Code. It puts a prompt catalog behind four fixed MCP tools rather than publishing each prompt as its own tool, which means tools/list never changes and a prompt saved ten seconds ago is callable with no reconnect. You point it at a prompts.toml, it resolves your prompts, connects to a gateway, and serves - over HTTP with bearer auth, or over stdio for a local spawn.

Starting the Server

Bind the streamable-HTTP transport:

promptforge-mcp-server serve prompts.toml

This serves at http://127.0.0.1:9310/mcp. Every request to /mcp must carry an Authorization: Bearer <token> header matching [server].api_key.

For a harness that spawns the server as a child process:

promptforge-mcp-server serve --stdio prompts.toml

Stdio speaks JSON-RPC over standard input and output, binds no port, and ignores [server].api_key entirely. Logs go to stderr so they do not corrupt the wire.

Configuration

A single prompts.toml carries everything the server needs. The MCP server uses a flat configuration - one .toml file with one name-matched .env file. There is no include or config chain.

Minimal Configuration

[server]
api_key = "shared-bearer"

[gateway]
url = "http://127.0.0.1:8081/v1"
api_key = "gateway-bearer"

Every string value supports ${VAR} interpolation. A variable resolves from the process environment first, then from the name-matched .env file beside the TOML (for example prompts.env beside prompts.toml), so the file supplies defaults and the environment overrides them. A missing .env file is skipped; a malformed one is ignored with a warning. Use $$ for a literal dollar. An unset variable fails the load everywhere except [server].api_key, where it drops the key silently so a stdio install can boot without a credential its transport never reads.

Full Configuration

[server]
bind = "127.0.0.1:9310"
api_key = "${PROMPTFORGE_MCP_SERVER_API_KEY}"
max_concurrent_runs = 4
admission_timeout = "30s"
reply_deadline = "240s"
retain_completed = "1h"
watch = true
watch_debounce = "500ms"
allowed_hosts = ["example.com", "example.com:8080"]

[paths]
prompts = "prompts"

[gateway]
url = "http://127.0.0.1:8081/v1"
api_key = "${PROMPTFORGE_GATEWAY_API_KEY}"

[catalog]
include = ["*.md", "governance/**/*.md"]
exclude = ["_*.md", "drafts/**"]

[prompts.scratch_test]
enabled = false

[prompts.staker]
file = "experiments/staker-v3.md"

Defaults

Durations use humantime format: "30s", "5m", "1h", "500ms".

Unknown keys are rejected outright - a misspelled key fails the load rather than being silently ignored.

Sections

[server] - Bind address, shared bearer token, concurrency limits, timing, and reload settings. allowed_hosts controls DNS-rebinding protection: on a loopback bind an empty list defaults to localhost, 127.0.0.1, ::1; a non-loopback bind with no hosts is refused.

[paths] - The prompts directory. Catalog patterns and [prompts.NAME].file paths are both relative to it.

[gateway] - The model gateway every run goes through. url must be a valid http/https URL with a host. api_key is the bearer credential sent on every model call.

[catalog] - Glob patterns that assemble the catalog. include names what to resolve; exclude subtracts from it. * does not cross a separator, ** does.

[prompts.NAME] - Per-prompt overrides keyed by the prompt's frontmatter name. Set enabled = false to drop one the globs caught. Set file = "path.md" to add a file no glob matches. The key must match the prompt-name shape: ^[a-z][a-z0-9_]{0,47}$.

The Tool Surface

The server publishes a fixed set of built-in tools. No prompt appears in tools/list - a prompt is reached only by naming it to run_prompt.

The picker feature is on by default. Without it the server publishes three tools and need_prompt is absent. A build without picker is smaller and removes the embedding model weights.

Running a Prompt

Call run_prompt with prompt (required) and args (optional):

{
  "prompt": "research_person",
  "args": "Herb Sutter, ABI stability positions"
}

File Parameters

Three additional parameters support file-based input and output:

• input_file - filesystem path; the server reads this file and places its content in the store at the prompt's declared input path
• input_text - literal text; the server places it in the store directly at the prompt's declared input path
• output_file - filesystem path; after the run completes, the server writes the output store file here

input_file and input_text are mutually exclusive. Providing both is an error.

If output_file is omitted, the output content is returned inline in the result's value field as usual.

The prompt itself never touches the real filesystem. It reads and writes through MemStore only - the server handles marshalling between the filesystem and the store boundary.

{
  "prompt": "gate_paper",
  "input_file": "/home/user/papers/p2996r7.md",
  "output_file": "/home/user/reports/p2996-gate.md"
}

list_prompts shows which prompts declare inputs and outputs, so callers know which file parameters apply.

What Happens

1. Name resolution - The name is matched case-normalized against the catalog. An unresolvable name returns all enabled names nearest-first so the model can correct itself.

2. Admission - The call waits for one of max_concurrent_runs slots. If none comes free within admission_timeout, the call gets a retryable refusal: "every run slot is busy and none came free within 30s. Retry in a moment."

3. Execution - The prompt runs against the gateway. Progress notifications stream to the client if it supplied a progressToken.

4. Reply deadline - If the run finishes in time, the result comes back inline. If it exceeds reply_deadline, the call returns immediately with status running and a run_id.

Background Runs

A run that outlives its call continues in background. Collect it with check_run:

{
  "run_id": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6"
}

A finished run stays collectable for retain_completed (default 1 hour), then is evicted.

If the client disconnects while a run is in progress, the run is cancelled cooperatively.

Result Format

Every result carries structured content:

{
  "run_id": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6",
  "prompt": "research_person",
  "status": "completed",
  "value": "The full artifact text...",
  "turns": 3,
  "elapsed_ms": 42000,
  "error": null
}

Status is one of running, completed, or failed. A completed run carries value; a failed run carries error; a running run carries neither.

Discovering Prompts

list_prompts

Browse the catalog with optional pagination:

{ "cursor": "100" }

Returns up to 100 entries per page:

{
  "prompts": [
    { "name": "research_person", "description": "Build a stakeholder profile...", "problem": null },
    { "name": "broken_one", "description": "", "problem": "parse error at line 3" }
  ],
  "next_cursor": "200"
}

A broken prompt appears in the listing with its problem visible, so the operator knows what to fix.

need_prompt

When you have a capability description rather than a name:

{ "capability": "Build a stakeholder position report for one entity." }

Returns up to three candidates ranked best-first:

{
  "prompts": [
    { "name": "research_person", "description": "Build a stakeholder profile..." },
    { "name": "staker", "description": "Assess positions on a proposal..." }
  ]
}

State the capability the way a tool author would document it: an imperative phrase naming the operation and what it acts on. Conversational phrasing resolves less reliably.

If retrieval is unavailable (model failed to load), need_prompt reports it and points you at list_prompts instead.

Live Reload

With watch = true (the default), saving a prompt file or prompts.toml triggers a re-resolution after the debounce window settles. The catalog and its retrieval index are published together as one atomic generation - no reader ever sees a torn pair.

What reload does:

• A healthy edit updates the catalog immediately. The tool list stays the same because tools are fixed; only the catalog behind run_prompt changes.
• A broken edit (parse error, bad name) retains the prompt as a listed entry carrying its problem rather than freezing the whole catalog.
• An edit to a prompt's body alone (no name or description change) carries the previous retrieval index forward without rebuilding it.
• A broken platform watch is re-registered on the next settled window rather than permanently losing live reload.

Set watch = false to serve a static catalog for the life of the process.

Transport and Security

HTTP

The streamable-HTTP transport puts MCP at /mcp and a liveness probe at /healthz. The bearer check wraps /mcp only - /healthz is unauthenticated by design.

Authentication is per-request, not per-session. The token is fixed for the life of the server, but the check happens on every HTTP request rather than once at initialization - so a session that already completed the MCP handshake is still refused if its credential does not match. The comparison is constant-time.

SSE keep-alive is 15 seconds, so a run that thinks between sections does not look dead to a proxy.

allowed_hosts is the DNS-rebinding defence. On a loopback bind it defaults to localhost, 127.0.0.1, ::1. On a non-loopback bind you must enumerate the authorities explicitly or the server refuses to start.

Stdio

Stdio binds no port, checks no token, and has a bounded line reader so a peer without newlines costs a fixed buffer rather than the process. The harness that spawned it is the only thing that can talk to it.

Shutdown

Ctrl-C triggers graceful shutdown on both transports. The SSE streams are closed, in-flight calls drain, and the watcher stops before the process exits. No late reload can publish after the shutdown signal.

Boot Sequence and Gateway

At startup the server:

1. Loads and validates prompts.toml
2. Resolves the catalog (refuses to start on any fault)
3. Builds the retrieval index over the catalog (if picker feature is present; a failure is logged and the server continues)
4. Fetches the gateway model catalog via GET /v1/models
5. Builds the live tool catalog (web_fetch, web_search) and the semantic tool picker
6. Starts the filesystem watcher
7. Serves the chosen transport

The gateway fetch distinguishes transient failures from fatal ones. A connection timeout or a 5xx is transient: the server warns and serves with an empty model catalog, so prompts without models.bind keep working. A 401, a bad URL, or a malformed response is fatal: the server refuses to boot rather than hiding a misconfiguration behind runtime failures.

Tool Picker

A sentence-embedding resolver that turns "read a file from disk" into the tool that does it - no LLM call, no network, no guessing. You describe your tools in prose, build a picker over the catalog, and ask it which tool a need refers to. It answers with a decision: one bound tool, a duplicate report, an ambiguous shortlist, or an abstention. The model is compiled into the library, so there is no path to configure and no weights to ship. Querying is a dot product, not an API call. Determinism is structural: the same inputs always produce the same answer.

Identity, Descriptors, and Catalogs

Tool Identity

Every tool is identified by a (server, name) pair. The pair is structural - never concatenated - so a server or name containing any delimiter stays unambiguous.

#![allow(unused)]
fn main() {
use promptforge_tool_picker::{ToolId, ToolDescriptor, ToolAnnotations, Catalog};
use serde_json::json;

let id = ToolId::new("files", "read_file");
assert_eq!(id.server(), "files");
assert_eq!(id.name(), "read_file");
}

Descriptors

A ToolDescriptor carries the identity, a prose description, a JSON Schema for the tool's arguments, and optional behavioral hints:

#![allow(unused)]
fn main() {
let tool = ToolDescriptor::new(
    ToolId::new("files", "read_file"),
    "Read a file from disk",
    json!({"properties": {"path": {"type": "string"}}}),
);

let tool = tool.with_annotations(
    ToolAnnotations::new()
        .with_read_only(true)
        .with_destructive(false),
);

assert_eq!(tool.name(), "read_file");
assert_eq!(tool.description(), "Read a file from disk");
assert_eq!(tool.annotations().read_only(), Some(true));
}

Annotations are optional and advisory. They affect ranking only as a tie-break between candidates that score identically. A positive read-only claim is preferred first, then non-destructive, then idempotent.

Catalogs

A Catalog is an ordered collection of descriptors:

#![allow(unused)]
fn main() {
let catalog = Catalog::new(vec![
    ToolDescriptor::new(
        ToolId::new("files", "read_file"),
        "Read a file from disk",
        json!({"properties": {"path": {"type": "string"}}}),
    ),
    ToolDescriptor::new(
        ToolId::new("net", "fetch_url"),
        "Fetch a web page over HTTP",
        json!({"properties": {"url": {"type": "string"}}}),
    ),
]);

assert_eq!(catalog.len(), 2);

let found = catalog.get(&ToolId::new("net", "fetch_url"));
assert_eq!(found.map(|t| t.name()), Some("fetch_url"));

for tool in &catalog {
    println!("{}: {}", tool.name(), tool.description());
}
}

You can also build a catalog from an iterator or a Vec:

#![allow(unused)]
fn main() {
let catalog: Catalog = vec![/* descriptors */].into();
let catalog: Catalog = some_iterator.collect();
}

JSON Deserialization

With the serde feature (enabled by default), catalogs deserialize from JSON. The identity fields are flat on each descriptor. The schema field accepts both input_schema and its MCP spelling inputSchema:

[
  {
    "server": "files",
    "name": "read_file",
    "description": "Read a file from disk",
    "inputSchema": {
      "properties": { "path": { "type": "string" } }
    },
    "annotations": { "readOnlyHint": true }
  }
]

Duplicate identities in a catalog are accepted. Two tools claiming the same identity is a result the engine reports, not an input it refuses.

Building a Picker

The simplest path loads the model and indexes a catalog in one call:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::{ToolPicker, Catalog, Config};

let picker = ToolPicker::build(catalog, Config::default())?;
assert_eq!(picker.len(), 2);
}

Sharing a Model

Loading the model is the expensive step - it materializes the compiled-in weights into memory. If you serve several catalogs, load the model once and build each picker against it:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::Model;

let model = Model::load()?;

let files_picker = ToolPicker::build_with_model(&model, files_catalog, Config::default())?;
let weather_picker = ToolPicker::build_with_model(&model, weather_catalog, Config::default())?;
}

Model is cheap to clone (it shares the loaded weights through an Arc), and it is Send + Sync + 'static, so you can pass it across threads.

Rebuilding

When your catalog changes - a reconnected server, a watched directory - rebuild from the existing picker to preserve its model and configuration:

#![allow(unused)]
fn main() {
let updated = picker.rebuild(new_catalog)?;
}

The original picker is immutable and still answers from its own catalog. The rebuilt picker answers from the new catalog with the same model and config.

You can iterate a picker's tools with picker.iter() or for tool in &picker, and look up a specific tool with picker.get(&id).

Resolving a Need

resolve takes a plain-English need and returns one of four outcomes:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::Outcome;

match picker.resolve("read a file from disk")? {
    Outcome::Bind(tool) => {
        println!("call {}", tool.name());
    }
    Outcome::Duplicate(group) => {
        println!("{} publishes {} twins", group.first().server(), group.len());
    }
    Outcome::Ambiguous(group) => {
        for tool in &group {
            println!("candidate: {}/{}", tool.server(), tool.name());
        }
    }
    Outcome::Absent => {
        println!("no tool covers this need");
    }
    _ => {}
}
}

Absent is a successful answer, not an error. An Err from resolve means the need could not be embedded (tokenization or inference failed), so no answer was produced at all.

Lifetime of Results

Results borrow the picker's descriptors. No schema or descriptor is deep-cloned. If you need to keep a tool identity beyond the picker's lifetime, clone the specific ToolId:

#![allow(unused)]
fn main() {
let kept_id: ToolId = match picker.resolve("read a file")? {
    Outcome::Bind(tool) => tool.id().clone(),
    _ => return Ok(()),
};
}

Candidate Groups

A CandidateGroup (from Duplicate or Ambiguous) always contains at least two entries. You can inspect them with group.first(), group.second(), group.get(index), group.len(), and group.iter().

Shortlisting

shortlist returns candidates above the similarity floor without making a final decision, so the caller can choose:

#![allow(unused)]
fn main() {
let candidates = picker.shortlist("read a file from disk", 3)?;

for tool in &candidates {
    println!("{}: {}", tool.name(), tool.description());
}

if candidates.is_empty() {
    println!("nothing relevant");
}
}

resolve and shortlist never contradict each other on relevance. If resolve abstains, shortlist returns nothing. If resolve binds a tool, shortlist offers exactly that tool.

The solo-candidate exception applies to both: when one candidate sits between the solo floor and the strict similarity floor, and no runner-up reaches the solo floor, that candidate is offered.

A limit of zero returns an empty shortlist without embedding the need. The Shortlist type offers .len(), .is_empty(), .first(), .get(index), and .iter().

Configuration

Config::default() provides justified defaults. A caller who has not measured their own catalog should change none of them:

Adjusting Thresholds

Adjust one threshold at a time with checked consuming setters:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::Config;

let config = Config::default()
    .with_similarity_floor(0.85)?
    .with_top_k(5)?;

assert_eq!(config.top_k().get(), 5);
}

Every Config is always valid. Thresholds must be finite and in 0.0..=1.0. top_k must be nonzero. There is no validate method because no public operation can produce an invalid value.

A setter that receives an out-of-domain value returns ConfigError, which names the rejected field:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::{ConfigError, ConfigField};

let error: ConfigError = Config::default()
    .with_similarity_floor(2.0)
    .expect_err("out of domain");
assert_eq!(error.field(), ConfigField::SimilarityFloor);
}

JSON Serialization

With the serde feature, configuration serializes and deserializes as JSON. Absent fields take their defaults, and checked deserialization rejects invalid wire values:

{"similarity_floor": 0.85, "top_k": 5}

Decision Precedence

Decision precedence is fixed: absent, then duplicate, then bind, then ambiguous. The similarity floor is checked first. Then same-server twins are detected against the duplicate threshold (measured between the tools' own embeddings, not against the query). Then the margin test separates a clear leader from a near-tie. Every threshold boundary is inclusive - a score exactly at the floor is considered.

The Solo-Candidate Rule

When the top candidate scores at or above solo_floor but below similarity_floor, and no runner-up reaches the solo floor, the leader binds. Two candidates between the floors abstain.

Near-Duplicate Detection

near_duplicates compares selected tools against the configured duplicate threshold using the picker's stored embeddings. The comparison is tool-to-tool, not need-to-tool - it measures how alike two tools' own descriptions are, independent of any query.

#![allow(unused)]
fn main() {
let pairs = picker.near_duplicates(&[
    ToolId::new("calendar", "create_event"),
    ToolId::new("calendar", "add_event"),
])?;

for pair in &pairs {
    println!(
        "{}/{} and {}/{} are {:.3} similar",
        pair.first().server(), pair.first().name(),
        pair.second().server(), pair.second().name(),
        pair.similarity(),
    );
}
}

Every requested identity must be present in the picker. An absent identity returns SelectionError before any comparison happens, naming the first missing ToolId via error.missing_id(). Repeated identities are idempotent set membership.

Pairs are output in catalog pair order. Each NearDuplicate provides .first(), .second(), and .similarity(). The NearDuplicates collection provides .len(), .is_empty(), .get(index), and .iter().

Error Handling

Each fallible operation returns its own error type. There is no crate-wide error enum.

QueryError::kind() returns a QueryErrorKind that classifies the failure without exposing dependency types:

#![allow(unused)]
fn main() {
use promptforge_tool_picker::{QueryError, QueryErrorKind};

match error.kind() {
    QueryErrorKind::Tokenization => { /* the need text could not be tokenized */ }
    QueryErrorKind::Inference => { /* the model's forward pass failed */ }
    QueryErrorKind::InvalidEmbedding => { /* the produced vector could not be normalized */ }
    _ => {}
}
}

BuildError wraps either a ModelLoadError or an IndexError, and implements From for both. All error types are Send + Sync + 'static.

Determinism and the Embedded Model

The crate promises deterministic results: the same model bytes, dependency versions, target, execution environment, catalog, configuration, and need always produce the same outcome. Cross-platform byte-identical vectors at floating-point boundaries are not promised.

The embedding model (BAAI/bge-small-en-v1.5, 384 dimensions) is compiled into the library. There is no model path in the configuration, no weights file to deploy, and no network call at runtime. The build script fetches the model from the Hugging Face Hub at a pinned immutable commit, verifies every file against a hardcoded SHA-256 digest, and downcasts the fp32 weights to fp16 to halve binary size. Subsequent builds reuse the Hugging Face cache.

At load time, the crate verifies that the embedded weights' provenance metadata matches the pinned repository and revision. A mismatched or substituted checkpoint fails loudly rather than silently altering rankings.

The first build requires network access to the Hugging Face Hub (about 130 MB download). Set HF_HUB_CACHE or HF_HOME to point at an existing cache, or HF_ENDPOINT to a reachable mirror.

Web Fetch

Hand a language model one tool and let it read the web. promptforge-webfetch fetches a URL, extracts the useful content, and returns it as markdown the model can cite - while enforcing an SSRF boundary that prevents the model from reaching your internal network no matter what URL it supplies. The common call is one argument (url). The security is layered and runs at DNS-resolution time on every hop, so it catches names that resolve inward, rebinding attacks, and redirect chains that point somewhere they should not.

By the end of this chapter you will know how to wire the tool into a promptforge pipeline, tune its policy for your deployment, and trust it with model-supplied URLs.

Fetching a Page

Construct the tool and call it with a URL:

#![allow(unused)]
fn main() {
use promptforge_webfetch::WebFetch;
use promptforge_core::tools::Tool;

let tool = WebFetch::new();
let output = tool.call(serde_json::json!({ "url": "https://example.com/article" })).await?;
println!("{}", output.text());
}

The tool accepts one required argument (url) and two optional ones (raw and max_chars). It performs a GET, classifies the response by content type, and returns the text behind a provenance header:

url: https://example.com/article
truncated: false
extraction: readability

# Article Title

The main content rendered as markdown...

The three header fields are a contract:

• url - the final URL after any redirects, so the model knows where its text came from
• truncated - whether the text was cut short by a size cap
• extraction - which of three processing paths produced the output: readability (article isolation), raw-html (whole-page render), or plain (non-HTML text returned verbatim)

How Content Is Processed

The response's Content-Type header decides the route before the body is downloaded.

HTML

Content types text/html and application/xhtml+xml are processed with a readability algorithm that isolates the main article and renders it to markdown. If the extracted article is shorter than 100 characters, the whole page is rendered instead, automatically. The extraction: header tells you which path fired.

Structured Text

Content types application/json, application/xml, text/xml, and any +json/+xml suffix are returned verbatim as decoded text. No extraction, no transformation.

Flat Text

All other text/* types are returned decoded. If the text exceeds the byte cap, the prefix is kept and truncated: true is set.

Unsupported Types

PDF, images, audio, video, and application/octet-stream are refused with a message naming the content type so the model can try a different URL.

Missing Content-Type

Refused. The tool does not sniff.

Raw Mode

Use raw when article extraction would discard the content you want - for example a page that is mostly a data table:

#![allow(unused)]
fn main() {
let output = tool.call(serde_json::json!({
    "url": "https://example.com/pricing",
    "raw": true
})).await?;
}

This forces whole-page rendering and reports extraction: raw-html. Ignored for non-HTML responses.

Responses compressed with gzip or brotli are decompressed transparently.

Size Limits and Truncation

Two caps govern how much data the tool accepts:

• Byte cap (max_bytes, default 8 MiB) - the largest decompressed response body. A declared Content-Length over this cap is refused before any bytes are read. A streaming body that crosses it mid-read is aborted.
• Character cap (max_chars, default 40,000) - the longest text returned to the model. Text is cut on a character boundary so multibyte characters are never split.

The two caps interact differently depending on the content type:

A per-call max_chars argument lets the model request less text for one call:

#![allow(unused)]
fn main() {
let output = tool.call(serde_json::json!({
    "url": "https://example.com/long-page",
    "max_chars": 5000
})).await?;
}

The per-call value is clamped to the configured ceiling - a model cannot request more than the policy allows, only less.

Security Policy

The default policy (WebFetch::new()) is safe for fetching the public internet: HTTPS only, ports 80 and 443, no bare IP-literal URLs, every non-globally-reachable address blocked.

Customizing the Policy

Use the builder to adjust the defaults:

#![allow(unused)]
fn main() {
use std::time::Duration;
use promptforge_webfetch::{FetchConfig, WebFetch};

let policy = FetchConfig::builder()
    .allow_http(true)
    .allow_ports([80, 443, 8080])
    .max_bytes(16 * 1024 * 1024)
    .max_chars(100_000)
    .timeout(Duration::from_secs(60))
    .user_agent("my-service/1.0")
    .build()?;

let tool = WebFetch::try_with_config(policy)?;
}

Every setter returns self for chaining. Validation happens once at .build(), which returns ConfigError for any invalid field. The available knobs:

Reaching an Internal Host

By default, every non-globally-reachable address is blocked. The only supported way to reach one is an exact host-plus-address pair:

#![allow(unused)]
fn main() {
use std::net::IpAddr;
use promptforge_webfetch::FetchConfig;

let addr: IpAddr = "10.0.5.42".parse()?;
let policy = FetchConfig::builder()
    .allow_http(true)
    .allow_ports([80, 443, 8080])
    .allow_host_address("wiki.internal.corp", addr)
    .build()?;
}

The escape hatch is deliberately narrow:

• Keyed on both host and address, so evil.com resolving to 10.0.5.42 does not inherit the exception
• Grants access to exactly one address, not a range
• The host is canonicalized (lowercased, trailing dot stripped) so case variants match

You can also block additional ranges for your deployment:

#![allow(unused)]
fn main() {
let policy = FetchConfig::builder()
    .deny_cidr("10.99.0.0/16")
    .deny_cidr("172.20.0.0/14")
    .build()?;
}

The SSRF Boundary

The tool enforces four layers of defense, in order:

URL Admission

Runs before any network access. Rejects bad schemes, embedded userinfo, non-allowed ports, and bare IP literals that map to blocked addresses. Catches obfuscated IPv4 encodings (0177.0.0.1, 2130706433, 127.1, [::ffff:127.0.0.1]).

Guarded DNS Resolver

Runs at connect time on every hop. Resolves the host, filters the answers through the address policy, hands only the allowed addresses to the HTTP client. A host that resolves entirely to blocked addresses fails. A host with mixed public/private answers connects to the public one. No verdict is cached, so a DNS-rebinding answer is caught on the hop that returns it.

Redirect Re-validation

Runs on every redirect hop. Re-runs the full URL policy on the redirect target. Refuses HTTPS-to-HTTP downgrades. Enforces the hop cap. The resolver re-classifies the redirect target's addresses at connect time.

No Ambient Identity

The client carries no cookies, no Authorization header, no Referer, and disables ambient proxy (HTTP_PROXY/HTTPS_PROXY). A redirect cannot smuggle credentials to a cross-origin target.

Blocked Address Table

The built-in table covers all IPv4 and IPv6 special-use space: loopback, RFC1918, CGNAT, link-local (including 169.254.169.254), documentation, benchmarking, multicast, reserved, and IPv6 equivalents including IPv4-mapped, NAT64, unique-local, and deprecated site-local. IPv4-embedded IPv6 addresses (::ffff:127.0.0.1, ::10.0.0.1) are normalized to their embedded IPv4 value and reclassified.

Error Behavior

Errors split into two categories based on whether a retry makes sense.

Soft Outcomes

Returned as tool text the model can act on:

• HTTP error status (404, 500, etc.)
• Timeouts
• DNS failures
• Unsupported or absent content type
• Body too large
• Body read failure mid-stream
• Redirect refused
• Blocked scheme (http when only https is allowed)

Hard Errors

The URL itself is invalid and no retry will help:

• Unparseable URL
• URL contains userinfo
• Port not on the allowlist
• IP literal not allowed
• Address is blocked / no allowed address for the host

When a blocked address is reported to the model, only the host name appears in the message - never the resolved address or the blocking range. Query strings and fragments are redacted from all diagnostic URLs so a ?token=secret never reaches logs or model output.

Development Runner

promptforge-dev is the edit-run-inspect loop for PromptForge prompts. Point it at a prompt file, and it runs the prompt against your already-running gateway, dumps the store for inspection, and optionally watches for saves so every edit triggers a fresh run. No gateway management, no model downloads, no weight files - just the prompt and its output, tight enough that your iteration cycle is limited by how fast you can think, not how long you wait.

Prerequisites

promptforge-dev requires a running promptforge-gateway. Start it yourself, then export two environment variables:

export PROMPTFORGE_GATEWAY_URL=http://127.0.0.1:8081/v1
export PROMPTFORGE_GATEWAY_API_KEY=<bearer from your gateway profile>

Both must be set and non-empty. If either is missing, the binary fails immediately with a message naming the missing variable and reminding you to start the gateway. No prompt file is read until both are validated.

Your First Run

From the PromptForge repository root:

cargo run -p promptforge-dev -- my-prompt.md

This runs my-prompt.md with an empty input. The second positional argument supplies an input string:

cargo run -p promptforge-dev -- my-prompt.md "summarize this paragraph"

The input becomes the prompt's args. If you omit it, it defaults to empty.

Model runtime parameters - context window, thinking mode, max tokens - are not CLI flags. Declare them on the prompt file under models.bind or models.default. The binary's argument surface is deliberately minimal:

promptforge-dev [--watch] [--capture-raw] <prompt.md> [input]

What Happens During a Run

Each invocation follows a fixed pipeline:

1. Validate environment. Confirm PROMPTFORGE_GATEWAY_URL and PROMPTFORGE_GATEWAY_API_KEY are set.
2. Fetch the model catalog. One HTTP call to the gateway. The catalog is fetched once and reused across watch-mode reruns.
3. Build the tool set. Two tools are always constructed: web_fetch (runs locally) and web_search (proxies through the gateway). A semantic tool picker is derived from the same live set, so no picker descriptor can advertise a tool without a matching callable.
4. Parse the prompt. The file must declare promptforge: in its YAML frontmatter. A file without it is refused: "is not a promptforge prompt."
5. Execute. The prompt runs against the gateway. The store stays in memory during execution - no filesystem writes happen on the async path.
6. Dump the store. After the run (success or failure), the in-memory store is reconciled to disk beside the prompt file.

Execution ID

A unique execution id is minted for each run: dev- followed by 128 random hex bits. It prints to stderr before any observer output, so you can always tell which run produced which output:

run id: dev-3a7f1b2c9e4d5a8f0011223344556677

Observer Output

Observer records stream to stderr as single trace lines:

[dev-3a7f1b2c9e4d5a8f0011223344556677] Research: Run started
[dev-3a7f1b2c9e4d5a8f0011223344556677] Research: Lua: checkpoint

The final result prints to stdout. This separation lets you pipe or redirect output without observer noise.

Inspecting Output

Every run dumps its store to <prompt-stem>.store/ beside the prompt file. For a prompt named briefer.md, the dump lands in briefer.store/:

briefer.md
briefer.store/
  evidence.md
  notes/
    deep.txt

The dump reconciles on every run:

• Changed files are overwritten with current contents.
• Files from a previous run that are no longer in the store are deleted.
• The .trace/ subdirectory (used by raw trace capture) is preserved across reconciles.
• When the store is empty and no trace files remain, the dump directory is removed entirely.

A failed run still dumps its partial store. That partial output is exactly what you need when debugging a prompt that errored partway through.

Watch Mode

Add --watch to enter a rerun loop:

cargo run -p promptforge-dev -- --watch my-prompt.md "test input"

The prompt runs once, then the file is watched for changes:

watching my-prompt.md for changes; press Ctrl-C to stop

Every save triggers a rerun after a 300 ms debounce quiet period. The debounce absorbs editor write-then-rename save bursts so a single save produces a single rerun, not two or three.

The gateway catalog, tools, and picker built at startup are reused across every rerun - no repeated network calls. Each rerun gets a fresh execution id.

If a rerun fails, the error prints to stderr and watching continues. A broken edit does not kill your session.

Watcher Internals

The watcher monitors the prompt's parent directory, filtered to the prompt's file name. Store dump writes (to the .store/ directory) do not retrigger reruns. The watcher uses a capacity-one bounded channel, so a slow rerun or a noisy filesystem cannot grow an unbounded event backlog. Watcher backend errors surface through a separate loss-proof slot - they are never silently dropped, even when the channel is full.

Raw Trace Capture

Add --capture-raw to persist the verbatim request and response bodies for each model turn:

cargo run -p promptforge-dev -- --capture-raw my-prompt.md

A warning prints to stderr:

warning: --capture-raw persists verbatim prompts, tool arguments and results, and model output to my-prompt.store/.trace

Each model turn writes two files under .trace/:

my-prompt.store/
  .trace/
    turn-1-request.json
    turn-1-response.json
    turn-2-request.json
    turn-2-response.json

These contain the full, unredacted request and response JSON. The material is sensitive - raw prompts, tool arguments and results, model output - which is why capture is off by default and requires an explicit flag.

Capture Internals

Trace capture uses a bounded queue (128 events) with a dedicated worker thread. The worker serializes and writes each payload with owner-only permissions and atomic semantics. If the worker falls behind, events are counted as dropped and the count is reported when the run finishes. I/O never blocks the run's async task.

All queued writes are flushed before the store dump reconcile, so trace files are always complete when you inspect the dump directory.

Filesystem Security

All dump writes - store files and trace captures - go through a security layer:

• Owner-only permissions. Directories are created 0o700 and files 0o600 on Unix. On Windows, inherited access is stripped and full control is granted to the current user alone via icacls.
• No symlink traversal. Every write checks the target and all existing ancestors for symlinks and Windows reparse points. A planted link at any path component is refused, preventing writes from escaping the dump tree.
• Atomic writes. Each file is written to a sibling temporary (.{name}.tmp{random}), flushed, permission-restricted, then renamed over the destination. An interrupted write cannot truncate a prior file. The temporary is cleaned up on failure.
• Path safety. Store paths that are absolute, traverse with .., contain backslashes, control characters, or Windows reserved characters (*, ?, ", <, >, |) are skipped with a status report. Windows reserved device names (CON, PRN, AUX, NUL, COM1-9, LPT1-9 - including Unicode superscript digit variants) are also rejected.

You do not configure any of this. It is always active.

Diagnostics

When a Lua error maps to a prompt line, the failure message leads with the file and line number:

dev run failed: briefer.md:51: run briefer.md: lua error: section `Web Search` epilog:51: assertion failed!

This format enables click-to-navigate in editors that recognize file:line: patterns.

Errors without a mapped prompt line omit the line prefix:

dev run failed: some transport error

Exit Codes

Ctrl-C is handled cooperatively: the run is cancelled, its completion is awaited (so blocking fanout joins are not abandoned), and the process exits with code 130.

Edge Cases and Validation

Unknown flags. Any flag starting with -- that is not --watch, --capture-raw, or -- is rejected with usage text. This includes former server knobs like --context, --max-tokens, and --no-think that were removed when model parameters moved to the prompt file.

Non-PromptForge files. A markdown file whose YAML frontmatter does not declare promptforge: is refused with a clear message rather than producing a confusing parse error.

The -- delimiter. Use -- to pass an input that begins with dashes:

cargo run -p promptforge-dev -- my-prompt.md -- --this-is-input-not-a-flag

Everything after -- is treated as a positional argument.

Credential protection. The bearer key is wrapped in a GatewayKey type that renders as <redacted> in Debug output. An accidental {:?} on a GatewayEnv cannot leak the credential.

Errors

PromptForge uses typed errors at every public boundary rather than a single crate-wide error enum. Each error type exposes a stable kind() classifier for programmatic handling, and public structs are #[non_exhaustive] so they can evolve without breaking downstream code.

This chapter covers the error taxonomy across all three crates and the debugging facilities that help you diagnose failures in practice.

Core Error Types

The core library defines five error types, each with its own set of kinds and boolean query methods. Backend error bodies are accessible through opt-in accessors but never leak into Display output.

RunError

RunError is the top-level error returned by the run function. It wraps failures from every subsystem - parsing, model inference, tool dispatch, Lua execution, and store operations - into a single type with discriminated kinds.

#![allow(unused)]
fn main() {
match run(&prompt, input, ctx, &store, config).await {
    Ok(result) => println!("{result}"),
    Err(e) if e.is_cancelled() => println!("run was cancelled"),
    Err(e) if e.is_retryable() => println!("transient failure: {e}"),
    Err(e) => println!("fatal: {e}"),
}
}

CompletionError

CompletionError covers model inference failures. The is_retryable() query distinguishes transient network issues from permanent configuration problems, and is_timeout() identifies request timeouts specifically. The status() accessor exposes the HTTP status code when the backend returned one.

Key kinds:

• Transport - network-level failure (DNS, connection refused, TLS). Retryable.
• Backend - the gateway or upstream returned an error HTTP status.
• MalformedResponse - response body could not be decoded.
• EmptyReply - the model returned no content.
• Disabled - the client was constructed with GatewayClient::disabled().
• Config - missing or invalid client configuration (bad URL, empty key).

StoreError

StoreError covers virtual filesystem operations. The is_not_found() query identifies missing-file reads, and path() returns the offending path when available.

Path validation rejects backslashes, traversal segments (. and ..), Windows reserved device names, trailing dots or spaces, and paths exceeding 1024 bytes.

ToolError

ToolError covers tool dispatch failures. Tools can fail from invalid arguments, backend errors, transport problems, or cancellation. The is_retryable() and is_cancelled() queries work the same as on RunError.

ParseError

ParseError reports prompt-file structural problems at parse time. Each error carries a stable kind() discriminant and an optional byte span() for editor diagnostics. Lua compilation errors include absolute source-line numbers that map back to the original prompt file.

Version Detection

promptforge_version(source) detects whether a file is a promptforge prompt without requiring a full parse - it needs only the promptforge: key in the frontmatter. Use this for fast filtering before committing to a parse.

Gateway Errors

The gateway uses the OpenAI error envelope for all HTTP error responses, so an unmodified OpenAI SDK surfaces these as its own error types rather than unparseable blobs.

{
  "error": {
    "message": "unknown model reasoning-large",
    "type": "invalid_request_error",
    "code": "model_not_found"
  }
}

Error Codes

A 502 with upstream_transport is a network-level failure (DNS, connection refused, TLS handshake). A 502 with upstream_protocol means the backend responded but its body could not be decoded. Both are transient and worth retrying.

A queue_full 503 means every concurrency slot on the endpoint is occupied and the queue depth has been exceeded. Retry after a back-off.

Boot-Time Failures

The gateway validates its TOML configuration strictly. Every configuration struct uses deny_unknown_fields, so a misspelled key is a boot failure rather than a setting silently ignored. An unresolved ${VAR} reference fails the load, so a deployment that forgot to export a credential never starts serving with a blank one.

MCP Server Errors

The MCP server surfaces errors through the run_prompt result envelope.

Run Result Status

Every run_prompt result carries a status field:

• completed - the run finished successfully; value contains the artifact.
• failed - the run finished with an error; error contains the message.
• running - the run exceeded reply_deadline and continues in background.

{
  "run_id": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6",
  "prompt": "research_person",
  "status": "failed",
  "value": null,
  "turns": 1,
  "elapsed_ms": 3200,
  "error": "CompletionError: transport timeout after 120s"
}

Admission and Timeouts

• Admission timeout - when all max_concurrent_runs slots are occupied and none frees within admission_timeout (default 30s), the call gets a retryable refusal.
• Reply deadline - when a run exceeds reply_deadline (default 240s, inside Cursor's 300s call ceiling), the call returns immediately with status running and a run_id. Collect the result later with check_run.

Gateway Connectivity

At startup, the MCP server distinguishes transient gateway failures from fatal ones:

• Transient (connection timeout, 5xx) - the server warns and serves with an empty model catalog. Prompts that do not use models.bind keep working.
• Fatal (401, bad URL, malformed response) - the server refuses to boot rather than hiding a misconfiguration behind runtime failures.

Catalog Errors

A broken prompt (parse error, invalid name) appears in list_prompts with its problem field populated rather than silently disappearing:

{
  "name": "broken_one",
  "description": "",
  "problem": "parse error at line 3"
}

Gateway Client Configuration

The gateway client is how errors from the model layer surface in practice. Two environment variables configure it:

export PROMPTFORGE_GATEWAY_URL="https://your-gateway.example.com"
export PROMPTFORGE_GATEWAY_API_KEY="your-bearer-token"

Or construct programmatically:

#![allow(unused)]
fn main() {
let client = GatewayClient::new(endpoint, key);
}

Point PROMPTFORGE_GATEWAY_URL at a local server or another gateway to retarget all model calls. The credential is automatically redacted in Debug output, Display, and logs. Empty credentials are rejected at construction time.

Gateway URLs are validated at construction:

• Non-HTTP schemes are rejected
• Embedded credentials are rejected
• Query strings and fragments are rejected
• Trailing slashes are normalized

For testing, GatewayClient::disabled() creates a client that always returns a Disabled error - useful for running parse-only or Lua-only tests without a live gateway.

Without an explicit .client() on RunConfig, the runtime lazily constructs one from the environment variables above.

Observation and Debugging

Observer Trait

The observer is a pluggable, report-only seam for watching execution in flight. Implement the Observer trait:

#![allow(unused)]
fn main() {
fn observe(&self, execution: &str, section: &str, event: Observation<'_>);
}

Events include:

• Parse started/completed
• Run started/succeeded/failed
• Section started/finished
• Model turn completed/truncated
• Tool call succeeded/failed
• Store operations
• Fanout arm lifecycle
• Lua log checkpoints

All observations are correlated by execution id and section name. NullObserver discards all events when no tracing is needed. Attaching or detaching an observer does not change execution results.

Debug Capture

A separate debug sink records raw request and response JSON for each model turn:

#![allow(unused)]
fn main() {
fn on_event(&self, execution: &str, section: &str, turn_index: u32, event: DebugEvent);
}

Debug events capture the full request body as JSON and the response finish reason with reasoning content. Events from nested model:infer calls and fanout arms are forwarded to the same sink.

Cancellation

Cancellation is cooperative via a caller-supplied CancelHandle. It propagates into tools, models, Lua instruction hooks, and fanout arms.

#![allow(unused)]
fn main() {
let cancel = CancelHandle::new();

// From another task:
cancel.cancel();

// The run returns:
match result {
    Err(e) if e.is_cancelled() => { /* clean shutdown */ }
    _ => {}
}
}

A cancelled run returns a RunError with is_cancelled() == true, distinguishable from faults. In the MCP server, client disconnection during a run triggers cancellation cooperatively.