Skip to content

Pipelex API Documentation

Welcome to the Pipelex API documentation. The API provides programmatic access to the Pipelex system.

The three-layer contract

This server is the source-available reference implementation of the MTHDS Protocol — the minimal HTTP contract every MTHDS runner implements. The contracts nest:

MTHDS Protocol  ⊂  Pipelex API (this server)  ⊂  Pipelex hosted API
(the standard)     (protocol + build tooling)    (+ durable runs, catalog, account)
  • MTHDS Protocol — five routes: POST /execute, POST /start, POST /validate, GET /models, GET /version. Tagged x-mthds-protocol: true in the committed OpenAPI artifact, and only those five — the flag is how a conformance suite or a third-party runner extracts the portable subset.
  • Pipelex API (this server) — the protocol verbatim, plus the Pipelex extensions: resolve and codegen (/resolve, /codegen), build tooling (/build/*), and editor tooling (/lint, /format). /upload and /resolve-storage-url were non-contract convenience routes and have been removed — see Storage Transport for where they went and why.
  • Pipelex hosted API (api.pipelex.com/v1) — everything here, same shapes, plus durable runs, the method catalog, and account management.

All routes are served under the /v1 base path (clients compose {base}/v1/{endpoint}).

What the API Offers

The API currently allows you to:

  1. Run any Pipelex pipeline with flexible inputs (sync or async)
  2. Validate any Pipelex pipeline to ensure correctness
  3. Resolve a library closure into its normalized crate, and codegen typed artifacts from it (TypeScript/Zod, Pydantic, Pipelex structures)
  4. Build pipeline components — generate input schemas, output representations, runner code, concepts, and pipe specs
  5. Lint and format single .mthds files for editor workflows
  6. List available model presets and configurations
  7. Upload files via presigned URLs

Deployment

Deploy the Pipelex API anywhere that runs Docker (your laptop, ECS, Cloud Run, Kubernetes, …) using our Docker image: pipelex/pipelex-api.

1. Run with Docker

The only required env var is PIPELEX_GATEWAY_API_KEY. Get a free key (with free credits) at https://app.pipelex.com — it's the default path to LLMs and gives you access to every supported model with a single credential. (If you'd rather call providers like OpenAI, Anthropic, Bedrock, or Vertex directly, you reconfigure that on the Pipelex side, not here — see https://docs.pipelex.com.)

docker run --name pipelex-api -p 8081:8081 \
  -e PIPELEX_GATEWAY_API_KEY=your-pipelex-gateway-api-key \
  pipelex/pipelex-api:latest

To require authentication on the API itself, add -e AUTH_MODE=api_key -e API_KEY=your-secret (or AUTH_MODE=jwt + JWT_SECRET_KEY). The full set of accepted env vars is documented in Configuration and .env.example.

If you'd rather keep config out of your shell history, use --env-file .env or a docker-compose.yml instead — see Configuration → Setting env vars in Docker for both patterns.

To build the image yourself instead of pulling, replace pipelex/pipelex-api:latest with a local tag after docker build -t pipelex-api ..

2. Verify

curl http://localhost:8081/health

3. Run your first pipeline

Send an inline MTHDS bundle and inputs to /v1/execute:

curl -s http://localhost:8081/v1/execute \
  -H "Content-Type: application/json" \
  -d '{
    "pipe_code": "summarize",
    "mthds_contents": ["domain = \"hello\"\nmain_pipe = \"summarize\"\n\n[pipe.summarize]\ntype = \"PipeLLM\"\ndescription = \"Summarize the input text in one sentence\"\ninputs = { text = \"Text\" }\noutput = \"Text\"\nprompt = \"Summarize in one sentence:\\n@text\"\n"],
    "inputs": { "text": "Pipelex turns plain-language pipeline definitions into reproducible AI workflows that run as HTTP endpoints." }
  }'

The response contains state: "COMPLETED" and the result under pipe_output.working_memory.root.<main_stuff_name>.content. See Pipe Run → for every input shape (text, structured objects, Document, Image, …) and the full /execute and /start reference.

4. Customize the configuration

Need to change the execution mode, point to a different storage backend, or ship your own model deck? See Configuration → for how to provide your own .pipelex/ config files to the Docker image. The base runs every pipeline in-process by default; distributed execution (Temporal, …) is added by a deployment flavor, not configured on the base.

Base URL

Once deployed locally, the API is available at:

http://localhost:8081/v1

Authentication

The API supports three authentication modes via the AUTH_MODE environment variable:

No Authentication (Default)

By default (AUTH_MODE=none), the API requires no authentication. This is the default for self-hosted deployments and for running behind an API Gateway that handles auth.

If you sit this API behind a trusted reverse proxy that authenticates users and forwards the caller identity via the X-User-Id header, set TRUST_FORWARDED_IDENTITY_HEADERS=true to honor it. The runner is a generic execution engine — it does not own user metadata (email, OAuth subject, auth method), so a single opaque caller id is the entire trusted surface. The value must be a single path-safe segment (is_safe_user_id). Default is off — without this flag the API ignores X-User-Id entirely and the deployment is treated as single-tenant. With it on, a request arriving without the header is rejected with 401: turning the flag on asserts that a proxy authenticates every caller, so a missing id means that proxy is absent, misconfigured or bypassed. Only enable it when your proxy strips any inbound copy of the header before adding its own; otherwise, any external client can spoof user identity by sending it directly.

API Key Authentication

Set AUTH_MODE=api_key and provide the API_KEY environment variable. Include it in the Authorization header:

Authorization: Bearer YOUR_API_KEY

docker run --name pipelex-api -p 8081:8081 \
  -e AUTH_MODE=api_key \
  -e API_KEY=your-api-key \
  pipelex/pipelex-api:latest

JWT Authentication

Set AUTH_MODE=jwt and provide the JWT_SECRET_KEY environment variable:

docker run --name pipelex-api -p 8081:8081 \
  -e AUTH_MODE=jwt \
  -e JWT_SECRET_KEY=your-jwt-secret-key \
  pipelex/pipelex-api:latest

JWT Requirements:

  • Tokens must be signed with the HS256 algorithm
  • Tokens must contain a user_id claim that is a single path-safe segment (is_safe_user_id) — it becomes a key segment in storage paths, so provider-issued sub values carrying /, # or : (like "google#abc") are NOT accepted. Deployments using OAuth must mint their own user_id claim mapping each caller to such a value.
  • Pass the JWT in the Authorization header: Authorization: Bearer YOUR_JWT_TOKEN

API Endpoints

Every failure is an RFC 7807 application/problem+json problem document — see Error Responses →, and the committed OpenAPI artifact for the statuses each route can produce.

Health & Version

  • GET / — Service identity banner (no auth required)
  • GET /health — Health check (no auth required)
  • GET /v1/version — MTHDS Protocol version handshake (no auth required): {protocol_version, implementation, implementation_version, runtime_version}. Replaces the former /pipelex_version and /api_version routes.

Pipe Run

Execute pipelines with flexible input formats, either synchronously or asynchronously.

  • POST /v1/execute — Run a pipeline and wait for completion (200 + full result)
  • POST /v1/start — Start a pipeline execution without waiting (202 + StartAck)

Learn more →

Pipe Validate

Validate MTHDS content to ensure pipelines are correctly defined before execution.

  • POST /v1/validate — Parse, validate, and dry-run pipelines

Learn more →

Resolve & Codegen

Resolve a library closure into its normalized crate, and project that crate into typed artifacts. Pipelex API extensions — not MTHDS Protocol routes, though the crate they emit is the standard's Library Crate Format.

  • POST /v1/resolve — Resolve a closure into the normalized library crate (fully qualified refs, flattened refinement, materialized natives, fingerprint)
  • POST /v1/codegen — Generate typed artifacts from the crate: kind (types) × target (ts-zod, python-pydantic, python-structures), plus the codegen.lock that makes the result reproducible offline

Learn more →

MTHDS Tools

Lint and format single .mthds files without loading or executing a pipeline.

  • POST /v1/lint — Return syntax, semantic, or schema diagnostics
  • POST /v1/format — Return formatted content, changed status, and blocking syntax diagnostics

Learn more →

Pipe Builder

Generate input templates, output representations, and runner code for one pipe of a library closure. All three take the same closure selector as /v1/resolve and /v1/codegen (inline files[] XOR method_ref), plus an optional qualified pipe_ref defaulting to the fetched package manifest's main_pipe on a method_ref request, else to the closure's declared main_pipe.

  • POST /v1/build/inputs — Generate an example inputs template for a pipe (JSON or TOML)
  • POST /v1/build/output — Generate an output representation (schema, JSON, or Python)
  • POST /v1/build/runner — Generate Python runner code for a pipe

Learn more →

Agent

Tools for AI agents building pipelines programmatically.

  • POST /v1/build/concept — Convert a JSON concept spec to TOML
  • POST /v1/build/pipe-spec — Convert a JSON pipe spec to TOML
  • GET /v1/models — The protocol model deck this runner routes to (flat models list, plus category-keyed aliases/waterfalls routing extensions); optional single ?type= category filter

Uploader (auth-gated, NON-CONTRACT)

These endpoints exist in the server but are NOT part of the published Pipelex API contract — they are deployment conveniences slated for replacement by the storage redesign. They require an authenticated user identity and reject unidentified requests with 401; AUTH_MODE=api_key does not establish one (the key is shared, not per-caller), so use AUTH_MODE=jwt, or a trusted proxy forwarding X-User-Id with TRUST_FORWARDED_IDENTITY_HEADERS=true.

POST /v1/upload and POST /v1/resolve-storage-url have been removed (Storage Transport). The storage provider itself is untouched — only the two HTTP routes are gone.

For most use cases you don't need either: pass any public HTTP(S) URL (or base64 data URL) directly as Document.content.url and skip the upload step entirely.