Skip to main content
Programmatic KB management is essential for any team with more than a handful of sources. Use it to:
  • Sync content from your CMS on a schedule
  • Build pattern-based ingestion (one KB per product, populated from canonical URLs)
  • Run KB queries from workflows without going through an agent
Full reference: Knowledge Base API. All paths are under the /knowledge-base prefix.

Create a KB

The response has the dataset_id you’ll use for all subsequent calls. (In the API and database a knowledge base is called a dataset.)

Add sources

One file at a time

One URL at a time

See Crawling URLs for the crawl options you can pass with a URL source.

Pattern-based bulk add

Poll source readiness

A source is queryable when its processing status reaches ready. Poll the dataset’s sources:
Response includes each source’s status (pending, converting_to_markdown, chunking, embedding, ready, failed) plus total_chunks and processed_chunks for in-flight sources.

Query a KB

Run retrieval without going through an agent:
Response: a list of matching chunks with similarity scores. This is also how HTTP steps in workflows can call your KB — point them at the /knowledge-base/datasets/{id}/query endpoint.

Sync pattern: idempotent CMS → KB

There is no in-place “refresh” of a source — re-ingesting means deleting and re-adding. A typical nightly sync workflow:
Wire this as a workflow on a daily cron trigger using a code step.

Limits and rate

  • The add-source endpoint accepts an array of sources; use /sources/pattern for large URL batches
  • Embedding throughput is bounded by your OpenAI/Azure quota
  • Query rate is bounded by your plan