Skip to content

dragiter Technical Reference

Version: derived from source (2026.10.22)
Language: British English
Scope: Configuration, file formats, CLI, defaults, output behaviour and activity log
Sources: Source code (src/dragiter/), in particular PromptCreator, OpenAIServiceExt and example TOML files under examples/

This document is the technical reference. It describes what the program actually accepts and does. It does not replace the user manual or how-to guides.


1. Configuration precedence

dragiter resolves every setting in the following order (highest priority first):

  1. Command-line arguments
  2. TOML configuration file (-c / DRAGITER_CONFIG_FILE / ~/.config/dragiter/config.toml)
  3. Environment variables (DRAGITER_*)
  4. Built-in defaults (applied only when a value is still unset)

Once a value has been set by a higher-priority source it cannot be overwritten by a lower-priority source.

Seven options are bootstrap options: -d, -v, -L, -b, -a, -s and -c. They decide how a run is set up (logging, path resolution, activity trace, simulation, configuration file), so they are read first, before anything else is built. They follow the same precedence as every other setting, so they may also be set in the configuration file. Only config_file cannot: a configuration file that contains it is rejected with an error naming the key. See section 3.

Four settings that can also appear inside a prompt template file are handled by the PromptCreator:

  • temperature
  • sequential_processing
  • output_delimiter
  • output_filename_schema

When a prompt file (-p) is supplied, the effective precedence for these four settings becomes:

  1. Command-line arguments
  2. Main configuration system (TOML config file or environment variables)
  3. Values present in the prompt template itself ([behaviour] / [outcome] sections)
  4. Hard-coded defaults inside PromptCreator
    (temperature = 0.0, sequential_processing = false,
    output_delimiter = "\n", output_filename_schema = "dragiter-out.txt")

In other words, the prompt template acts as an additional, lower-priority layer that is consulted only when the setting has not already been supplied by the CLI or the main configuration system.

Config file discovery order

When no --config-file / -c is given:

  1. Environment variable DRAGITER_CONFIG_FILE
  2. Default location ~/.config/dragiter/config.toml (only if the file exists)

When both -b / --base-directory and -c / --config-file are set, a relative -c path is rebased onto -b before the file is read. Absolute -c paths are left unchanged. config_file is not part of the later general path-rebase pass. Only a -b from the command line or DRAGITER_BASE_DIRECTORY applies to -c; a base_directory set inside the configuration file does not, because the location of the file cannot depend on the file itself.

If -c or DRAGITER_CONFIG_FILE names a file that does not exist, cannot be read or is not valid TOML, the run ends before anything else happens, with an error on stderr naming the file. No activity file is created.


1a. LLM client and runtime requirements

The CLI wires OpenAIServiceExt (src/dragiter/infrastructure/llm/openai_service_ext.py).

  • Live completions always stream (stream=True).
  • The HTTP stack is httpx2 via openai.DefaultHttpx2Client.
  • Read timeout is unlimited; connect / write / pool timeouts stay bounded.
  • --ca-bundle-file, --client-cert-file and --client-key-file are combined into one ssl.SSLContext, passed to httpx2 as verify. Without a CA bundle the context keeps the httpx2 default trust (the system trust store).
  • --tcp-keep-alive maps to httpx2.HTTPTransport(socket_options=...); that transport receives the same SSL context. On a platform that cannot set the keepalive times (neither TCP_KEEPIDLE nor TCP_KEEPALIVE), the run ends with exit code 3; set tcp_keep_alive = false there.
  • Install range: openai>=3.0.0,<4.0.0 and httpx2>=2.7.0,<3.0.0. openai 1.x / 2.x do not export DefaultHttpx2Client and do not install httpx2. The upper bounds keep a future openai 4.x from dropping that client under a live install.
  • The legacy non-streaming openai_service.py adapter has been removed from the tree; OpenAIServiceExt is the only LLM adapter the CLI wires. httpx (not httpx2) is now a development-only dependency, kept solely for test doubles.

With -v / --verbose dragiter writes a labelled board on stderr: a start block (prefix ▷ on every line: live api on or simulate api off, model, mode, sessions, chunks/files, small/over, pack, window, peak, warns, output), one request line per completion, and a closing block (prefix □ on every line). The request line carries a single pulse mark on the left (◷◶◵◴, one cell, same family as ▷ and □). The mark advances from the streaming for chunk in stream: loop, at most every ten seconds; there is no thread. The board is never written to stdout. -d / --debug keeps the existing logger behaviour; with both flags the board is mixed into the debug stream. Without -d, httpx2 request lines stay off. The context-window estimate and the shortened list of outgoing messages are log lines: they appear with -d, with or without -v.

Cut-off replies. A provider that stops a reply at the output limit reports finish_reason length. dragiter does not deliver an incomplete reply: the run ends at once, with exit code 7 (§8, "Exit status") and this error line on stderr:

A critical error occurred: Session 3 of 10: reply cut off at the output limit (out=512/512); raise max_output_tokens

out= is the reported output tokens against the limit sent (max_output_tokens); a figure that is missing shows --. Without max_output_tokens the line says "cut off by the provider's output limit" instead. If the reply is empty and the provider reported reasoning tokens, a WARNING before it says that all output tokens went to reasoning. With -v the request line of the cut-off reply starts with △ and shows cut off out=N/M.

The cut-off reply is not written anywhere, and no later session is sent. Nothing is committed to -o, -O or stdout; the replies of the sessions before it stay in the staging workspace. Raise max_output_tokens and run again; with -O, --resume (§9) adopts the finished sessions and sends only the cut-off one and those after it. The activity log keeps the cut-off session's llm.completed record with finish_reason length.

Each successful completion is also persisted immediately as a shard inside this run's staging workspace (not the final sink), so a later failed call does not drop earlier replies. There is no .dragiter-partial/ directory; the workspace is one of three sink-specific, PID-suffixed directories:

  • .tmp_staging_dir_<PID>/ — inside the -O directory
  • .tmp_staging_file_<PID>/ — beside (never inside) the -o file
  • .tmp_staging_stdout_<PID>/ — inside the user's temp directory, when neither -o nor -O is set (never the current working directory)

See §9 ("Directory output staging") for the full staging, Assembly and commit rules.

Payload dumps stay on DEBUG.

The OpenAI SDK’s own retries are disabled (max_retries=0). The adapter owns the retry loop via CompletionRetryPolicy (src/dragiter/infrastructure/llm/openai_runtime.py).

Retries

max_retry is the number of completion attempts, not the number of extra tries after a failure.

Setting When unset When set
max_retry one attempt max(1, value) attempts
retry_delay default 3 seconds (set by loader) that many seconds as the base, 0 = no wait

Backoff before attempt n (1-based): no wait on attempt 1; then retry_delay, 2 × retry_delay, 4 × retry_delay, …

Retried (until max_retry is exhausted):

  • HTTP 429 (RateLimitError)
  • transport / connection failures (APIConnectionError)
  • transient HTTP 5xx from a proxy or backend (500, 502, 503, and any other 5xx that is not a gateway timeout)

Not retried (the attempt fails immediately):

  • HTTP 504, and responses whose message contains gateway timeout or stream timeout
  • APITimeoutError
  • an Ollama runner crash (model runner has unexpectedly stopped)
  • client 4xx other than 429

Transport and SDK construction failures are reported as “Failed to initialise OpenAI client …”. Failures after the client exists are reported as “Attempt n/m failed: …”. They are not labelled as certificate or CA-bundle problems.


2. Configuration settings (complete list)

All settings that appear in the configuration loader are listed below.

Key (TOML / long CLI) Short CLI Type Default (when unset) Description
debug -d bool false Enable debug logging
simulate -s bool false Simulation mode (no API calls)
verbose -v bool false Verbose output
sequential_processing — bool false --sequential-processing: one request per chunk
resume — bool false --resume: adopt the newest leftover staging workspace under -O DIR before the first completion (requires -O)
tcp_keep_alive — bool false --tcp-keep-alive on the httpx2 transport
api_key — string (none) API key for the LLM service
base_url — string (none) Base URL of the OpenAI-compatible endpoint
model_name — string (none) Model identifier
output_delimiter — string (none) Delimiter written between multiple results
output_filename_schema — string (none) Filename template for output files
output_mode -m string "x" File open mode: x (exclusive), w (overwrite), a (append)
task -t string (none) Direct task text (alternative to a prompt file)
max_context_tokens — int (none) Maximum total tokens the model can handle
max_output_tokens — int (none) Maximum tokens the model may generate; a reply that reaches it ends the run with exit code 7 (§1a)
chars_per_token — float (none)* Average characters per token used for estimation
temperature — float (none) Sampling temperature
retry_delay — int 3 (origin DEFAULT) Base wait between attempts 2…n; doubles each time
max_retry — int 1 attempt if unset Maximum number of completion attempts (not extra retries)
pack_limit_chars — int (none) / off Pack consecutive regex chunks per file up to N characters
max_chunks — int 200 (origin DEFAULT) Maximum chunks per run. Must be ≥ 1
base_directory -b path (unset) Base for relative paths. Not pre-filled with CWD
activity_file -a path (none) Write activity log to this file
ca_bundle_file — path (none) Custom CA certificate bundle (PEM)
client_cert_file — path (none) Client certificate for mTLS
client_key_file — path (none) Client private key for mTLS
config_file -c path (see discovery) Path to TOML configuration file
log_file -L path (none) Write log output to file (append, no rotation)
prompt_file -p path (none) Path to prompt template (.toml)
loop_file -l path (none) Path to loop file (.txt or .jsonl)
resource_file -r path (none) Path to resource definition (.toml)
output_file -o path (none) Write all output to a single file; suppress stdout echo
output_directory -O path (none) Write outputs into this directory; suppress stdout echo

* The example configuration files document chars_per_token = 3.8 as the conventional default used for estimation when the setting is left unset.

The settings debug, verbose, log_file, base_directory, activity_file, simulate and config_file are bootstrap options (section 1). All but config_file may also be set in the TOML file.

Mandatory settings

  • Either a prompt_file (-p) or a direct task (-t) must be supplied, regardless of simulation mode. A task that is set but blank (empty after stripping) is also rejected.
  • base_url must be set (CLI, config file or environment) unless simulation mode (-s) is active — base_url is the only one of these settings gated by simulate.
  • When a loop is used, a resource_file is normally required as well.

Validation rules (from the validator)

  • output_file (-o) and output_directory (-O) are mutually exclusive; setting both is rejected before any other check runs.
  • task (-t) and resource_file (-r) cannot be combined: task mode has no material template, so the material would reach the model as empty messages. Use a prompt file with material instead.
  • output_mode must be one of a, w, x.
  • retry_delay must be between 0 and 20 inclusive (default 3).
  • max_retry (when set) must be between 0 and 9 inclusive. The adapter still performs at least one attempt (max(1, value)), so 0 and “unset” both mean a single try.
  • chars_per_token (when set) must be greater than 0.0.
  • max_context_tokens and max_output_tokens (when set) must be ≥ 1.
  • pack_limit_chars (when set) must be ≥ 0. 0 disables packing.
  • max_chunks must be ≥ 1 (default 200).
  • Paths that are required must be readable (or writable for output paths).
  • The sink parent — the -O directory itself, or the -o file's parent directory — must already exist and be writable. dragiter never creates it.
  • client_key_file without client_cert_file is rejected.
  • base_url is optional in simulation mode.

Hard limits (circuit breakers)

These limits exist to protect against accidental combinatorial explosion, excessive memory consumption and runaway API costs.

Limit Value Component Behaviour on breach
Maximum file size 100 MB SimpleTextFileReader Raises TextFileReaderError and aborts
Maximum total chunks max_chunks, default 200 MaterialTokenizer Raises an error and aborts processing
Maximum loop items 50 LoopBuilder Raises LoopBuilderError and aborts

In addition the MaterialTokenizer emits warnings (but continues) when a final chunk is unusually small: fewer than WARN_MIN_CHARS (50) characters. There is no fixed-size warning for large chunks. Instead, a chunk is flagged "oversize" when it is larger than the effective pack budget (the resolved pack_limit_chars, from CLI/config or a resource section — see §6 and §9's pack / pack from cells); when no pack budget is set, the oversize count is always zero. These warnings help detect poorly chosen regular expressions or an unset pack budget.

The limits are intentional design decisions. When a limit is hit the recommended action is to split the input (smaller files, fewer loop entries, or a tighter chunking regex) and run dragiter multiple times.


3. Environment variables

Every configuration key can be supplied as an environment variable by prefixing it with DRAGITER_ and converting it to upper case:

DRAGITER_API_KEY
DRAGITER_BASE_URL
DRAGITER_MODEL_NAME
DRAGITER_MAX_CONTEXT_TOKENS
DRAGITER_MAX_OUTPUT_TOKENS
DRAGITER_CHARS_PER_TOKEN
DRAGITER_TEMPERATURE
DRAGITER_RETRY_DELAY
DRAGITER_MAX_RETRY
DRAGITER_PACK_LIMIT_CHARS
DRAGITER_MAX_CHUNKS
DRAGITER_BASE_DIRECTORY
DRAGITER_CONFIG_FILE
DRAGITER_PROMPT_FILE
DRAGITER_RESOURCE_FILE
DRAGITER_LOOP_FILE
DRAGITER_OUTPUT_FILE
DRAGITER_OUTPUT_DIRECTORY
DRAGITER_OUTPUT_MODE
DRAGITER_ACTIVITY_FILE
DRAGITER_LOG_FILE
DRAGITER_TCP_KEEP_ALIVE
DRAGITER_SEQUENTIAL_PROCESSING
DRAGITER_OUTPUT_FILENAME_SCHEMA
DRAGITER_OUTPUT_DELIMITER
...

Boolean environment values consumed by ConfigurationLoader are parsed by BoolSetting.from_string after strip and case-fold:

  • truthy: TRUE, 1, YES, ON, Y
  • falsy: FALSE, 0, NO, OFF, N
  • anything else (including the empty string) raises

The bootstrap options -d, -v, -L, -b, -a, -s and -c are read before everything else, with the same precedence as every setting: command line, then configuration file, then environment. They use the same boolean parser as above, with these differences:

  • An empty variable counts as not set, and an unparsable DRAGITER_DEBUG / DRAGITER_VERBOSE / DRAGITER_SIMULATE ends the run with an error naming the variable.
  • A relative base_directory, from any source, is resolved against the current working directory.
  • A relative path (-L, -a, the matching key in the configuration file or the matching variable) is resolved against base_directory, wherever that was set, else against the working directory.
  • The target is checked at once: -b must be a readable directory, and the directory of -L and -a must exist and be writable. If it is not, the run ends with an error naming the option before anything else happens.
  • The flags -d, -v and -s may be combined (-sv is read as -s -v). A group holds these flags only, optionally followed by -L, -b, -a or -c with its value (-sa PATH); give every other option separately (-sv -t TEXT, not -svt TEXT, which some Python versions reject). Long options must be written in full (section 8).

The TOML config-file path does not run that parser. It assigns the native TOML type. Use tcp_keep_alive = true / false, not "true". Quoted boolean strings in a config file are rejected.

Environment variables are applied after the configuration file and only fill values that are still unset.

A value from the configuration file or the environment that starts with $ has every $NAME or ${NAME} replaced by that environment variable before type conversion (origin C$ or E$). A reference to a variable that does not exist ends the run with an error naming the setting and the reference; the text is never used as it is.


4. Main configuration file (TOML)

Example taken from examples/01_md_sample/config-ollama.toml:

api_key = "Ollama"
base_url = "http://localhost:11434/v1"
model_name = "qwen3:8b"
tcp_keep_alive = true

# Working window: input budget is max_context_tokens minus max_output_tokens.
# chars_per_token is an estimate (characters / n, truncated), not a vendor tokenizer.
chars_per_token = 3.8
max_context_tokens = 8192
max_output_tokens = 4096

# Optional advanced settings (uncomment and adjust as needed)
# retry_delay = 5
# max_retry = 3

The same keys appear in config-claude.toml, config-google.toml and config-grok.toml, with different endpoint and window values per provider (see docs/window-starting-values.md for the authoritative starting figures).

All keys are flat (no nested tables are required for the main configuration).

The bootstrap options debug, verbose, log_file, base_directory, activity_file and simulate may appear in the file; the command line still wins over them, and they win over the environment. config_file must not appear: a file that contains it is rejected with an error naming the key. A file that cannot be read or is not valid TOML ends the run before anything else happens, with an error naming the file.


5. Prompt file format (TOML)

Observed structure from examples/01_md_sample/01_prompt_md.toml and the PromptTemplate domain model:

[system]
instruction = """
You are a precise analyst.
Always answer in English and use Markdown tables.
"""

[task]
first = """
### REFERENCE MATERIAL
Use the following material as a knowledge base:
"""

material = """
ID: {CHUNK_NUM_ID:04d}
FILE: {CHUNK_FILE_NAME}
SECTION: {CHUNK_SECTION_NAME}
SECTION NUM ID: {CHUNK_SECTION_NUM_ID}
CONTENT:

{CHUNK_CONTENT}
"""

synthesis = """
As a precise analyst, evaluate the provided historical reference material
to answer the following core question:

**"{LOOP_CONTENT}"**

Extract the key arguments from the text and present your findings
strictly as a Markdown table with the columns:
"Historical Concept", "Core Argument", and "Meaning for Modern AI".
"""

[behaviour]
temperature = 0.0
sequential_processing = false

[outcome]
output_delimiter = "\n---\n"
output_filename_schema = "sample_01.txt"

Required and optional keys

Table Key Required When absent
[system] instruction no No system message is sent
[task] first no No introductory user message is sent
[task] material with -r With -r: the run ends with exit code 3; without -r: not needed
[task] synthesis yes The run ends with exit code 3
[behaviour] temperature, sequential_processing no Hard-coded default (see below)
[outcome] output_delimiter, output_filename_schema no Hard-coded default (see below)

The table [system] may be left out as a whole. A key that is present keeps its value, also the empty string: first = "" sends an empty message. Any other key in [system] or [task] (for example a misspelt frist) ends the run with exit code 3. The message names the file and every missing or unknown key.

With a resource file, a material without {CHUNK_CONTENT} is allowed - the CHUNK_* placeholders may serve only to name output files (section 9) - but it produces a warning that the contents of the material files are not sent.

Placeholders recognised in templates

  • {CHUNK_NUM_ID} / {CHUNK_NUM_ID:04d}
  • {CHUNK_FILE_NAME} - the absolute, resolved path of the source file. It is sent to the provider as it is, so local directory names leave your machine.
  • {CHUNK_SECTION_NAME}
  • {CHUNK_SECTION_NUM_ID}
  • {CHUNK_CONTENT}
  • {LOOP_CONTENT}
  • {LOOP_NUM_ID} (and other keys that may appear in a loop dictionary)
  • {STDIN} (replaced by content read from standard input, if any). Standard input is read only when this placeholder appears in task.first, or when -t / --task is used (stdin then becomes the first field). Templates without the placeholder do not touch stdin.

Defaults applied by PromptCreator

The example values shown in the sample prompt.toml above are not the code defaults.
They are merely the values chosen for that particular demonstration file.

When a prompt file is loaded, the PromptCreator applies the following hard-coded defaults for any key that is missing from both the main configuration system and the prompt template itself:

Key Hard-coded default
temperature 0.0
sequential_processing false
output_delimiter "\n"
output_filename_schema "dragiter-out.txt"

These form the lowest priority layer (see section 1).
Consequently a prompt template that omits [behaviour] or [outcome] entirely still receives a fully populated PromptTemplate object.

The PromptTemplate object stores:

  • instruction
  • first
  • material
  • synthesis
  • temperature
  • sequential_processing
  • output_filename_schema
  • output_delimiter

6. Resource file format (TOML)

Observed structure from examples/01_md_sample/01_resource_md.toml:

[config01]
glob_patterns = ["**/*_engine.md", "**/*_thought.md"]
regex_patterns = ['(^#+\s+.*$)']

[config02]
glob_patterns = ["**/*_automata.txt"]
regex_patterns = ['(^\d+\.\s+.*$)']

Each table name becomes a section name.
Inside a section the following keys are used:

  • glob_patterns - list of glob patterns relative to the section search root
  • base_directory (optional, per section) - search root for those globs:
  • absolute: used as it is; -b does not move it.
  • relative: taken relative to -b when that is set, otherwise relative to the current working directory.
  • omitted: -b when that is set, otherwise the current working directory.

The global BaseDirectoryPathSetting itself is never pre-filled with CWD. - regex_patterns - list of regular expressions used to split the matched files into chunks. The first pattern always cuts at match starts (the match text stays on the following piece; capturing groups are ignored). Later patterns are applied only to pieces that still exceed pack_limit_chars. Default when omitted: a pattern that matches nothing. The singular key regex_pattern is no longer accepted; a section that still sets it aborts collection and names the section. - Optional filters (supported by the ResourceSection model): exclude_filters, include_filters - chunk_substitutions (optional, per section) - ordered list of tables { pattern, replacement }. After the staged split, each piece is rewritten with re.sub in list order. The replacement is literal: \1 and \g<name> are ordinary characters, not group references. Flags are re.MULTILINE only (not IGNORECASE). An empty replacement deletes the match. A piece that is empty or only whitespace afterwards is discarded. Include/exclude filters and pack_limit_chars then see the rewritten text. The first split still runs on the raw file. A missing pattern or replacement, a non-list value, or a non-table entry aborts collection and names the section. An invalid pattern fails tokenisation and names the section and index. There is no CLI or environment override. - pack_limit_chars (optional, per section) - after the staged regex split and any chunk_substitutions, join consecutive chunks of the same file until this many characters would be exceeded. Measured in characters. 0 or omitted means no packing and no overflow patterns after the first. A globally set pack_limit_chars (CLI --pack-limit-chars, config file or DRAGITER_PACK_LIMIT_CHARS) overrides the section value, including when the global value is 0. Packing never crosses a file or section boundary. Only runs of chunks that share the same valid flag are joined. A piece that remains larger than the budget after every overflow pattern is left intact.

Resolved matches that escape the section root (including symlinks that point outside) are skipped.

A matched file that cannot be read as text - binary (it contains null bytes), empty, gone or not readable - is skipped as well, so that a broad pattern such as docs/**/* need not exclude every image by hand. Each skipped file is reported as a warning on stderr (shown without -d) naming the section, the file and the reason, and the activity log lists it under skipped_files in the Resources record (§10). If every file the resource file matches is skipped, the run ends with exit code 3 and names them, rather than sending a request without material.

Multiple sections may be defined; they are processed independently.


7. Loop file formats

Plain text (.txt)

One task per line. Example from 01_loop_md.txt:

What exactly separates genuine human reasoning from mere mechanical calculation?
What fundamental limitations of artificial machines are identified here?
How is the relationship between symbols, language, and true intelligence described?

Each non-empty line becomes a loop item with the key LOOP_CONTENT.

JSON Lines (.jsonl)

Supported by the loop builder (one JSON object per line).
Any keys present in the objects are available as placeholders in the prompt templates.


8. CLI reference (main flags)

Long options must be written in full, for every option: --temperature, not --temp; --verbose, not --verb. An abbreviated long option is rejected with an error naming it (exit status 2).

dragiter [options]

  -d, --debug
  -s, --simulate
  -v, --verbose
  -c, --config-file PATH
  -p, --prompt-file PATH
  -r, --resource-file PATH
  -l, --loop-file PATH
  -o, --output-file PATH
  -O, --output-directory PATH
  -b, --base-directory PATH
  -a, --activity-file PATH
  -L, --log-file PATH
  -m, --output-mode {x|w|a}
  -t, --task TEXT
  --api-key TEXT
  --base-url URL
  --model-name NAME
  --max-context-tokens INT
  --max-output-tokens INT
  --chars-per-token FLOAT
  --temperature FLOAT
  --retry-delay INT
  --max-retry INT
  --pack-limit-chars INT
  --max-chunks INT
  --ca-bundle-file PATH
  --client-cert-file PATH
  --client-key-file PATH
  --tcp-keep-alive
  --sequential-processing
  --resume
  --output-delimiter TEXT
  --output-filename-schema TEXT
  -h, --help
  --info
  --version

-h/--help prints argparse's own short usage summary; --info prints the full man-page-style information page. --info is also recognised as -info or /info (matched case-insensitively, anywhere in the argument list). Running dragiter with no arguments at all prints the banner and the usage summary, then exits (cli.py's pre-selector in main(), checked before configuration loading starts).

-h/--help and --version do not locate or read the configuration file, so they work even when it is missing or not valid TOML.

Additional entry points:

  • dragiter-gen-examples - extract the example trees
  • dragiter-gen-docs - extract the documentation

Exit status

The exit status tells why a run ended, one code per cause. Every error ends the run at once, so a run has exactly one code.

Code Meaning
0 Success
1 Any other error: staging workspace, output commit, activity file that cannot be written during the run (§10.9), internal error
2 Usage error on the command line: an unknown or abbreviated option, an option without its value
3 Configuration or input not usable: an invalid setting or configuration file, a missing or unreadable input file, standard input, the TLS set-up of the client
4 No material: no chat session, because no chunk is valid (sequential processing)
5 Context window exceeded. A live run ends before anything is sent; a simulate run (-s) completes, writes its output and still ends with 5, the code the live run would end with
6 Provider error, after the retries (§1a, "Retries")
7 A reply was cut off at the output limit (§1a)
130 Interrupted by the user (Ctrl+C)

Scripts that test for "not 0" keep working; a test for exactly 1 no longer matches every error. --info, dragiter-gen-docs and dragiter-gen-examples end with 0 or 1. The activity log records the same code in run.finished (§10.3).


9. Output behaviour

Stdout

Stdout is the default result sink. Once output_file (-o) or output_directory (-O) is set from any configuration source, file routing replaces that sink: live replies and the simulate run board are written only to the requested file(s).

Redirection of stdout (>, >>, |) is not inspected. isatty() cannot distinguish a user pipe from CI or test capture, and treating a redirected stream as an extra sink would duplicate output the caller already routed to a file. To keep a stream as well as a file, omit -o/-O and redirect, or read the written file.

Filename generation

When an output_filename_schema is supplied, the following placeholders are substituted (see format_output_filename in src/dragiter/infrastructure/io/filename_utils.py):

  • Chunk-related: CHUNK_NUM_ID, CHUNK_FILE_NAME, CHUNK_SECTION_NAME, CHUNK_SECTION_NUM_ID. CHUNK_FILE_NAME is the absolute path of the source file; in a filename its path separators become _, so the name carries the whole path, cut to 200 characters.
  • Loop-related: any key present in the current loop dictionary (especially LOOP_NUM_ID, LOOP_ID, LOOP_CONTENT)
  • TIMESTAMP - formatted YYYYMMDD_HHMMSS_nnnnnnnnn (format_timestamp_ns). This is not generated by an internal clock: it is derived from the real filesystem st_mtime_ns of the shard being committed, read at commit time. Using {TIMESTAMP} in an output_mode x schema defers the exclusive-create name check from before the first completion to the commit itself (a notice is printed to stderr).

If a schema contains none of CHUNK_, LOOP_ or TIMESTAMP, it is discarded outright and the fallback name is used instead: session_<NNNN>.md (zero-padded to four digits) when a session index is available, otherwise output.md. The same fallback also applies if formatting the schema against the known placeholders fails.

Filenames are sanitised before being written.

File open modes

Controlled by output_mode:

Value Behaviour
x Exclusive create (default). Fails if the file already exists.
w Overwrite
a Append

Delimiter

When multiple results are written to the same file, the value of output_delimiter is inserted between them.

Directory output staging

-o and -O are mutually exclusive. This is validated twice: once by the configuration validator (rejected outright before any path or completion is touched), and again when the run's workspace is located (WorkspaceLayout.locate) — so no workspace can ever be derived from an invalid combination.

Every run writes to exactly one hidden, PID-suffixed staging workspace before anything is committed to its real sink (src/dragiter/infrastructure/io/workspace_service.py):

Sink Workspace Location
-O (directory) .tmp_staging_dir_<PID>/ inside the target -O directory
-o (file) .tmp_staging_file_<PID>/ beside (never inside) the target -o file
neither (stdout) .tmp_staging_stdout_<PID>/ inside the user's temp directory ($TMPDIR if set, else the platform default) — never the current working directory

Each sink's parent (the -O directory, or the -o file's parent) must already exist and be writable; dragiter never creates it. Each completion is persisted into the workspace as its own numbered shard (res000001, res000002, …) as soon as it arrives, so a later failed call does not drop earlier replies.

Only after every session has completed does dragiter commit:

  • -O: each shard is renamed to its own final filename (from output_filename_schema), one file per session. Under output_mode x, any existing target aborts the whole commit before the first rename.
  • -o and stdout-only: this is an Assembly step, not a plain concatenation. Non-empty shards are joined, in sequence, with the real output_delimiter; empty shards are dropped entirely; and if every shard is empty, the target file (or stdout) is not created or modified at all. The assembled result is then written under the chosen output_mode (x/w/a).

If a write conflict occurs (such as an existing file under exclusive mode -m x), the commit aborts to prevent data corruption, leaving the shards intact inside the staging workspace for recovery. The workspace is removed only after a successful commit.

Resume

--resume picks up exactly the staging workspace a previous, aborted -O run left behind, instead of paying for every session again. It requires -O; combined with -o or with neither, the run is rejected before anything is touched.

Right after the run's own, freshly created workspace exists (same PID-suffixed naming as always — see the table above), dragiter looks for other .tmp_staging_dir_* directories directly inside the target -O directory. Finding none is not an error: the run simply proceeds as if --resume had not been given. Finding one or more, it picks the single most recently modified one, moves every one of its shards into the new workspace (same filesystem, so a plain rename — no shard is read or rewritten), and removes the now-empty source directory. Any other leftover workspace is left untouched, the same as without --resume. A line naming the source and the number of shards adopted is always printed to stderr, independent of -v.

Once a shard already exists for a given session, that session's request is never sent — neither to the live endpoint nor to the mock service — and the session is reported as reused instead of generating a new reply. Every other session runs exactly as in an ordinary run. The final commit does not distinguish an adopted shard from a freshly persisted one. An adopted shard never holds a cut-off reply: a run that ends with a cut-off reply does not persist it (§1a), so --resume sends that session again.

--resume trusts the shard sequence positionally: it assumes the material, loop file, prompt and --sequential-processing are unchanged from the aborted run, so that a shard's position still means the same session. There is no manifest or fingerprint checking this — changing those inputs between the aborted run and the resumed one will silently attribute an old reply to the wrong session.

Simulate boards

ChatManager builds and persists simulate content as ordinary shards, the same way it persists live replies, so the commit path (§9 "Directory output staging") does not distinguish stdout from -o. Consequently, when neither -o nor -O is set, stdout now shows the same content -o would have written to a file: the four-column run board once, followed by one session board plus that session's complete outgoing request, for every session — not the run board alone. -O files each start with a session board followed by that session's request; no aggregate run board is written under -O.

The mock service answers every session with its request as a role/content table; the result presenter chosen for a simulate run (BoardResultPresenter) puts the session board before it and, for -o and stdout-only, the run board before all sessions. A live run uses the plain presenter: no boards.

Within a single shard, *** (a Markdown thematic break) still separates the session board from the request. Between shards — i.e. between the run board and the first session, and between sessions — Assembly joins with the real, configured output_delimiter, not a hardcoded separator.

sessions is the number of chat requests. Sequential mode multiplies valid chunks by loop lines (or by one when no loop file is present).

pack is the effective character budget. pack from is cli when pack_limit_chars is set globally (including 0), section when a single positive section budget applies, mixed when section budgets differ, and off when no budget is set. small / over uses that same effective budget. The -v board shows the same counts at the end of its second line (small N over M), with over -- when the budget is off or mixed.

window, peak / limit and peak at / warns stay -- / n/a until chars_per_token, max_context_tokens and max_output_tokens are all set. window no means at least one session exceeds the context window; warns is the number of such sessions. A simulate run completes and shows them; a live run with the same settings is refused before the first request, with an error that lists every exceeding session with its estimated input, the reserved output and the limit. replies / stdout is always the result count plus the fixed word brief — that cell describes the run board itself, not what else is committed to the sink; per the stdout-only behaviour above, the full session boards and requests still follow the run board on stdout.

The session board lists session index, file, chunk, section, valid (yes / no), chars, tokens, pack, pack from, loop index and loop count.

chars on a session board is the character length of that session's chunk (or of all valid chunks when batched). tokens is not chars / chars_per_token. It is the estimator's calculated total for the request: estimated input tokens of every message in the session (system, material framing, chunk text and synthesis) plus the reserved max_output_tokens. The same total feeds peak / limit on the run board. Estimated input alone is written to the activity file as estimated_input_tokens.


10. Activity log format

When the setting activity_file (CLI -a / configuration file / environment DRAGITER_ACTIVITY_FILE) is set, dragiter writes a trace of the run. The log states what the provider reported and what dragiter sent or did, and nothing in between: it does not fill in, default or interpret a value.

-a is a bootstrap option (section 3). It is resolved and checked before anything else happens: if its directory is missing or not writable, the run ends at once with an error naming -a, and no file is created. An error in the configuration file itself (unreadable, or not valid TOML) is also reported on stderr before the activity file is created, so no file is created then either. Without -a, no file is created and nothing else changes.

10.1 File format

The activity file is a JSON Lines (JSONL) file.

  • Encoding: UTF-8, one JSON object per line.
  • Records are appended and flushed as they occur: another process reading the file sees every record produced so far, also while the run is in progress and after it ended abruptly. A run that fails or is interrupted keeps what it wrote.
  • The file is never truncated. Several runs may append to the same file; their records are told apart by run_id.
  • Characters that act as line breaks in some readers (U+0085, U+2028, U+2029) are written as \uXXXX escapes, so a record never spans two lines. All other Unicode stays readable.

Every record has the same envelope. Payload keys never appear at the top level.

Key Type Meaning
schema integer Record schema version; 1
run_id string Random identifier of the run (32 lower-case hex digits), the same on every record of the run
seq integer Position within the run, from 1, without gaps
ts string UTC time with milliseconds, e.g. 2026-10-04T11:02:55.877Z
event string The kind of record, section 10.2
source string The component that produced it; for result, the class name of the stored result
session_index integer Only on records of one session; 1-based, as on the session board
data object The payload

10.2 Event catalogue

event Written by When
run.started ApplicationManager Always the first record of a run
result ApplicationManager A pipeline result that carries data for the log is stored (10.6)
worker.finished ApplicationManager A worker returned or raised
llm.completed ChatManager One per session, as soon as it is completed
output.committed OutputWriter One per output target, after the commit wrote to it
run.failed ApplicationManager The run ends with an error
run.finished ApplicationManager Always the last record of a run

10.3 Lifecycle records

run.started

{"schema": 1, "run_id": "257adf1359be4cb1acbe14cc807a574a", "seq": 1, "ts": "2026-10-04T11:02:55.877Z",
 "event": "run.started", "source": "ApplicationManager",
 "data": {"tool": "dragiter", "version": "2026.10.22", "python": "3.13.5", "pid": 99165}}

worker.finished - one per pipeline worker, with its duration and ok or failed:

{"schema": 1, "run_id": "257adf…", "seq": 2, "ts": "2026-10-04T11:02:55.878Z",
 "event": "worker.finished", "source": "ApplicationManager",
 "data": {"worker": "ConfigurationLoader", "duration_ms": 0, "outcome": "ok"}}

run.failed - the original error, not the wrapper that ApplicationManager raises:

data key Meaning
type, message, module The exception class name, its message and its module
filename, lineno, location The last frame of the traceback (the source line); null without a traceback
traceback The full text, including the chain of causes (raise ... from ...)

run.finished - status and exit_code of the process:

status exit_code When
success 0 The run completed; also --help and --version
interrupted 130 Ctrl+C; no run.failed
failure 1 to 7 The code of the process (§8, "Exit status"). An error of the run comes after run.failed; a command-line usage error (2) has run.finished only; a simulate run that completes with an exceeded context window (5) has no run.failed
{"schema": 1, "run_id": "257adf…", "seq": 32, "ts": "2026-10-04T11:02:55.883Z",
 "event": "run.finished", "source": "ApplicationManager",
 "data": {"status": "success", "exit_code": 0,
          "summary": {"sessions_total": 2, "sessions_completed": 0, "sessions_resumed": 0,
                      "sessions_simulated": 2, "finish_reasons": {}, "finish_reason_unreported": 0,
                      "prompt_tokens": 0, "prompt_tokens_unreported": 0,
                      "completion_tokens": 0, "completion_tokens_unreported": 0,
                      "reasoning_tokens": 0, "reasoning_tokens_unreported": 0}}}

The summary counts what the run's llm.completed records state; on a failure or interruption it covers the sessions completed until then.

  • sessions_total is the number of records, sessions_resumed those adopted by --resume, sessions_simulated those answered by the simulation, and sessions_completed the rest: sessions a provider answered in this run.
  • finish_reasons counts every finish_reason value that was reported; finish_reason_unreported counts the sessions whose response contained none.
  • The token sums cover the sessions that reported them; …_unreported counts the sessions that did not.
  • Resumed and simulated sessions are not part of finish_reasons and the token figures, so no figure mixes what a provider reported with the mock service's own figures or with "nobody was asked".

10.4 The llm.completed record

One record per session, with session_index, written as soon as the session is complete. A run that fails at session k still shows sessions 1 to k - 1.

data key Meaning
resumed, simulated The session was adopted by --resume, or answered by the mock service of a simulation; otherwise false
model_requested The configured model_name (null if none is configured, as in a simulation)
model_reported, response_id, system_fingerprint As the response reported them
attempts, attempt_errors Requests made (1 = no retry; 0 = none made) and one short description per failed attempt
temperature_sent, max_tokens_requested What was put in the request
started_at, first_token_at, ended_at UTC times of the successful attempt; first_token_at is null if no content arrived
ttft_ms, duration_ms, elapsed_ms Time to first content; duration of the successful attempt; time from the first attempt's start, waits included
prompt_tokens, completion_tokens, reasoning_tokens The provider's usage figures; in a simulation the mock's estimate of the request and 0
prompt_tokens_estimated, prompt_tokens_ratio dragiter's own estimate of the input, and prompt_tokens / prompt_tokens_estimated (3 decimals)
finish_reason As reported, unchanged; mock in a simulation
output_chars, output_sha256 Length and SHA-256 of the answer handed to the output sink (section 10.7)
request_sha256 SHA-256 of the session's messages (section 10.7)
reasoning_chars, reasoning_sha256 Length and SHA-256 of reasoning text the provider returned in separate fields

What null stands for. For a session that a provider answered, a key describing the response is null if, and only if, the response did not contain it; temperature_sent and max_tokens_requested are null if dragiter did not send them. For a simulated session the mock service answered, and its record carries what the mock reported: prompt_tokens its estimate of the request (null when chars_per_token is not set), completion_tokens 0, finish_reason mock, and measured started_at, ended_at, duration_ms and elapsed_ms. model_reported, response_id, system_fingerprint, first_token_at, ttft_ms, reasoning_tokens and the reasoning keys are null, and attempts is 0: no request was made. output_chars and output_sha256 describe the persisted content, session board and request table. For a resumed session nothing answered in this run: resumed is true, and the keys describing the response and its timing, output_chars and output_sha256 are null for that reason alone.

What the log does not say. There is no verdict: the log does not say whether an answer was cut off or an input was shortened. (A reply reported as length ends the run with exit code 7, §1a.) finish_reason is the provider's own word, and providers use it differently - length, for example, can mean a reached output limit or a full context window. Whether a low prompt_tokens_ratio means that a server shortened the input, or only that chars_per_token is inaccurate, cannot be told from the log.

{"schema": 1, "run_id": "257adf…", "seq": 25, "ts": "2026-10-04T11:03:01.512Z",
 "event": "llm.completed", "source": "ChatManager", "session_index": 1,
 "data": {"resumed": false, "simulated": false, "model_requested": "qwen3:8b", "model_reported": "qwen3:8b",
          "response_id": "chatcmpl-42", "system_fingerprint": null, "attempts": 2,
          "attempt_errors": ["HTTP 503: overloaded"], "temperature_sent": 0.0, "max_tokens_requested": 4000,
          "started_at": "2026-10-04T11:02:58.100Z", "first_token_at": "2026-10-04T11:02:58.640Z",
          "ended_at": "2026-10-04T11:03:01.511Z", "ttft_ms": 540, "duration_ms": 3411, "elapsed_ms": 6420,
          "prompt_tokens": 4150, "completion_tokens": 630, "reasoning_tokens": null,
          "prompt_tokens_estimated": 4310, "prompt_tokens_ratio": 0.963, "finish_reason": "stop",
          "output_chars": 612, "output_sha256": "2c70…", "request_sha256": "d7a8…",
          "reasoning_chars": null, "reasoning_sha256": null}}

10.5 The output.committed record

One record per output target, after the commit wrote to it, without session_index: one per file for -O, one for -o, one for stdout. A target the commit did not write to (all answers empty; -m a with nothing to append) has no record. If the commit fails after some targets, those targets are still reported.

data key Meaning
target file or stdout
path Absolute path of the file; null for stdout
bytes, sha256 Size and SHA-256 of exactly what this run wrote to the target
session_indices The sessions whose answers the target received (the simulation board is not a session)

For a file the run created or overwrote, bytes and sha256 describe the whole file and can be checked with wc -c and sha256sum. With -m a on an existing file they describe the appended block, delimiter included. For stdout they describe the UTF-8 text written there.

10.6 Result records

Whenever a pipeline result that carries data for the log is stored, one result record per returned object is written; source is the result's class name. None of them contains the text of a message or an answer.

Settings (AIServiceParameters, LoggingParameters, ExecutionParameters, InputParameters, OutputParameters, WorkspaceParameters): every setting is an object with its value, where it came from and whether it was set at all.

origin Source
A Command line
C Configuration file
C$ Configuration file, $VAR expanded
E Environment variable
E$ Environment variable, $VAR expanded
D Built-in default
null Never set (is_set is false)

Settings of a secret type - the API key - have the value ***MASKED***; origin and is_set are shown. Masking is decided by the type of the setting, not by the name of its key. File paths, including client_key_file, are shown.

{"event": "result", "source": "AIServiceParameters", "data": {
   "api_key":    {"value": "***MASKED***", "origin": "A", "is_set": true},
   "base_url":   {"value": "http://localhost:11434/v1", "origin": "C", "is_set": true},
   "model_name": {"value": "qwen3:8b", "origin": "E", "is_set": true},
   "temperature": {"value": null, "origin": null, "is_set": false}}}

Resources, Material, Loop, PromptTemplate and ContextValidationReport carry counts, sizes, names and estimates, for example (the envelope is left out):

{"event": "result", "source": "Material", "data": {"material_chunks_count": 12, "material_valid_chunks_count": 12,
   "total_characters": 45821, "unique_sources": 3, "average_chunk_size": 3818.4, "max_chunk_size": 9200,
   "min_chunk_size": 412}}
{"event": "result", "source": "Loop", "data": {"loop_items_count": 3, "has_json_structure": false,
   "average_keys_per_item": 1.0, "total_keys_across_all_items": 3}}

Material is one summary record followed by one record per source file. Resources lists, per section, the globs, patterns, filters and chunk substitutions of the resource file; if matched files were skipped (§6), it adds skipped_files, one entry per file with section, path and reason. ContextValidationReport is a summary (with exceeded_sessions, the 1-based numbers of the sessions whose estimate exceeds the context window) followed by one record per session. ChatResults sums the token figures of all results; in a simulation total_input_tokens holds the mock's estimates.

ChatSessions is a summary, then one record per session, with session_index:

{"event": "result", "source": "ChatSessions", "data": {"chat_sessions_count": 2, "total_input_messages": 8,
   "sessions_with_chunk": 2, "sessions_with_loop_item": 2}}
{"event": "result", "source": "ChatSessions", "session_index": 1,
 "data": {"message_count": 4, "chars": 186, "request_sha256": "d7a8…"}}

ChatResults is a summary only (chat_results_count, token and duration totals); what happened per session is in llm.completed.

10.7 Checksums and content

No record contains the text of a message or an answer. Checksums link the log to what you can see elsewhere.

  • request_sha256 is the SHA-256 over the session's messages, each reduced to role and content, in order (a canonical JSON form, so equal input gives an equal value in every run). Two runs with the same input show the same request_sha256 per session.
  • output_sha256 is the plain SHA-256 of the UTF-8 answer handed to the output sink. For -O, the output file of a session holds exactly that content, so sha256sum of the file matches. For -o and stdout the answers are joined with the delimiter; compare with the sha256 of output.committed instead.

10.8 Order of records

A successful run writes roughly this order (workers run in a fixed order; a result is written when its worker has returned):

  1. run.started
  2. worker.finished for ConfigurationLoader, then the six settings records
  3. ConfigurationValidator, ResourceCollector (+ Resources), MaterialTokenizer (+ Material), LoopBuilder (+ Loop), PromptCreator (+ PromptTemplate), MessageBuilder (+ ChatSessions), ContextWindowEstimator (+ ContextValidationReport), each with its worker.finished
  4. One llm.completed per session, as the sessions complete
  5. worker.finished for ChatManager, then ChatResults
  6. One output.committed per target, worker.finished for OutputWriter
  7. run.finished

A failing run ends with run.failed and run.finished; an interrupted one with run.finished only. The exact content depends on the path taken (simulation, loop file, output sink).

10.9 If the log cannot be written

If writing fails during the run - the disk is full, the file became read-only - the run ends: without an error of its own, it stops at the first record that cannot be written, with exit code 1 and an error naming the activity file and the cause. Outputs committed before that stay in place. A run that already failed keeps its own error and exit status; the logging failure is reported once on stderr, and the log is incomplete from there on.


11. Built-in defaults summary

Main configuration system

Setting Applied default
base_directory Unset. Not pre-filled with Path.cwd()
output_mode "x" (factory default in ConfigurationLoader)
Boolean flags (debug, simulate, verbose, sequential_processing, tcp_keep_alive) Unset until supplied; treated as false when read as .value on an unset flag
All other settings Remain unset until supplied by CLI, config file or environment

Token-related values (chars_per_token, max_context_tokens, max_output_tokens) have no hard-coded numeric default inside the validator; the example configuration files document the conventional values used for estimation (chars_per_token = 3.8).

PromptCreator hard-coded defaults

These defaults are applied by the PromptCreator when a prompt file is loaded and the corresponding key has not been supplied by a higher-priority source (CLI, main config, environment, or the prompt template itself). See also section 1 and section 5.

Setting Hard-coded default
temperature 0.0
sequential_processing false
output_delimiter "\n"
output_filename_schema "dragiter-out.txt"

Configuration

  • Configuration loading: src/dragiter/application/config/configuration_loader.py
  • Validation & defaults: src/dragiter/application/config/configuration_validator.py
  • Settings classes: src/dragiter/domain/models/settings.py
  • Parameter groups: src/dragiter/domain/models/parameters.py
  • Streaming LLM adapter: src/dragiter/infrastructure/llm/openai_service_ext.py
  • Retry policy and HTTP transport factory: src/dragiter/infrastructure/llm/openai_runtime.py

File formats

  • Prompt model: src/dragiter/domain/models/prompt_template.py
  • Resource model: src/dragiter/domain/models/resources.py
  • Output writing (commit trigger): src/dragiter/application/pipeline/output_writer.py
  • Run workspace, staging, Assembly and commit: src/dragiter/infrastructure/io/workspace_service.py
  • Filename schema formatting and sanitisation: src/dragiter/infrastructure/io/filename_utils.py
  • Simulate result presenter (run board, session board): src/dragiter/infrastructure/cli/board_result_presenter.py
  • Simulate board rendering helpers: src/dragiter/infrastructure/cli/simulation_brief.py
  • Mock service and its request table: src/dragiter/infrastructure/llm/mockai_service.py, src/dragiter/infrastructure/llm/payload_table.py
  • -v board: src/dragiter/infrastructure/cli/text_session_board.py

Activity log

  • ActivityProvider protocol: src/dragiter/domain/common/activity_provider.py
  • ActivityLogger protocol: src/dragiter/domain/ports/activity_logger.py
  • Buffered logger: src/dragiter/infrastructure/logging/buffered_activity_logger.py
  • File logger: src/dragiter/infrastructure/logging/file_activity_logger.py
  • JSONL writer: src/dragiter/infrastructure/io/io_services.py → append_jsonl_to_file
  • Value-settings masking: src/dragiter/domain/models/value_settings_activity_provider.py
  • Domain producers: src/dragiter/domain/models/{resources,material,loop,prompt_template,chat_sessions,chat_results,application_result}.py

Examples

  • Example files: examples/01_md_sample/

End of technical reference