Observability API

queryLogs

GET /api/v2/logs

Requires: invoke role, worker role

Read log lines of one execution (traverser_id) or run (root_id) after a sequence cursor, filtered by levels (comma-separated), stream, since/until (logged_at) and a literal contains substring (at most 256 characters). Same read authority as usage. next_cursor is the last sequence returned, or after when the page is empty.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.query_logs()

Parameters

Responses

Example response 200

{
  "items": [
    {
      "attempt": 0,
      "created_at": 0,
      "fields": {
        "key": {}
      },
      "level": "trace",
      "logged_at": 0,
      "message": "string",
      "root_id": "string",
      "sequence": 0,
      "source": "string",
      "stream": "stdout",
      "traverser_id": "string",
      "truncated": false
    }
  ],
  "next_cursor": 0,
  "root_id": "string",
  "traverser_id": "string"
}

appendLogs

POST /api/v2/logs

Requires: worker role

Ship bounded node log lines for the executing attempt (worker lease or admin traverser_id, as for usage). Ingestion is opt-in per namespace: 409 node_logs_disabled until an admin enables it through observability settings. Messages are cut to the namespace byte bound on a character boundary and counted in truncated; sequences are gap-free per namespace. Lines older than the retention period are pruned opportunistically.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.append_logs()

Request body

LogAppendRequest (required)

{
  "entries": [
    {
      "message": "string"
    }
  ],
  "entry": {
    "message": "string"
  },
  "lease_token": "string",
  "step_id": "string",
  "task_id": "string",
  "traverser_id": "string"
}

Responses

Example response 200

{
  "accepted": 0,
  "attempt": 0,
  "first_sequence": 0,
  "last_sequence": 0,
  "max_message_bytes": 0,
  "root_id": "string",
  "traverser_id": "string",
  "truncated": 0
}

modelHealth

GET /api/v2/model-health

Requires: invoke role

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.model_health()

Responses

Example response 200

{
  "enabled": false,
  "endpoint_count": 0,
  "endpoints": [
    {}
  ],
  "evaluated_at": 0,
  "next_check_at": 0,
  "policy": {},
  "running": false,
  "status": "string"
}

getObservabilitySettings

GET /api/v2/observability/settings

Requires: admin role

Effective namespace node-log policy after applying the process configuration and clamps, the namespace's explicit overrides, and the process limits.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.get_observability_settings()

Responses

Example response 200

{
  "limits": {
    "node_logs_max_entries_per_call": 0,
    "node_logs_max_message_bytes": 0,
    "node_logs_retention_days": 0,
    "usage_max_entries_per_call": 0,
    "webhook_max_attempts": 0
  },
  "namespace": "string",
  "node_logs": {
    "configured": {
      "enabled": false,
      "max_message_bytes": 0
    },
    "enabled": false,
    "max_message_bytes": 0
  }
}

updateObservabilitySettings

PATCH /api/v2/observability/settings

Requires: admin role

Set the namespace node-log opt-in and message byte bound (256 up to the process limit). Unknown node_logs keys are rejected with 400 invalid_settings.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.update_observability_settings()

Request body

ObservabilitySettingsUpdate (required)

{
  "node_logs": {
    "enabled": false,
    "max_message_bytes": 0
  }
}

Responses

Example response 200

{
  "limits": {
    "node_logs_max_entries_per_call": 0,
    "node_logs_max_message_bytes": 0,
    "node_logs_retention_days": 0,
    "usage_max_entries_per_call": 0,
    "webhook_max_attempts": 0
  },
  "namespace": "string",
  "node_logs": {
    "configured": {
      "enabled": false,
      "max_message_bytes": 0
    },
    "enabled": false,
    "max_message_bytes": 0
  }
}

getObservabilityWebhook

GET /api/v2/observability/webhook

Requires: admin role

The namespace webhook configuration, or {configured:false}. The secret is never returned; secret_present reports whether the named environment variable is set on the daemon.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.get_observability_webhook()

Responses

Example response 200

{
  "configured": false,
  "created_at": 0,
  "enabled": false,
  "event_types": [
    "usage.recorded"
  ],
  "headers": {
    "key": "string"
  },
  "secret_env": "string",
  "secret_present": false,
  "updated_at": 0,
  "updated_by": "string",
  "url": "string"
}

putObservabilityWebhook

PUT /api/v2/observability/webhook

Requires: admin role

Create or replace the single namespace webhook. url must be absolute http(s) without credentials or fragment; loopback and private-network targets are refused with 400 webhook_target_forbidden unless the daemon policy allows them. secret_env names the environment variable holding the HMAC-SHA256 secret (a plain secret field is rejected). Deliveries carry x-atfgraph-event, x-atfgraph-delivery, x-atfgraph-attempt, x-atfgraph-timestamp and, when a secret is present, x-atfgraph-signature `sha256=<hex>` over the body.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.put_observability_webhook()

Request body

ObservabilityWebhookPut (required)

{
  "url": "string"
}

Responses

Example response 200

{
  "configured": false,
  "created_at": 0,
  "enabled": false,
  "event_types": [
    "usage.recorded"
  ],
  "headers": {
    "key": "string"
  },
  "secret_env": "string",
  "secret_present": false,
  "updated_at": 0,
  "updated_by": "string",
  "url": "string"
}

deleteObservabilityWebhook

DELETE /api/v2/observability/webhook

Requires: admin role

Remove the namespace webhook. deleted is false when none was configured.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.delete_observability_webhook()

Responses

Example response 200

{
  "deleted": false
}

redriveObservabilityWebhook

POST /api/v2/observability/webhook/redrive

Requires: admin role

Requeue up to limit deliveries (default 100) that failed or have been pending for longer than the process abandonment window, for the current webhook configuration; the response counts each source. 409 webhook_not_configured without a webhook.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.redrive_observability_webhook()

Request body

ObservabilityWebhookRedrive (required)

{
  "limit": 0
}

Responses

Example response 202

{
  "abandoned_after_ms": 0,
  "from_abandoned_pending": 0,
  "from_failed": 0,
  "redriven": 0
}

observabilityWebhookStatus

GET /api/v2/observability/webhook/status

Requires: admin role

Delivery ledger counts by status and the 20 most recent deliveries with their last HTTP status and error.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.observability_webhook_status()

Responses

Example response 200

{
  "configured": false,
  "delivered": 0,
  "enabled": false,
  "failed": 0,
  "pending": 0,
  "recent": [
    {
      "attempts": 0,
      "created_at": 0,
      "delivered_at": 0,
      "event_type": "usage.recorded",
      "id": "string",
      "last_error": "string",
      "last_status": 0,
      "status": "string",
      "updated_at": 0
    }
  ],
  "secret_present": false
}

testObservabilityWebhook

POST /api/v2/observability/webhook/test

Requires: admin role

Queue a webhook.test event and return its delivery id; delivery happens on a background task with bounded retries. 409 webhook_not_configured when no enabled webhook accepts webhook.test.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.test_observability_webhook()

Request body

ObservabilityWebhookTest (required)

{
  "message": "string"
}

Responses

Example response 202

{
  "delivery_id": "string",
  "queued": false
}

recordRetrievalFeedback

POST /api/v2/retrieval/feedback

Requires: invoke role, worker role

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.record_retrieval_feedback()

Request body

RetrievalFeedback (required)

{
  "item_key": "string",
  "scope": "session",
  "verdict": "useful"
}

Responses

Example response 200

{
  "capability": "string",
  "id": "string",
  "item_key": "string",
  "recorded_at": 0,
  "source": "string",
  "verdict": "string"
}

retrievalFeedbackTotals

POST /api/v2/retrieval/feedback/totals

Requires: invoke role

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.retrieval_feedback_totals()

Request body

RetrievalFeedbackQuery (required)

{
  "group_by": "string",
  "since": 0
}

Responses

Example response 200

{
  "group_by": "string",
  "groups": [
    {
      "irrelevant": 0,
      "key": "string",
      "outdated": 0,
      "total": 0,
      "useful": 0,
      "wrong": 0
    }
  ],
  "scanned": 0,
  "since": 0,
  "totals": {
    "irrelevant": 0,
    "key": "string",
    "outdated": 0,
    "total": 0,
    "useful": 0,
    "wrong": 0
  }
}

stats

GET /api/v2/stats

Requires: invoke role

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.stats()

Parameters

Responses

Example response 200

{
  "complete": false,
  "namespace": "string",
  "oldest_retained_at": 0,
  "traversers": {},
  "window": {},
  "workers": {}
}

getRunMetadata

GET /api/v2/traversers/{id}/metadata

Requires: invoke role, worker role

The key/value metadata document of a root traverser (display names, MLflow trace ids, links). The path id must be a root (400 invalid_input otherwise). Readable by the owner, an admin, or the leased worker.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.get_run_metadata()

Parameters

Responses

Example response 200

{
  "entries": [
    {
      "key": "string",
      "set_by": "string",
      "updated_at": 0,
      "value": {}
    }
  ],
  "metadata": {
    "key": {}
  },
  "root_id": "string",
  "updated_at": 0
}

updateRunMetadata

PATCH /api/v2/traversers/{id}/metadata

Requires: invoke role, worker role

Merge set and remove unset on a root traverser's metadata. Keys are 1-128 characters of [A-Za-z0-9_.:/-], values at most 4096 bytes, at most 64 keys and 65536 bytes per run (400 invalid_metadata). Owners and admins write directly; a worker's lease must belong to this run (403 workflow_forbidden). Emits run_metadata.updated.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.update_run_metadata()

Parameters

Request body

RunMetadataUpdate (required)

{
  "lease_token": "string",
  "set": {
    "key": {}
  },
  "step_id": "string",
  "task_id": "string",
  "unset": [
    "string"
  ]
}

Responses

Example response 200

{
  "entries": [
    {
      "key": "string",
      "set_by": "string",
      "updated_at": 0,
      "value": {}
    }
  ],
  "metadata": {
    "key": {}
  },
  "removed": 0,
  "root_id": "string",
  "set": [
    "string"
  ],
  "unset": [
    "string"
  ],
  "updated_at": 0
}

listUsage

GET /api/v2/usage

Requires: invoke role, worker role

Usage rows of one execution (traverser_id) or one whole run (root_id must name a root). Readable by the owning principal, an admin, or a worker whose active lease covers the traverser (task_id + lease_token in the query). Ordered by created_at then id; pass next_cursor back as cursor. attempt_status joins the attempt table and is null for unknown attempts.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.list_usage()

Parameters

Responses

Example response 200

{
  "items": [
    {
      "attempt": 0,
      "attempt_status": "string",
      "cache_creation_tokens": 0,
      "cache_read_tokens": 0,
      "call_id": "string",
      "capability": "string",
      "confident": false,
      "cost_known": false,
      "cost_source": "string",
      "cost_usd": 0.0,
      "created_at": 0,
      "extra": {
        "key": {}
      },
      "fallback": "string",
      "id": "string",
      "input_tokens": 0,
      "label": "string",
      "model": "string",
      "operation": "string",
      "output_tokens": 0,
      "principal": "string",
      "provider": "string",
      "recorded_by": "string",
      "root_id": "string",
      "source": "string",
      "total_tokens": 0,
      "traverser_id": "string"
    }
  ],
  "next_cursor": "string",
  "root_id": "string",
  "traverser_id": "string"
}

recordUsage

POST /api/v2/usage

Requires: worker role

Record model or media calls of the attempt the caller is executing. Workers pass task_id (or step_id) + lease_token; the root tree is locked and the lease re-read before the write (409 parent_lease_invalid on a stale or cancelled lease, 403 lease_required without one). Admins may backfill with traverser_id. Provide entries (1-100) or a single entry. The same (traverser, attempt, call_id) is deduplicated and reported in items; without call_id the server assigns one and never deduplicates. cost_usd is stored only when supplied so unknown prices stay distinct from zero. Rows of failed attempts are retained. Committed writes fan out usage.recorded to the namespace webhook.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.record_usage()

Request body

UsageRecordRequest (required)

{
  "entries": [
    {
      "model": "string"
    }
  ],
  "entry": {
    "model": "string"
  },
  "lease_token": "string",
  "step_id": "string",
  "task_id": "string",
  "traverser_id": "string"
}

Responses

Example response 200

{
  "attempt": 0,
  "deduplicated": 0,
  "items": [
    {
      "call_id": "string",
      "deduplicated": false,
      "id": "string"
    }
  ],
  "recorded": 0,
  "root_id": "string",
  "totals": {
    "cache_creation_tokens": 0,
    "cache_read_tokens": 0,
    "calls": 0,
    "cost_complete": false,
    "cost_usd": 0.0,
    "fallback_calls": 0,
    "input_tokens": 0,
    "output_tokens": 0,
    "priced_calls": 0,
    "total_tokens": 0,
    "unconfident_calls": 0,
    "unpriced_calls": 0
  },
  "traverser_id": "string"
}

usageTotals

GET /api/v2/usage/totals

Requires: invoke role

Aggregate usage over the caller's own runs in [since, until] (default: the last 30 days). Admins may pass principal or all_principals; other callers get 403 principal_forbidden. group_by buckets by run, model, provider, day, attempt_status, label, principal, capability or none. capability is the traversal or skill that made the call, so it answers per-decision questions across runs. At most 50000 rows are aggregated; truncated is true when more matched. cost_usd is null unless at least one call was priced.

Generated Python client internals: atfgraph._generated.api.observability_api.ObservabilityApi.usage_totals()

Parameters

Responses

Example response 200

{
  "group_by": "run",
  "groups": [
    {
      "cache_creation_tokens": 0,
      "cache_read_tokens": 0,
      "calls": 0,
      "cost_complete": false,
      "cost_usd": 0.0,
      "fallback_calls": 0,
      "input_tokens": 0,
      "key": "string",
      "output_tokens": 0,
      "priced_calls": 0,
      "total_tokens": 0,
      "unconfident_calls": 0,
      "unpriced_calls": 0
    }
  ],
  "scanned": 0,
  "scope": {
    "namespace": "string",
    "principal": "string",
    "root_id": "string",
    "since": 0,
    "until": 0
  },
  "totals": {
    "cache_creation_tokens": 0,
    "cache_read_tokens": 0,
    "calls": 0,
    "cost_complete": false,
    "cost_usd": 0.0,
    "fallback_calls": 0,
    "input_tokens": 0,
    "output_tokens": 0,
    "priced_calls": 0,
    "total_tokens": 0,
    "unconfident_calls": 0,
    "unpriced_calls": 0
  },
  "truncated": false
}