Semantic & hybrid retrieval

Directory examples/python/features/retrieval

Compare lexical, semantic and hybrid search over one global document collection. The server stores chunks and BM25 citations; SemanticKnowledge embeds chunks and stores vectors as kb:desk:<doc_id>#<ordinal> in scoped memory. Returned semantic hits are checked against current document revisions so stale vectors are excluded.

Agent node label retrieval_desk, version 1.0.0.

Capabilities

Capability	Kind	Signature	Module	Summary
`ask`	traversal	`(question: str, mode: str = 'hybrid', top_k: int = 5, kind: Optional[str] = None) -> Answer`	`examples/python/features/retrieval/agent.py`	Ranked passages with citations. ``mode`` is ``hybrid``, ``semantic`` or
`forget`	traversal	`(doc_id: str) -> dict`	`examples/python/features/retrieval/agent.py`	Delete one document and its vectors.
`ingest`	traversal	`(documents: List[Dict[str, Any]]) -> dict`	`examples/python/features/retrieval/agent.py`	Atomic lexical batch, then indexing. A failed embedding step is reported per
`reindex`	traversal	`(doc_id: Optional[str] = None) -> dict`	`examples/python/features/retrieval/agent.py`	Rebuild vectors for one document or for the whole scope.
`status`	traversal	`() -> dict`	`examples/python/features/retrieval/agent.py`

Commands

Taken from this example's own README and module docstrings.

python -m atfgraph.worker --module examples.python.features.retrieval.agent:app
python -m examples.python.features.retrieval.call

Files

examples/python/features/retrieval/README.md
examples/python/features/retrieval/__init__.py
examples/python/features/retrieval/agent.py
examples/python/features/retrieval/agentnode.yaml
examples/python/features/retrieval/call.py
examples/python/features/retrieval/embeddings.py
examples/python/features/retrieval/fixtures.py

README

Rendered from examples/python/features/retrieval/README.md; the source file is not modified.

Traversal	Surface	Notes
`ingest(documents)`	`SemanticKnowledge.put_many(strict=False)`	atomic lexical batch, then bounded-concurrency embedding and `memory.vector.set`; per-document failures are returned, not raised
`ask(question, mode, top_k, kind)`	`SemanticKnowledge.search`	`hybrid` (default) / `semantic` / `lexical`; `kind` filters document metadata on both rankings; passages carry both scores and ranks, `stale` lists dropped vectors
`forget(doc_id)`	`SemanticKnowledge.delete`	tombstone plus vector removal
`reindex(doc_id?)`	`SemanticKnowledge.reindex` / `reindex_all`	repair after an interrupted `ingest` or a model change
`status()`	`SemanticKnowledge.stats`	knowledge counts plus vectors, indexed documents, model, dimension

Embedding provider

Set ATFGRAPH_EMBEDDING_URL and ATFGRAPH_EMBEDDING_MODEL for an OpenAI-compatible embeddings provider. Without them, embeddings.py uses a deterministic concept lexicon plus token hashing. It deliberately maps terms such as luggage/baggage and charges/fees together. That makes the offline workflow reproducible; it does not measure a learned embedding model's quality.

Run

python -m atfgraph.worker --module examples.python.features.retrieval.agent:app
python -m examples.python.features.retrieval.call

The driver ingests fixtures and queries them in all three modes, showing scores and ranks side by side. It also filters metadata, deletes policy:baggage, checks its removal from results and prints index status.

BM25 and cosine scores use different scales and meanings. Hybrid results retain both rather than treating them as interchangeable. After changing the embedding model or recovering an interrupted ingestion, run reindex.

The semantic-knowledge suite covers request bodies, concurrency, retries, partial failures, stale-vector checks, fake-server execution and real-daemon HTTP behavior. Fixture success does not verify a live embedding endpoint.