Models

The domain data, all frozen dataclasses. Nothing here knows about a database, a provider or a web request.

Documents, pages and chunks

The domain vocabulary the whole engine speaks.

Every value that crosses a port boundary is one of these frozen dataclasses. They are deliberately plain data — no behaviour that touches a model, a store, or the network — so the same objects flow unchanged whether the engine is driven from a CLI with fakes or from the SaaS backend with real adapters.

The pipeline reads bottom-to-top of this file:

RawSource -> (parser) -> Document + Page[] -> (route + chunk) -> Chunk[] Chunk[] -> (embed) -> EmbeddedChunk[] -> (store) question -> (retrieve/rerank) -> ScoredChunk[] -> (generate) -> Answer

None of these types know about tenants: isolation is carried by the opaque Scope, which the caller supplies alongside them.

ragsage.models.Vector

A dense embedding. A plain tuple keeps domain values hashable and immutable; adapters convert to/from numpy or pgvector at their own edge.

alias of tuple[float, …]

class ragsage.models.RawSource(name: str, content: bytes | None = None, path: str | None = None, media_type: str | None = None)

[source]

Bases: object

An opaque document to ingest, as handed to IngestionPipeline.ingest().

A source is either in-memory (content) or a filesystem path; the DocumentParser decides how to read it. Keeping the engine agnostic to where bytes come from is what lets the CLI feed local files and the backend feed object-storage blobs through the same port.

name : str

content : bytes | None = None

path : str | None = None

media_type : str | None = None

read() → bytes

[source]

The raw bytes, from memory or from path.

Lives on the model rather than in a parser helper because the pipeline needs the bytes too, to compute the content hash its parse cache keys on before any parser has run. Reaching into ragsage.parsing for that would pull the whole parser stack — pdfplumber and friends — into a bare import ragsage, which the dependency guard exists to prevent.

class ragsage.models.PageImage(ref: str, data: bytes | None = None)

[source]

Bases: object

A pointer to a page rendered as an image, for the vision/OCR route.

ref is an opaque handle the parser understands (a filesystem path, an object key); data is the optional raw image when it is already in hand.

ref : str

data : bytes | None = None

class ragsage.models.PageLayout(text_chars: int = 0, image_area_ratio: float = 0.0, bitmap_area: float = 0.0)

[source]

Bases: object

The geometric signals a parser measures about a page, for route classification.

A PageClassifier weighs these to tell a page whose born-digital text layer is usable from a scanned or photographed one that only a vision model can read — without re-parsing the page itself.

text_chars is the length of the extractable text layer (whether that text is rendered or a hidden OCR layer — both count as “there is text to trust”). image_area_ratio is the fraction [0, 1] of the page’s area covered by raster images; bitmap_area is the absolute area, in the page’s own units, of the largest embedded bitmap — a page-filling scan reads high on both while a small inline figure on a typed page reads low.

text_chars : int = 0

image_area_ratio : float = 0.0

bitmap_area : float = 0.0

class ragsage.models.Page(number: int, text: str = '', image: PageImage | None = None, layout: PageLayout | None = None)

[source]

Bases: object

One page as the parser found it, before routing decides how to read it.

A born-digital page arrives with a usable text layer; a scanned or photographed page arrives with an empty text and an image to be transcribed by the vision model. Most real pages have both. layout carries the measured signals a PageClassifier routes on; a parser that can’t measure them (a flow format with no page grid) leaves it None and the classifier falls back to whether a text layer is present.

number : int

text : str = ''

image : PageImage | None = None

layout : PageLayout | None = None

class ragsage.models.PageRoute(*values)

[source]

Bases: StrEnum

How a page’s text gets extracted — the two-path routing decision.

TEXT uses the born-digital text layer directly. VISION sends the page image to a premium vision model, skipping any traditional-OCR tier.

TEXT = 'text'

VISION = 'vision'

class ragsage.models.Document(id: str, source: str, content_hash: str, metadata: Mapping[str, object]=)

[source]

Bases: object

A source document once parsed: identity plus its provenance metadata.

id is stable and content-derived (see content_hash) so re-ingesting the same bytes is recognisably a duplicate. source is the human label (original filename); metadata carries anything the caller wants to filter on later (author, tags), never anything tenant-shaped.

id : str

source : str

content_hash : str

metadata : Mapping[str, object]

class ragsage.models.ParsedDocument(document: Document, pages: Sequence[Page])

[source]

Bases: object

A Document together with its ordered pages, as a parser returns it.

document : Document

pages : Sequence[Page]

class ragsage.models.Chunk(id: str, document_id: str, text: str, page: int, ordinal: int, embed_text: str = '', metadata: Mapping[str, object]=)

[source]

Bases: object

A retrievable unit of a document, carrying everything a citation needs.

text is the verbatim passage shown to the user. embed_text is what actually gets embedded — identical to text unless contextual retrieval has prepended a document-level context sentence. Splitting the two is what lets an answer quote the clean passage while retrieval matches on the enriched one.

id : str

document_id : str

text : str

page : int

ordinal : int

embed_text : str = ''

metadata : Mapping[str, object]

class ragsage.models.EmbeddedChunk(chunk: Chunk, vector: tuple[float, ...])

[source]

Bases: object

A chunk paired with its dense embedding, as written to the vector store.

chunk : Chunk

vector : tuple[float, ...]

class ragsage.models.ScoredChunk(chunk: Chunk, score: float)

[source]

Bases: object

A chunk with a relevance score attached by retrieval or reranking.

The same shape flows out of dense search, lexical search, fusion, and the reranker; score is only meaningful relative to its siblings in one list.

chunk : Chunk

score : float

class ragsage.models.Citation(marker: int, chunk_id: str, document_id: str, page: int, quote: str)

[source]

Bases: object

Binds a marker in the answer text to the exact passage it came from.

marker is the [n] the reader sees; the rest points at the source so the UI can open the precise page/passage. A grounded answer’s every marker resolves to one of these.

marker : int

chunk_id : str

document_id : str

page : int

quote : str

class ragsage.models.Turn(question: str, answer: str)

[source]

Bases: object

One prior exchange in a conversation — a question and the answer it got.

A sequence of these is the conversation history handed to a query so a follow-up can be resolved against it: QueryEngine condenses the latest question plus the turns before it into a standalone question before retrieving. Deliberately plain — no citations or usage — because rewriting a pronoun only needs the words that were said, not how they were sourced.

question : str

answer : str

class ragsage.models.Outcome(*values)

[source]

Bases: StrEnum

How a query resolved — the three ways a reply can end.

ANSWERED is the product: prose drawn from retrieved sources, every claim carrying a citation. NOT_FOUND is the honest refusal — the corpus was searched and couldn’t support an answer. CONVERSATIONAL is a reply to a message that was never a question about the corpus (a greeting, a thanks, “what can you do?”); nothing was searched, so there is nothing to have found.

The distinction between the last two is not cosmetic: both are ungrounded and uncited, but only NOT_FOUND means “I looked in your documents”. Telling a user their greeting wasn’t in their documents is a lie about what happened.

ANSWERED = 'answered'

NOT_FOUND = 'not_found'

CONVERSATIONAL = 'conversational'

class ragsage.models.Answer(text: str, citations: Sequence[Citation] = (), outcome: Outcome = Outcome.ANSWERED)

[source]

Bases: object

The result of a query: grounded prose plus verifiable citations.

When the corpus can’t support an answer, outcome is Outcome.NOT_FOUND, text is the honest not-found message, and citations is empty — the engine never fabricates a confident answer out of thin retrieval. grounded stays the one-bit read of that (“is this backed by sources?”), derived rather than stored so it can never disagree with the outcome it summarises.

text : str

citations : Sequence[Citation] = ()

outcome : Outcome = 'answered'

property grounded : bool

Whether the text is supported by the cited sources.

class ragsage.models.AnswerToken(text: str)

[source]

Bases: object

One streamed fragment of the answer text, as the model emits it.

text : str

class ragsage.models.Usage(sources: int = 0, completion_tokens: int = 0)

[source]

Bases: object

A best-effort accounting of one answer’s shape.

sources is how many chunks were placed in the model’s context; completion_tokens counts the fragments streamed back. These are what the engine can observe without a tokenizer — provider-reported token counts can layer in later through the gateway without changing this shape.

sources : int = 0

completion_tokens : int = 0

class ragsage.models.AnswerComplete(text: str, citations: Sequence[Citation] = (), outcome: Outcome = Outcome.ANSWERED)

[source]

Bases: object

The terminal streamed event: the fully-assembled, citation-bound answer.

Carries the same fields as Answer, so a consumer that only wants the final result can ignore the incremental events and read this one.

text : str

citations : Sequence[Citation] = ()

outcome : Outcome = 'answered'

property grounded : bool

Whether the text is supported by the cited sources.

ragsage.models.AnswerEvent = ragsage.models.AnswerToken | ragsage.models.Citation | ragsage.models.Usage | ragsage.models.AnswerComplete

The union a streamed query yields; consumers dispatch on the concrete type.

Scope

The engine’s only isolation concept, and the boundary that lets one engine serve a script and a multi-tenant SaaS unchanged. A pricing, auth or tenancy change must never require editing this file — that is the litmus test for the boundary.

The opaque retrieval scope handed to the engine by its caller.

ragsage is deliberately tenancy-agnostic: it must work unchanged whether it is driven single-tenant from a CLI or multi-tenant from the SaaS backend. The backend maps tenant_id -> Scope; the library only ever sees a namespace string plus optional metadata filters (e.g. a subset of document ids). A pricing/auth/tenancy change must never require editing this file — that is the litmus test for the boundary.

class ragsage.scope.Scope(namespace: str, filters: Mapping[str, object]=)

[source]

Bases: object

An opaque isolation boundary for ingestion and retrieval.

• Parameters:
  • namespace (str) – The hard partition every store keys on. The backend passes the tenant id here; a CLI user might pass "local". Must be non-empty.
  • filters (Mapping [str , object ]) – Optional metadata narrowing within a namespace — for example restricting a query to a chosen subset of documents. Never widens access beyond namespace.

namespace : str

filters : Mapping[str, object]

with_filters(**extra: object) → Scope

[source]

Return a copy narrowed by additional metadata filters.