RAG

Retrieval-augmented knowledge lives in Django app agency.corpus.

Shape

Tenant → Workspace → Document → (extraction) → Chunk → Chroma collection

Workspaces carry embedding / chunk defaults and deterministic collection names (per tenant + workspace). Agents can query private knowledge without mixing that infra into ThreadRunner itself.

In this reference stack, vectors are stored in ChromaDB. That is a host deployment choice; the package talks to the configured store.

API surface (today)

REST viewsets and actions under /api/ include workspace and document resources (search / ask / reindex-style actions as implemented on the viewsets). See API and OpenAPI at /api/.

Related: miner (Scrap / Block) is scrape ingest, not the RAG index — Miner.

Module reference

Django app agency.corpus: document indexing and retrieval for RAG.

Models, vector store wiring, retrieval/generation helpers, and Celery tasks for indexing live under this package.

Corpus domain models: tenants, workspaces, documents, chunks, extractions.

agency.corpus.models.slugify_for_collection(value)[source]

Slug simple y estable para nombres de colección. Chroma suele tolerar bastante, pero conviene restringir a [a-z0-9_:-].

class agency.corpus.models.Tenant(*args, **kwargs)[source]

Bases: Model

Database table: corpus_tenant

Organización / cliente. Límite duro de aislamiento de datos.

Parameters:

id (UUIDField) – Primary key: Id
name (CharField) – Name
slug (SlugField) – Slug
is_active (BooleanField) – Is active
created_at (DateTimeField) – Created at

Reverse relationships:

Parameters:

memberships (Reverse ForeignKey from TenantMembership) – All memberships of this tenant (related name of tenant)
workspaces (Reverse ForeignKey from Workspace) – All workspaces of this tenant (related name of tenant)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

name

Type: CharField

Name

A wrapper for a deferred-loading field. When the value is read from this

slug

Type: SlugField

Slug

A wrapper for a deferred-loading field. When the value is read from this

is_active

Type: BooleanField

Is active

A wrapper for a deferred-loading field. When the value is read from this

created_at

Type: DateTimeField

Created at

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

get_next_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=True, **kwargs): Finds next instance based on created_at. See get_next_by_FOO() for more information.

get_previous_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=False, **kwargs): Finds previous instance based on created_at. See get_previous_by_FOO() for more information.

memberships

Type: Reverse ForeignKey from TenantMembership

All memberships of this tenant (related name of tenant)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

objects = <django.db.models.Manager object>

workspaces

Type: Reverse ForeignKey from Workspace

All workspaces of this tenant (related name of tenant)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

class agency.corpus.models.TenantMembership(*args, **kwargs)[source]

Bases: Model

Database table: corpus_tenantmembership

Links an operator to a tenant for corpus / workspace access.

Parameters:

id (UUIDField) – Primary key: Id
created_at (DateTimeField) – Created at

Relationship fields:

Parameters:

tenant (ForeignKey to Tenant) – Tenant (related name: memberships)
user (ForeignKey to User) – User (related name: tenant_memberships)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

tenant

Type: ForeignKey to Tenant

Tenant (related name: memberships)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

user

Type: ForeignKey to User

User (related name: tenant_memberships)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

created_at

Type: DateTimeField

Created at

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

get_next_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=True, **kwargs): Finds next instance based on created_at. See get_next_by_FOO() for more information.

get_previous_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=False, **kwargs): Finds previous instance based on created_at. See get_previous_by_FOO() for more information.

objects = <django.db.models.Manager object>

tenant_id: Internal field, use tenant instead.

user_id: Internal field, use user instead.

class agency.corpus.models.Workspace(*args, **kwargs)[source]

Bases: Model

Database table: corpus_workspace

Espacio dentro de un tenant (proyecto/departamento). Normalmente: 1 colección por workspace (recomendado) o por tenant+workspace.

Parameters:

id (UUIDField) – Primary key: Id
name (CharField) – Name
slug (SlugField) – Slug
description (TextField) – Description
default_embedding_model (CharField) – Default embedding model
default_chunk_size (PositiveIntegerField) – Default chunk size
default_chunk_overlap (PositiveIntegerField) – Default chunk overlap
created_at (DateTimeField) – Created at

Relationship fields:

Parameters:: tenant (ForeignKey to Tenant) – Tenant (related name: workspaces)

Reverse relationships:

Parameters:

ui_panels (Reverse ForeignKey from UiPanel) – All ui panels of this workspace (related name of workspace)
documents (Reverse ForeignKey from Document) – All documents of this workspace (related name of workspace)
display_assignments (Reverse ForeignKey from DisplayAssignment) – All display assignments of this workspace (related name of workspace)
display_views (Reverse ForeignKey from DisplayView) – All display views of this workspace (related name of workspace)
displays (Reverse ForeignKey from Display) – All displays of this workspace (related name of workspace)
display_scopes (Reverse ForeignKey from DisplayScope) – All display scopes of this workspace (related name of workspace)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

tenant

Type: ForeignKey to Tenant

Tenant (related name: workspaces)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

name

Type: CharField

Name

A wrapper for a deferred-loading field. When the value is read from this

slug

Type: SlugField

Slug

A wrapper for a deferred-loading field. When the value is read from this

description

Type: TextField

Description

A wrapper for a deferred-loading field. When the value is read from this

default_embedding_model

Type: CharField

Default embedding model

A wrapper for a deferred-loading field. When the value is read from this

default_chunk_size

Type: PositiveIntegerField

Default chunk size

A wrapper for a deferred-loading field. When the value is read from this

default_chunk_overlap

Type: PositiveIntegerField

Default chunk overlap

A wrapper for a deferred-loading field. When the value is read from this

created_at

Type: DateTimeField

Created at

A wrapper for a deferred-loading field. When the value is read from this

collection_name(env='prod', schema_version=1)[source]

Nombre determinista de colección. - env: dev/prod/staging - schema_version: súbelo si cambias metadatos/estrategia de ids

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

display_assignments

Type: Reverse ForeignKey from DisplayAssignment

All display assignments of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

display_scopes

Type: Reverse ForeignKey from DisplayScope

All display scopes of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

display_views

Type: Reverse ForeignKey from DisplayView

All display views of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

displays

Type: Reverse ForeignKey from Display

All displays of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

documents

Type: Reverse ForeignKey from Document

All documents of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

get_next_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=True, **kwargs): Finds next instance based on created_at. See get_next_by_FOO() for more information.

get_previous_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=False, **kwargs): Finds previous instance based on created_at. See get_previous_by_FOO() for more information.

objects = <django.db.models.Manager object>

tenant_id: Internal field, use tenant instead.

ui_panels

Type: Reverse ForeignKey from UiPanel

All ui panels of this workspace (related name of workspace)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

class agency.corpus.models.Document(*args, **kwargs)[source]

Bases: Model

Database table: corpus_document

A document is a file or a URL that is indexed by the system.

Parameters:

id (UUIDField) – Primary key: Id
title (CharField) – Title
file (FileField) – File
source_url (URLField) – Source url
summary (TextField) – Summary
checksum_sha256 (CharField) – Checksum sha256
status (CharField) – Status
error_message (TextField) – Error message
page_count (PositiveIntegerField) – Page count
chunk_count (PositiveIntegerField) – Chunk count
indexed_at (DateTimeField) – Indexed at
created_at (DateTimeField) – Created at
updated_at (DateTimeField) – Updated at

Relationship fields:

Parameters:: workspace (ForeignKey to Workspace) – Workspace (related name: documents)

Reverse relationships:

Parameters:

extractions (Reverse ForeignKey from Extraction) – All extractions of this document (related name of document)
assistants (Reverse ManyToManyField from Assistant) – All assistants of this document (related name of documents)

class Status(value)[source]

Bases: TextChoices

UPLOADED = 'UPLOADED'

PARSING = 'PARSING'

INDEXING = 'INDEXING'

INDEXED = 'INDEXED'

FAILED = 'FAILED'

DELETED = 'DELETED'

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

workspace

Type: ForeignKey to Workspace

Workspace (related name: documents)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

title

Type: CharField

Title

A wrapper for a deferred-loading field. When the value is read from this

file

Type: FileField

File

The descriptor for the file attribute on the model instance. Return a FieldFile when accessed so you can write code like:

>>> from myapp.models import MyModel
>>> instance = MyModel.objects.get(pk=1)
>>> instance.file.size

Assign a file object on assignment so you can do:

>>> with open('/path/to/hello.world') as f:

source_url

Type: URLField

Source url

A wrapper for a deferred-loading field. When the value is read from this

summary

Type: TextField

Summary

A wrapper for a deferred-loading field. When the value is read from this

checksum_sha256

Type: CharField

Checksum sha256

A wrapper for a deferred-loading field. When the value is read from this

status

Type: CharField

Status

Choices:

UPLOADED
PARSING
INDEXING
INDEXED
FAILED
DELETED

A wrapper for a deferred-loading field. When the value is read from this

error_message

Type: TextField

Error message

A wrapper for a deferred-loading field. When the value is read from this

page_count

Type: PositiveIntegerField

Page count

A wrapper for a deferred-loading field. When the value is read from this

chunk_count

Type: PositiveIntegerField

Chunk count

A wrapper for a deferred-loading field. When the value is read from this

indexed_at

Type: DateTimeField

Indexed at

A wrapper for a deferred-loading field. When the value is read from this

created_at

Type: DateTimeField

Created at

A wrapper for a deferred-loading field. When the value is read from this

updated_at

Type: DateTimeField

Updated at

A wrapper for a deferred-loading field. When the value is read from this

checksum_changes()[source]

mark_indexed(chunk_count=None)[source]

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

assistants

Type: Reverse ManyToManyField from Assistant

All assistants of this document (related name of documents)

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager

extractions

Type: Reverse ForeignKey from Extraction

All extractions of this document (related name of document)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

get_next_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=True, **kwargs): Finds next instance based on created_at. See get_next_by_FOO() for more information.

get_next_by_updated_at(*, field=<django.db.models.DateTimeField: updated_at>, is_next=True, **kwargs): Finds next instance based on updated_at. See get_next_by_FOO() for more information.

get_previous_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=False, **kwargs): Finds previous instance based on created_at. See get_previous_by_FOO() for more information.

get_previous_by_updated_at(*, field=<django.db.models.DateTimeField: updated_at>, is_next=False, **kwargs): Finds previous instance based on updated_at. See get_previous_by_FOO() for more information.

get_status_display(*, field=<django.db.models.CharField: status>): Shows the label of the status. See get_FOO_display() for more information.

objects = <django.db.models.Manager object>

workspace_id: Internal field, use workspace instead.

class agency.corpus.models.Extraction(id, document, created_at, method)[source]

Bases: Model

Database table: corpus_extraction

Parameters:

id (UUIDField) – Primary key: Id
created_at (DateTimeField) – Created at
method (CharField) – Method

Relationship fields:

Parameters:: document (ForeignKey to Document) – Document (related name: extractions)

Reverse relationships:

Parameters:

extracts (Reverse ForeignKey from Extract) – All extracts of this extraction (related name of extraction)
sections (Reverse ForeignKey from Section) – All sections of this extraction (related name of extraction)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

document

Type: ForeignKey to Document

Document (related name: extractions)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

created_at

Type: DateTimeField

Created at

A wrapper for a deferred-loading field. When the value is read from this

method

Type: CharField

Method

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

document_id: Internal field, use document instead.

extracts

Type: Reverse ForeignKey from Extract

All extracts of this extraction (related name of extraction)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

get_next_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=True, **kwargs): Finds next instance based on created_at. See get_next_by_FOO() for more information.

get_previous_by_created_at(*, field=<django.db.models.DateTimeField: created_at>, is_next=False, **kwargs): Finds previous instance based on created_at. See get_previous_by_FOO() for more information.

objects = <django.db.models.Manager object>

sections

Type: Reverse ForeignKey from Section

All sections of this extraction (related name of extraction)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

class agency.corpus.models.Extract(id, extraction, text, page)[source]

Bases: Model

Database table: corpus_extract

Parameters:

id (UUIDField) – Primary key: Id
text (TextField) – Text
page (PositiveIntegerField) – Page

Relationship fields:

Parameters:: extraction (ForeignKey to Extraction) – Extraction (related name: extracts)

Reverse relationships:

Parameters:: chunks (Reverse ForeignKey from Chunk) – All chunks of this extract (related name of extract)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

extraction

Type: ForeignKey to Extraction

Extraction (related name: extracts)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

text

Type: TextField

Text

A wrapper for a deferred-loading field. When the value is read from this

page

Type: PositiveIntegerField

Page

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

chunks

Type: Reverse ForeignKey from Chunk

All chunks of this extract (related name of extract)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

extraction_id: Internal field, use extraction instead.

objects = <django.db.models.Manager object>

class agency.corpus.models.Section(*args, **kwargs)[source]

Bases: Model

Database table: corpus_section

One section can be se

Parameters:

id (UUIDField) – Primary key: Id
title (CharField) – Title

Relationship fields:

Parameters:: extraction (ForeignKey to Extraction) – Extraction (related name: sections)

Reverse relationships:

Parameters:: chunks (Reverse ForeignKey from Chunk) – All chunks of this section (related name of section)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

extraction

Type: ForeignKey to Extraction

Extraction (related name: sections)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

title

Type: CharField

Title

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

chunks

Type: Reverse ForeignKey from Chunk

All chunks of this section (related name of section)

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager

extraction_id: Internal field, use extraction instead.

objects = <django.db.models.Manager object>

class agency.corpus.models.Chunk(id, extract, section, metadata, text, clear)[source]

Bases: Model

Database table: corpus_chunk

Parameters:

id (UUIDField) – Primary key: Id
metadata (JSONField) – Metadata
text (TextField) – Text
clear (BooleanField) – Clear

Relationship fields:

Parameters:

extract (ForeignKey to Extract) – Extract (related name: chunks)
section (ForeignKey to Section) – Section (related name: chunks)

id

Type: UUIDField

Primary key: Id

A wrapper for a deferred-loading field. When the value is read from this

extract

Type: ForeignKey to Extract

Extract (related name: chunks)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

section

Type: ForeignKey to Section

Section (related name: chunks)

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

metadata

Type: JSONField

Metadata

A wrapper for a deferred-loading field. When the value is read from this

text

Type: TextField

Text

A wrapper for a deferred-loading field. When the value is read from this

clear

Type: BooleanField

Clear

A wrapper for a deferred-loading field. When the value is read from this

exception DoesNotExist: Bases: ObjectDoesNotExist

exception MultipleObjectsReturned: Bases: MultipleObjectsReturned

extract_id: Internal field, use extract instead.

objects = <django.db.models.Manager object>

section_id: Internal field, use section instead.