From a84af5d464e014c0ea6d7f22d90bdf88b02d9802 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=CE=A7=CE=B3=CF=86=CF=84=20Kompanion?= Date: Wed, 15 Oct 2025 18:43:47 +0200 Subject: [PATCH] doc: add anything-llm compatibility plan --- doc/anything-llm-eval.md | 63 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 doc/anything-llm-eval.md diff --git a/doc/anything-llm-eval.md b/doc/anything-llm-eval.md new file mode 100644 index 0000000..93290e4 --- /dev/null +++ b/doc/anything-llm-eval.md @@ -0,0 +1,63 @@ +# AnythingLLM ↔ Kompanion Memory Compatibility Evaluation + +## Current Kompanion Memory Stack +- **Primary store**: Postgres 14+ with `pgvector` ≥ 0.6, accessed via the C++ `PgDal` implementation (`embeddings`, `memory_chunks`, `memory_items`, `namespaces` tables). Each embedding row keeps `id`, `chunk_id`, `model`, `dim`, `vector`, and a `normalized` flag. +- **Chunking & metadata**: Items are broken into chunks; embeddings attach to chunks via `chunk_id`. Item metadata lives as structured JSON on `memory_items` with tags, TTL, and revision controls. +- **Namespace model**: Logical scopes (e.g. `project:user:thread`) are first-class rows. Retrieval joins embeddings back to items to recover text + metadata. +- **Fallback mode**: Local-only path uses SQLite plus a FAISS sidecar (see `docs/MEMORY.md`) but the production design assumes Postgres. + +## AnythingLLM Vector Stack (PGVector path) +- Supports multiple vector backends; the overlapping option is `pgvector` (`server/utils/vectorDbProviders/pgvector/index.js`). +- Expects a single table (default `anythingllm_vectors`) shaped as `{ id UUID, namespace TEXT, embedding vector(n), metadata JSONB, created_at TIMESTAMP }`. +- Metadata is stored inline as JSONB; namespace strings are arbitrary workspace slugs. The embed dimension is fixed per table at creation time. +- The NodeJS runtime manages chunking, caching, and namespace hygiene, and assumes CRUD against that flat table. + +## Key Differences +- **Schema shape**: Kompanion splits data across normalized tables with foreign keys; AnythingLLM uses a single wide table per vector store. Kompanion’s embeddings currently lack a JSONB metadata column and instead rely on joins. +- **Identifiers**: Kompanion embeddings key off `chunk_id` (uuid/text) plus `model`; AnythingLLM expects a unique `id` per stored chunk and does not expose the underlying chunk relationship. +- **Metadata transport**: Kompanion keeps tags/TTL in `memory_items` (JSON) and chunk text in `memory_chunks`. AnythingLLM packs metadata (including document references and source identifiers) directly into the vector row’s JSONB. +- **Lifecycle hooks**: Kompanion enforces sensitivity flags before embedding; AnythingLLM assumes documents are already filtered and will happily ingest any chunk. Deletion flows differ (Kompanion uses soft-delete semantics; AnythingLLM issues hard deletes by namespace/document). +- **Embeddings contract**: Kompanion records embedding model and dimension per row; AnythingLLM fixes dimension at table creation and stores model choice in JSON metadata. + +## Compatibility Plan +1. **Agree on a shared pgvector table** + - Create (or reuse) a Postgres schema reachable by both systems. + - Define a composite view or materialized view that maps `embeddings` + `memory_chunks` + `memory_items` into the `anythingLLM` layout (columns: `id`, `namespace`, `embedding`, `metadata`, `created_at`). + - Add a JSONB projection that captures Kompanion metadata (`chunk_id`, `item_id`, `tags`, `model`, `revision`, sensitivity flags). This becomes the `metadata` field for AnythingLLM. + +2. **Write a synchronization job** + - Option A: database triggers on `embeddings` to insert/update a mirror row in `anythingllm_vectors`. + - Option B: periodic worker that scans for new/updated embeddings (`revision` or `updated_at`) and upserts into the shared table through SQL. + - Ensure deletions (soft or hard) propagate by expiring mirrored rows or respecting a `deleted_at` flag in metadata (AnythingLLM supports document purges via namespace filtering). + +3. **Normalize namespace semantics** + - Reuse Kompanion’s namespace string as the AnythingLLM workspace slug. + - Document mapping rules (e.g. replace `:` with `_` if AnythingLLM slugs disallow colons). + - Provide a compatibility map in metadata so both systems resolve back to Kompanion’s canonical namespace identity. + +4. **Unify embedding models** + - Select a shared embedding model (e.g., `text-embedding-3-large` or local Nomic). + - Record the chosen model in the mirrored metadata and enforce dimension on the `anythingllm_vectors` table creation. + - Update Kompanion’s embedding pipeline to fail fast if the produced dimension differs from the table’s fixed size. + +5. **Expose retrieval APIs** + - For Kompanion → AnythingLLM: implement a thin adapter that reads from the shared table instead of internal joins when responding to AnythingLLM requests (or simply let AnythingLLM talk directly to Postgres). + - For AnythingLLM → Kompanion: ensure the metadata payload includes the necessary identifiers (`item_id`, `chunk_id`) so Kompanion can resolve back to full context. + +6. **Security & sensitivity handling** + - Extend the metadata JSON to include Kompanion’s sensitivity/embeddable flags. + - Patch AnythingLLM ingestion to respect a `sensitivity` key (skip or mask secrets) before inserting into its table, or filter at the view level so secret rows never surface. + +7. **Validation & tooling** + - Add a migration checklist covering table creation, index alignment (`USING ivfflat`), and permission grants for the AnythingLLM service role. + - Create integration tests that: + 1. Upsert an item in Kompanion. + 2. Confirm mirrored row appears in `anythingllm_vectors`. + 3. Query through AnythingLLM API and verify the same chunk text + metadata round-trips. + +## Near-Term Tasks +1. Draft SQL for the projection view/materialized view, including JSONB assembly. +2. Prototype a synchronization worker (Python or C++) that mirrors embeddings into the AnythingLLM table. +3. Define namespace slug normalization rules and document them in both repos. +4. Coordinate on embedding model selection and update configuration in both stacks. +5. Add automated compatibility tests to CI pipelines of both projects.