362 lines
14 KiB
Markdown
362 lines
14 KiB
Markdown
Below is a **single copy‑pastable Markdown file** that proposes a client‑side architecture which treats memory as a living, hierarchical JSON **dictionary‑of‑dictionaries** (HDoD), adds *semantic + episodic activation* and pruning, and composes prompts for your coding agent. It maps cleanly onto the **MCP tools you currently expose**:
|
||
|
||
* `kom.meta.v1.project_snapshot`
|
||
* `kom.local.v1.backup.import_encrypted`
|
||
* `kom.local.v1.backup.export_encrypted`
|
||
* `kom.memory.v1.search_memory`
|
||
* `kom.memory.v1.upsert_memory`
|
||
* `kom.memory.v1.recall_context`
|
||
* `kom.memory.v1.save_context`
|
||
|
||
It keeps the server simple and pushes *intelligence about memory* into the **client orchestrator**, so you can get the behavior you want **today**, without first redesigning the server.
|
||
|
||
---
|
||
|
||
# Kompanion Client Memory Architecture (HDoD)
|
||
|
||
**Version:** 0.2 • **Scope:** Client‑side AI interface to Kompanion MCP Server
|
||
**Author:** Χγφτ (Kompanion of Esus / Andre)
|
||
**Purpose:** Make memory *behave* like a nested JSON of concepts that “lights up” semantically and episodically, prunes naturally, and feeds agent prompts with high‑quality domain tokens (e.g., C++ patterns) — *without* waiting on a more complex server.
|
||
|
||
---
|
||
|
||
## 1. Why this exists
|
||
|
||
Large models feel “smart but memoryless.” We want:
|
||
|
||
1. **A hierarchical mental map** (JSON dictionary‑of‑dictionaries, “HDoD”),
|
||
2. **Activation dynamics** (semantic + episodic “lighting up” of nodes/paths),
|
||
3. **Organic pruning** (cool down; unload),
|
||
4. **Prefilled knowledge packs** (domain seeds: e.g., C++ idioms),
|
||
5. **Deterministic prompt composition** for coding agents (Codex/Qwen/etc.).
|
||
|
||
The server currently provides a **minimal memory API**. That’s fine: we’ll implement the cognitive part **client‑side** and use the server as a persistence/search/recall backbone.
|
||
|
||
---
|
||
|
||
## 2. The mental model (HDoD = dictionary‑of‑dictionaries)
|
||
|
||
Think of memory as a normalized **graph**, *presented* to the user/agent as a **dictionary tree**. Each node is a **Concept** with:
|
||
|
||
```json
|
||
{
|
||
"id": "skill.cpp.templates.sfinae",
|
||
"type": "concept", // concept | skill | fact | snippet | pattern | episode | tool | persona | task
|
||
"label": "SFINAE",
|
||
"payload": { "definition": "...", "examples": ["..."], "anti_patterns": ["..."] },
|
||
"embeddings": { "model": "local-bge|text-embed-3", "vector": [/* ... */] },
|
||
"links": [
|
||
{"to": "skill.cpp.templates.metaprogramming", "rel": "is_a", "w": 0.8},
|
||
{"to": "pattern.cpp.enable_if", "rel": "uses", "w": 0.7}
|
||
],
|
||
"children": { "intro": "skill.cpp.templates.sfinae.intro", "advanced": "skill.cpp.templates.sfinae.advanced" },
|
||
"stats": { "uses": 12, "last_used_at": "2025-10-14T18:42:00Z" },
|
||
"resonance": { "value": 0.0, "decay": 0.98, "last_activated_at": null },
|
||
"meta": { "source": "seedpack:cpp-core", "version": "1.0.0" }
|
||
}
|
||
```
|
||
|
||
> **Presentation vs. storage:** on the wire we keep nodes **normalized** (graph), but for agent prompts we can **materialize a subtree** as a JSON dictionary (exactly your intuition).
|
||
|
||
---
|
||
|
||
## 3. Core client components
|
||
|
||
### 3.1 Memory Orchestrator (library)
|
||
|
||
A small client library (TypeScript or Python) that owns:
|
||
|
||
* **Local Cache & Working Set**
|
||
|
||
* in‑memory map of *hot nodes* (activated concepts, episodes, tasks)
|
||
* TTL + **resonance decay** keeps it naturally pruned
|
||
|
||
* **Activation Engine**
|
||
Computes a **Resonance Score** used for selection/exploration:
|
||
|
||
```
|
||
score(node | query, task) =
|
||
α * cosine(embedding(node), embed(query))
|
||
+ β * max_edge_weight_to(frontier)
|
||
+ γ * recency(node)
|
||
+ δ * usage(node)
|
||
+ ε * task_affinity(node, task.tags)
|
||
+ ζ * persona_weight(node, active_persona)
|
||
```
|
||
|
||
Typical: α=0.45, β=0.20, γ=0.15, δ=0.10, ε=0.07, ζ=0.03
|
||
|
||
* **HDoD Composer**
|
||
Given a set of nodes, **materialize** a JSON dictionary tree (merge, order by score, trim to token budget).
|
||
|
||
* **Context Frames**
|
||
Structured blocks that the agent can consume:
|
||
|
||
* *Identity Frame* (who am I / tools)
|
||
* *Problem Frame* (task/spec)
|
||
* *Knowledge Frame* (HDoD subtree from semantic activation)
|
||
* *Episodic Frame* (recent steps/outcomes, e.g., compilation logs)
|
||
* *Constraints Frame* (APIs, signatures, tests)
|
||
* *Scratchpad Frame* (space for chain‑of‑thought *outside* model hidden state—LLM sees a compact, explicit scratch area)
|
||
|
||
* **Server Adapters** (mapping to your MCP tools)
|
||
|
||
* `search_memory(query)` → seeds for *semantic activation*
|
||
* `recall_context(task|session)` → seeds for *episodic activation*
|
||
* `save_context(blocks)` → write back learned episodes/summaries
|
||
* `upsert_memory(nodes)` → persist new/updated concepts/snippets
|
||
* `project_snapshot()` → immutable snapshot of current mental state
|
||
* `backup.export_encrypted()` / `backup.import_encrypted()` → **Knowledge Packs** (see §5)
|
||
|
||
> This is enough to *behave* like a cognitive system, while your server stays simple and fast.
|
||
|
||
---
|
||
|
||
## 4. Algorithms (client‑side, concise)
|
||
|
||
### 4.1 Probe → Bloom → Trim (context building)
|
||
|
||
```
|
||
build_context(query, task, budget):
|
||
seeds_sem = kom.memory.search_memory(query, k=32)
|
||
seeds_epi = kom.memory.recall_context(task_id=task.id, k=32)
|
||
frontier = normalize(seeds_sem ∪ seeds_epi)
|
||
|
||
for hop in 1..H (H=2 or 3):
|
||
neighbors = expand(frontier, max_per_node=6) // via node.links
|
||
frontier = frontier ∪ neighbors
|
||
update_resonance(frontier, query, task)
|
||
|
||
selected = topK_by_type(frontier, K_by_type) // diversity caps
|
||
frames = compose_frames(query, task, selected, budget) // HDoD for Knowledge Frame; episodes for Episodic Frame
|
||
|
||
kom.memory.save_context({ task_id: task.id, frames })
|
||
return frames
|
||
```
|
||
|
||
**Natural pruning**: each tick, `node.resonance.value *= node.resonance.decay`; nodes fall out of the Working Set unless re‑activated.
|
||
|
||
### 4.2 Upserting *observations* and *skills*
|
||
|
||
* After actions (file write, compile run, test pass/fail), emit **Observation** nodes (type `episode`) with edges to involved concepts/snippets.
|
||
* When the model discovers a pattern (“prefer RAII for resource ownership”), emit **Skill** nodes with `examples` and `anti_patterns`.
|
||
|
||
### 4.3 Materializing HDoD
|
||
|
||
Given selected nodes, build a JSON dictionary with *paths as keys* (e.g., `skill.cpp.templates.sfinae`) and nested maps for child grouping. Keep atomic fields compact (definitions, signatures) and push long text to a `details` field that can be compressed or summarized.
|
||
|
||
---
|
||
|
||
## 5. Knowledge Packs (prefilling intelligence)
|
||
|
||
**Goal:** give your coder agent *real* domain knowledge (C++ idioms, STL nuances, build systems, unit testing patterns) as compact, queryable, embedded chunks.
|
||
|
||
* **Format:** Encrypted tar/zip with manifest
|
||
|
||
```
|
||
pack.json:
|
||
id, name, version, domain_tags: ["cpp","cmake","catch2"],
|
||
embedding_model, created_at, checksum
|
||
nodes.jsonl:
|
||
{"id":"skill.cpp.raii", "type":"skill", "payload":{...}, "embeddings":{...}, "links":[...]}
|
||
...
|
||
```
|
||
* **Import/Export via existing tools:**
|
||
|
||
* `kom.local.v1.backup.import_encrypted(pack)`
|
||
* `kom.local.v1.backup.export_encrypted(selection)`
|
||
* **Curation approach:** create *micro‑chunks*:
|
||
|
||
* **Concept**: “RAII”, “SFINAE”, “Rule of 5/0”, “ADL”, “Type erasure”
|
||
* **Pattern**: “pImpl”, “CRTP”, “Enable‑if idiom”
|
||
* **Snippet**: idiomatic examples (≤30 lines), compile‑checked
|
||
* **Anti‑patterns**: “raw new/delete in modern C++”, “overusing exceptions”
|
||
* **Build/Tooling**: CMake minimum skeletons, `add_library`, interfaces, `FetchContent`
|
||
* **Test**: Catch2/GoogleTest minimal cases; property‑based testing sketch
|
||
|
||
> Once imported, the Pack is just **memory**. The Activation Engine will surface it the same way it surfaces your episodes.
|
||
|
||
---
|
||
|
||
## 6. Client ↔ Server mapping (today’s API)
|
||
|
||
### 6.1 Search (semantic seeds)
|
||
|
||
```ts
|
||
await kom.memory.v1.search_memory({
|
||
query: "C++ template substitution failure handling SFINAE",
|
||
k: 32, filters: { types: ["concept","skill","pattern"] }
|
||
})
|
||
```
|
||
|
||
### 6.2 Recall (episodic seeds)
|
||
|
||
```ts
|
||
await kom.memory.v1.recall_context({
|
||
scope: "task", id: task.id, k: 32
|
||
})
|
||
```
|
||
|
||
### 6.3 Save context (write-back frames)
|
||
|
||
```ts
|
||
await kom.memory.v1.save_context({
|
||
task_id: task.id,
|
||
frames: [
|
||
{ kind: "knowledge", format: "hdod.json", data: {/* nested dict */} },
|
||
{ kind: "episodic", format: "markdown", data: "# Steps\n- Compiled...\n- Tests..." }
|
||
]
|
||
})
|
||
```
|
||
|
||
### 6.4 Upsert memory (new skills/snippets)
|
||
|
||
```ts
|
||
await kom.memory.v1.upsert_memory({
|
||
nodes: [ /* normalized nodes like in §2 */ ],
|
||
merge: true
|
||
})
|
||
```
|
||
|
||
### 6.5 Snapshots & Packs
|
||
|
||
```ts
|
||
await kom.meta.v1.project_snapshot({ include_memory: true })
|
||
await kom.local.v1.backup.export_encrypted({ selection: "domain:cpp", out: "packs/cpp-core.kpack" })
|
||
```
|
||
|
||
> **Important:** Client keeps **resonance state** locally. Server remains simple KV/search/recall/persist.
|
||
|
||
---
|
||
|
||
## 7. Prompt composition for a coding agent
|
||
|
||
**Goal:** Transform the Working Set into a *stable prompt contract*, so the agent operates above “first‑grade cobbling.”
|
||
|
||
**Prompt Frames (ordered):**
|
||
|
||
1. **Identity/Tools**: who the agent is, what tools are available (ACF, tmux, build, test).
|
||
2. **Problem Frame**: concise task, interfaces, constraints.
|
||
3. **Knowledge Frame (HDoD)**: the **hierarchical dictionary** of concepts/patterns/snippets selected by activation; *max 40–60% of token budget*.
|
||
4. **Episodic Frame**: last N steps + outcomes; keep terse.
|
||
5. **Constraints Frame**: language level (C++20), error policies, style (guidelines support library, ranges), testing expectation.
|
||
6. **Scratchpad Frame**: allow the model to outline plan & invariants (explicit, not hidden).
|
||
|
||
**Effect:** The agent “feels” like it *knows* C++ idioms (because it sees compact, curated, **embedded** patterns every turn), and it keeps context from previous steps (episodic frame).
|
||
|
||
---
|
||
|
||
## 8. Data shape & invariants
|
||
|
||
* **IDs are path‑like**: `skill.cpp.templates.sfinae` (hierarchy is explicit).
|
||
* **Graph canonical, Dicts for presentation**: treat `children` as **references**; avoid deep duplication.
|
||
* **Embeddings are per node**; you may add **type‑specific** vectors later.
|
||
* **Edges carry weights**; they contribute to resonance.
|
||
* **Resonance decays** every tick; any node with `value < ε` leaves the Working Set.
|
||
* **Budgets**: Top‑K per type (e.g., 6 skills, 10 snippets, 4 patterns) to avoid monoculture.
|
||
|
||
---
|
||
|
||
## 9. Minimal TypeScript client surface (sketch)
|
||
|
||
```ts
|
||
type NodeType = "concept"|"skill"|"fact"|"snippet"|"pattern"|"episode"|"tool"|"persona"|"task";
|
||
|
||
interface KomNode {
|
||
id: string;
|
||
type: NodeType;
|
||
label: string;
|
||
payload?: any;
|
||
embeddings?: { model: string; vector: number[] };
|
||
links?: { to: string; rel: string; w?: number }[];
|
||
children?: Record<string, string>;
|
||
stats?: { uses?: number; last_used_at?: string };
|
||
resonance?: { value: number; decay: number; last_activated_at?: string | null };
|
||
meta?: Record<string, any>;
|
||
}
|
||
|
||
interface Frames {
|
||
identity?: string;
|
||
problem: string;
|
||
knowledgeHDoD?: Record<string, any>;
|
||
episodic?: string;
|
||
constraints?: string;
|
||
scratchpad?: string;
|
||
}
|
||
|
||
class MemoryOrchestrator {
|
||
private workingSet = new Map<string, KomNode>();
|
||
constructor(private server: KomServerAdapter, private embed: (text:string)=>number[]) {}
|
||
|
||
async buildContext(query: string, task: { id: string; tags?: string[] }, budgetTokens: number): Promise<Frames> {
|
||
const seeds = await this.server.searchMemory(query, 32);
|
||
const epis = await this.server.recallContext(task.id, 32);
|
||
this.seed(seeds.concat(epis)); // normalize to nodes in workingSet
|
||
this.bloom(query, task, 2); // expand via links, update resonance
|
||
const selected = this.selectByTypeCaps(task); // diversity caps
|
||
const knowledgeHDoD = this.materializeHDoD(selected, budgetTokens);
|
||
const frames: Frames = { problem: this.renderProblem(task), knowledgeHDoD };
|
||
await this.server.saveContext(task.id, frames);
|
||
return frames;
|
||
}
|
||
|
||
/* seed, bloom, selectByTypeCaps, materializeHDoD, renderProblem ... */
|
||
}
|
||
```
|
||
|
||
> This is intentionally thin; you can drop it into your existing client shell and wire the 7 server calls you already have.
|
||
|
||
---
|
||
|
||
## 10. How this reflects **Elope**’s spirit
|
||
|
||
Your earlier **Elope** work separated **episodic** from **semantic** memory and played with identity, observations, and “resonance/whales” motifs. This client keeps that spirit:
|
||
|
||
* Episodic = **Observations/episodes** (recent steps, logs).
|
||
* Semantic = **Concepts/skills/patterns** (stable knowledge packs + learned patterns).
|
||
* Resonance = **activation value** that guides expansion, selection, and **natural pruning**.
|
||
|
||
---
|
||
|
||
## 11. Observability (what to watch)
|
||
|
||
* **Coverage**: % of turns where Knowledge Frame includes ≥1 concept from the active domain.
|
||
* **Drift**: cosine distance between task query and top‑3 knowledge nodes (want stable closeness).
|
||
* **Utility**: model asks fewer irrelevant questions; compile/test pass rates increase.
|
||
* **Memory hygiene**: working set size stays under target (e.g., < 800 nodes), average resonance > threshold.
|
||
|
||
---
|
||
|
||
## 12. Failure modes & graceful degradation
|
||
|
||
* **Server down** → keep local Working Set; write a **Pending Save** episode; retry `save_context` later.
|
||
* **Search sparse** → fall back to Pack defaults (seed nodes by domain tag).
|
||
* **Prompt over-budget** → trim per type; compress long `payload.details` into bullet summaries.
|
||
* **Bad seeds** → down‑weight sources with low subsequent utility.
|
||
|
||
---
|
||
|
||
## 13. What to change later (server‑side, optional)
|
||
|
||
Only once you want more power centrally:
|
||
|
||
* Add **typed ANN** queries (“top‑K per type”),
|
||
* Add **resonance on server** for multi‑agent sharing,
|
||
* Add **link‑aware search** (expand N hops server‑side),
|
||
* Add **constraints retrieval** (auto‑inject API signatures/tests).
|
||
|
||
Until then, the **client gives you the behavior you want now**.
|
||
|
||
---
|
||
|
||
## 14. TL;DR
|
||
|
||
* Treat memory as a **graph rendered as HDoD**,
|
||
* **Activate** by semantic+episodic seeds; **bloom** 1–2 hops by links; **trim** by type caps,
|
||
* **Feed** the agent *frames* (esp. Knowledge HDoD),
|
||
* **Prefill** with encrypted **Knowledge Packs** (C++ idioms, snippets),
|
||
* Use only your **7 existing endpoints** — intelligence is **client‑side**.
|
||
|
||
---
|
||
|