Section 11 Implementation Plan: Memory & Learning

Complete implementation guide for Forge's memory system — the foundation where all feedback lands and learning accumulates.

---

Overview

The Memory & Learning system is the persistent knowledge layer that transforms ephemeral execution events into durable intelligence. Every feedback loop (Loops 1-5 from the system design) ultimately stores its learnings here. Every agent execution retrieves relevant memories to inform its decisions.

This plan covers the complete implementation from database schema through consolidation jobs.

Build Timeline: Week 1-2 (parallel with Tools layer)

Dependencies:

• Core skeleton (types, bus, schema) ✓
• Event bus for capturing signals ✓
• LLM provider for embeddings and reflection ✓

What this enables:

• Episodic memory: "What happened last time we did this?"
• Semantic memory: "What patterns have we learned?"
• Procedural memory: "How do we solve this type of problem?"
• Smart context building: Inject relevant memories into agent prompts
• Knowledge consolidation: Merge, prune, promote memories over time

---

1. Database Schema (Extended)

The core schema from SYSTEM-DESIGN.md Section 5 needs extension for memory-specific fields.

typescript
// ─── drizzle/schema.ts (additions to existing schema) ─────────

import { sqliteTable, text, integer, real, blob, index } from 'drizzle-orm/sqlite-core';

// ═══ MEMORIES TABLE (already in base schema, showing full detail) ═══
export const memories = sqliteTable('memories', {
  id:          text('id').primaryKey(),              // ulid
  type:        text('type').notNull(),               // episodic | semantic | procedural
  content:     text('content').notNull(),            // Human-readable description
  context:     text('context').notNull(),            // When is this relevant?
  embedding:   blob('embedding'),                    // Float32Array (1536 dims for OpenAI, 768 for local)
  confidence:  real('confidence').notNull(),         // 0.0 - 1.0
  source:      text('source'),                       // Event ID or 'human' or 'production'
  tags:        text('tags', { mode: 'json' }),       // ["typescript", "testing", "auth"]

  // Metadata
  createdAt:   integer('created_at', { mode: 'timestamp_ms' }).notNull(),
  lastAccessed:integer('last_accessed', { mode: 'timestamp_ms' }).notNull(),
  accessCount: integer('access_count').notNull().default(0),

  // Episodic-specific fields (null for other types)
  episodeData: text('episode_data', { mode: 'json' }), // { input, actions, outcome, duration, cost }

  // Semantic-specific fields
  frequency:   integer('frequency'),                 // How often this pattern appears
  successRate: real('success_rate'),                 // 0.0 - 1.0

  // Procedural-specific fields
  trigger:     text('trigger'),                      // What situation activates this procedure?
  steps:       text('steps', { mode: 'json' }),      // Procedure steps

  // Soft delete
  archivedAt:  integer('archived_at', { mode: 'timestamp_ms' }),
}, (table) => ({
  // Indexes for fast queries
  typeIdx: index('memories_type_idx').on(table.type),
  confidenceIdx: index('memories_confidence_idx').on(table.confidence),
  tagsIdx: index('memories_tags_idx').on(table.tags),  // SQLite JSON index
  createdAtIdx: index('memories_created_at_idx').on(table.createdAt),
  lastAccessedIdx: index('memories_last_accessed_idx').on(table.lastAccessed),
}));

// ═══ PATTERNS TABLE (already in base schema) ═══
// Semantic memories promote to patterns when seen 3+ times
export const patterns = sqliteTable('patterns', {
  id:          text('id').primaryKey(),
  type:        text('type').notNull(),               // success | failure | approach
  trigger:     text('trigger').notNull(),            // What situation activates this?
  pattern:     text('pattern').notNull(),            // The pattern itself
  resolution:  text('resolution'),                   // What to do when triggered
  frequency:   integer('frequency').notNull().default(1),
  successRate: real('success_rate'),                 // How often this works
  confidence:  real('confidence').notNull(),
  lastSeen:    integer('last_seen', { mode: 'timestamp_ms' }).notNull(),

  // GEP Protocol fields
  geneId:      text('gene_id'),                      // Maps to GEP gene
  capsuleId:   text('capsule_id'),                   // Maps to GEP capsule
}, (table) => ({
  triggerIdx: index('patterns_trigger_idx').on(table.trigger),
  frequencyIdx: index('patterns_frequency_idx').on(table.frequency),
}));

// ═══ MEMORY_RELATIONSHIPS TABLE (new) ═══
// Tracks which memories are related (for consolidation and recall)
export const memoryRelationships = sqliteTable('memory_relationships', {
  id:           text('id').primaryKey(),
  memoryAId:    text('memory_a_id').notNull().references(() => memories.id),
  memoryBId:    text('memory_b_id').notNull().references(() => memories.id),
  relationship: text('relationship').notNull(),      // 'similar' | 'contradicts' | 'extends' | 'replaces'
  similarity:   real('similarity'),                  // Cosine similarity score
  createdAt:    integer('created_at', { mode: 'timestamp_ms' }).notNull(),
}, (table) => ({
  memoryAIdx: index('memory_relationships_a_idx').on(table.memoryAId),
  memoryBIdx: index('memory_relationships_b_idx').on(table.memoryBId),
}));

---

2. Core Types

typescript
// ─── src/memory/types.ts ──────────────────────────────────────

import type { z } from 'zod';

// ═══ MEMORY TYPES ═══
export type MemoryType = 'episodic' | 'semantic' | 'procedural';

export interface Memory {
  id: string;
  type: MemoryType;
  content: string;
  context: string;
  embedding?: Float32Array;
  confidence: number;
  source?: string;
  tags: string[];
  createdAt: Date;
  lastAccessed: Date;
  accessCount: number;

  // Type-specific data
  episodeData?: EpisodeData;
  frequency?: number;
  successRate?: number;
  trigger?: string;
  steps?: ProcedureStep[];

  archivedAt?: Date;
}

// ═══ EPISODIC MEMORY ═══
export interface EpisodeData {
  input: {
    task: string;
    phase: string;
    context: Record<string, unknown>;
  };
  actions: Action[];
  outcome: {
    success: boolean;
    result?: unknown;
    error?: string;
  };
  duration: number;      // milliseconds
  cost: {
    tokens: number;
    usd: number;
  };
  bounces?: number;      // How many fix cycles
}

export interface Action {
  type: string;
  tool?: string;
  input: unknown;
  output: unknown;
  timestamp: Date;
  duration: number;
}

// ═══ SEMANTIC MEMORY (PATTERNS) ═══
export interface Pattern {
  id: string;
  type: 'success' | 'failure' | 'approach';
  trigger: string;       // "When X happens"
  pattern: string;       // "We observe Y"
  resolution?: string;   // "Do Z"
  frequency: number;
  successRate?: number;
  confidence: number;
  lastSeen: Date;

  // GEP Protocol integration
  geneId?: string;
  capsuleId?: string;
}

// ═══ PROCEDURAL MEMORY ═══
export interface Procedure {
  id: string;
  trigger: string;       // "When implementing auth endpoints"
  steps: ProcedureStep[];
  context: string;
  successRate: number;
  avgDuration: number;
  confidence: number;
  tags: string[];
}

export interface ProcedureStep {
  order: number;
  description: string;
  tool?: string;
  input?: unknown;
  expectedOutcome?: string;
}

// ═══ MEMORY QUERIES ═══
export interface MemoryQuery {
  context: string;             // Natural language description of what we're looking for
  type?: MemoryType;           // Filter by type
  tags?: string[];             // Filter by tags
  minConfidence?: number;      // Filter by confidence threshold
  limit?: number;              // Max results
  embedding?: Float32Array;    // Pre-computed embedding (skip re-embedding)
}

export interface MemoryRecallResult {
  memory: Memory;
  relevance: number;           // Combined score: similarity + recency + confidence
  similarityScore: number;     // Cosine similarity
  recencyScore: number;        // How recent
  confidenceScore: number;     // Memory confidence
}

// ═══ CONFIDENCE SOURCES ═══
export const CONFIDENCE_SOURCES = {
  LLM_EXTRACTED: 0.5,          // Unvalidated AI learning
  HUMAN_CONFIRMED: 0.7,        // Human approved
  PRODUCTION_VALIDATED: 0.9,   // Proven in production
} as const;

// ═══ CONFIDENCE ADJUSTMENTS ═══
export const CONFIDENCE_ADJUSTMENTS = {
  APPLIED: 0.1,                // Pattern successfully applied
  CONFIRMED: 0.2,              // Human confirmed it worked
  DECAY_WEEKLY: 0.05,          // Per week without access
  DISMISSED: -0.2,             // Human dismissed suggestion
} as const;

// ═══ THRESHOLDS ═══
export const THRESHOLDS = {
  AUTO_APPLICATION: 0.8,       // Confidence needed for auto-apply
  ARCHIVE: 0.2,                // Below this, archive the memory
  EPISODIC_TO_PATTERN: 3,      // 3+ similar episodes → promote to pattern
  SIMILARITY_MERGE: 0.9,       // Cosine similarity to merge memories
  ACCESS_STALENESS_DAYS: 90,   // Archive if not accessed in 90 days
} as const;

---

3. MemoryStore Class

The core CRUD interface for all memory operations.

typescript
// ─── src/memory/store.ts ──────────────────────────────────────

import { db } from '@/db';
import { memories, memoryRelationships } from '@/db/schema';
import { eq, and, gte, isNull, sql, desc } from 'drizzle-orm';
import { ulid } from 'ulid';
import type { Memory, MemoryQuery, MemoryRecallResult } from './types';
import { calculateCosineSimilarity } from './similarity';
import { EmbeddingService } from './embeddings';

export class MemoryStore {
  private embeddings: EmbeddingService;

  constructor(embeddings: EmbeddingService) {
    this.embeddings = embeddings;
  }

  // ═══ STORE ═══
  async store(memory: Omit<Memory, 'id' | 'createdAt' | 'lastAccessed' | 'accessCount'>): Promise<string> {
    const id = ulid();
    const now = new Date();

    // Generate embedding if not provided
    let embedding = memory.embedding;
    if (!embedding) {
      const textToEmbed = `${memory.content}\n\nContext: ${memory.context}\nTags: ${memory.tags.join(', ')}`;
      embedding = await this.embeddings.embed(textToEmbed);
    }

    // Convert Float32Array to Buffer for SQLite blob storage
    const embeddingBuffer = Buffer.from(embedding.buffer);

    await db.insert(memories).values({
      id,
      type: memory.type,
      content: memory.content,
      context: memory.context,
      embedding: embeddingBuffer,
      confidence: memory.confidence,
      source: memory.source,
      tags: JSON.stringify(memory.tags),
      createdAt: now.getTime(),
      lastAccessed: now.getTime(),
      accessCount: 0,

      // Type-specific fields
      episodeData: memory.episodeData ? JSON.stringify(memory.episodeData) : null,
      frequency: memory.frequency ?? null,
      successRate: memory.successRate ?? null,
      trigger: memory.trigger ?? null,
      steps: memory.steps ? JSON.stringify(memory.steps) : null,
    });

    // Find and store relationships with similar memories
    await this.findAndStoreRelationships(id, embedding);

    return id;
  }

  // ═══ RECALL (Similarity Search + Tag Matching + Recency Weighting) ═══
  async recall(query: MemoryQuery): Promise<MemoryRecallResult[]> {
    // Generate query embedding
    const queryEmbedding = query.embedding ?? await this.embeddings.embed(query.context);

    // Build base SQL query
    let sqlQuery = db
      .select()
      .from(memories)
      .where(and(
        isNull(memories.archivedAt),
        query.type ? eq(memories.type, query.type) : undefined,
        query.minConfidence ? gte(memories.confidence, query.minConfidence) : undefined,
      ));

    // Tag filtering (SQLite JSON query)
    if (query.tags && query.tags.length > 0) {
      // Filter memories that have ALL specified tags
      sqlQuery = sqlQuery.where(
        sql`json_extract(${memories.tags}, '$') LIKE '%${query.tags[0]}%'`
      );
    }

    const results = await sqlQuery;

    // Calculate relevance scores
    const scored = results.map(mem => {
      const memEmbedding = this.deserializeEmbedding(mem.embedding as Buffer);

      // Similarity score (cosine similarity: -1 to 1, normalized to 0-1)
      const similarityScore = (calculateCosineSimilarity(queryEmbedding, memEmbedding) + 1) / 2;

      // Recency score (exponential decay: newer = higher)
      const ageMs = Date.now() - mem.lastAccessed;
      const ageDays = ageMs / (1000 * 60 * 60 * 24);
      const recencyScore = Math.exp(-ageDays / 30); // Half-life of 30 days

      // Confidence score (already 0-1)
      const confidenceScore = mem.confidence;

      // Combined relevance: weighted average
      const relevance = (
        similarityScore * 0.5 +
        recencyScore * 0.2 +
        confidenceScore * 0.3
      );

      return {
        memory: this.deserializeMemory(mem),
        relevance,
        similarityScore,
        recencyScore,
        confidenceScore,
      };
    });

    // Sort by relevance (highest first)
    scored.sort((a, b) => b.relevance - a.relevance);

    // Limit results
    const limited = scored.slice(0, query.limit ?? 10);

    // Update access tracking
    await Promise.all(
      limited.map(r => this.recordAccess(r.memory.id))
    );

    return limited;
  }

  // ═══ UPDATE ═══
  async update(id: string, changes: Partial<Memory>): Promise<void> {
    const updateData: Record<string, any> = {};

    if (changes.content !== undefined) updateData.content = changes.content;
    if (changes.context !== undefined) updateData.context = changes.context;
    if (changes.confidence !== undefined) updateData.confidence = changes.confidence;
    if (changes.tags !== undefined) updateData.tags = JSON.stringify(changes.tags);

    // Update embedding if content changed
    if (changes.content || changes.context) {
      const memory = await this.get(id);
      if (memory) {
        const textToEmbed = `${changes.content ?? memory.content}\n\nContext: ${changes.context ?? memory.context}\nTags: ${(changes.tags ?? memory.tags).join(', ')}`;
        const embedding = await this.embeddings.embed(textToEmbed);
        updateData.embedding = Buffer.from(embedding.buffer);
      }
    }

    await db
      .update(memories)
      .set(updateData)
      .where(eq(memories.id, id));
  }

  // ═══ ADJUST CONFIDENCE ═══
  async adjustConfidence(id: string, delta: number): Promise<void> {
    const memory = await this.get(id);
    if (!memory) return;

    const newConfidence = Math.max(0, Math.min(1, memory.confidence + delta));

    await db
      .update(memories)
      .set({ confidence: newConfidence })
      .where(eq(memories.id, id));
  }

  // ═══ ARCHIVE (Soft Delete) ═══
  async archive(id: string): Promise<void> {
    await db
      .update(memories)
      .set({ archivedAt: Date.now() })
      .where(eq(memories.id, id));
  }

  // ═══ GET BY ID ═══
  async get(id: string): Promise<Memory | null> {
    const result = await db
      .select()
      .from(memories)
      .where(eq(memories.id, id))
      .limit(1);

    if (result.length === 0) return null;

    return this.deserializeMemory(result[0]);
  }

  // ═══ RECORD ACCESS (Increment Count & Update Timestamp) ═══
  private async recordAccess(id: string): Promise<void> {
    await db
      .update(memories)
      .set({
        lastAccessed: Date.now(),
        accessCount: sql`${memories.accessCount} + 1`,
      })
      .where(eq(memories.id, id));
  }

  // ═══ FIND RELATIONSHIPS ═══
  private async findAndStoreRelationships(newMemoryId: string, embedding: Float32Array): Promise<void> {
    // Find similar existing memories (only top 5)
    const existing = await db
      .select()
      .from(memories)
      .where(and(
        isNull(memories.archivedAt),
        sql`${memories.id} != ${newMemoryId}`,
      ))
      .limit(100); // Check top 100 most recent

    for (const mem of existing) {
      const memEmbedding = this.deserializeEmbedding(mem.embedding as Buffer);
      const similarity = calculateCosineSimilarity(embedding, memEmbedding);

      // Only store if similarity is high enough
      if (similarity > 0.7) {
        await db.insert(memoryRelationships).values({
          id: ulid(),
          memoryAId: newMemoryId,
          memoryBId: mem.id,
          relationship: similarity > 0.9 ? 'similar' : 'related',
          similarity,
          createdAt: Date.now(),
        });
      }
    }
  }

  // ═══ SERIALIZATION HELPERS ═══
  private deserializeEmbedding(buffer: Buffer): Float32Array {
    return new Float32Array(buffer.buffer, buffer.byteOffset, buffer.byteLength / 4);
  }

  private deserializeMemory(row: any): Memory {
    return {
      id: row.id,
      type: row.type,
      content: row.content,
      context: row.context,
      embedding: row.embedding ? this.deserializeEmbedding(row.embedding) : undefined,
      confidence: row.confidence,
      source: row.source,
      tags: JSON.parse(row.tags || '[]'),
      createdAt: new Date(row.createdAt),
      lastAccessed: new Date(row.lastAccessed),
      accessCount: row.accessCount,
      episodeData: row.episodeData ? JSON.parse(row.episodeData) : undefined,
      frequency: row.frequency,
      successRate: row.successRate,
      trigger: row.trigger,
      steps: row.steps ? JSON.parse(row.steps) : undefined,
      archivedAt: row.archivedAt ? new Date(row.archivedAt) : undefined,
    };
  }
}

---

4. Episodic Memory

Records specific execution experiences.

typescript
// ─── src/memory/episodes.ts ───────────────────────────────────

import type { ForgeEvent } from '@/core/types';
import type { Memory, EpisodeData } from './types';
import { MemoryStore } from './store';
import { CONFIDENCE_SOURCES } from './types';

export class EpisodicMemory {
  constructor(private store: MemoryStore) {}

  /**
   * Record a complete episode from a trace of events.
   * An "episode" is one complete agent execution (one pipeline run or one phase).
   */
  async recordEpisode(traceId: string, events: ForgeEvent[]): Promise<string> {
    // Extract episode data from events
    const episodeData = this.extractEpisodeData(events);

    // Generate human-readable summary
    const summary = this.summarizeEpisode(episodeData);

    // Extract tags from episode
    const tags = this.extractTags(episodeData);

    // Determine success/failure
    const outcome = episodeData.outcome.success ? 'success' : 'failure';

    // Store as episodic memory
    const memoryId = await this.store.store({
      type: 'episodic',
      content: summary,
      context: `${episodeData.input.phase} phase: ${episodeData.input.task}`,
      confidence: CONFIDENCE_SOURCES.LLM_EXTRACTED,
      source: traceId,
      tags: [...tags, outcome, episodeData.input.phase],
      episodeData,
    });

    return memoryId;
  }

  /**
   * Find similar past episodes for a new task.
   */
  async findSimilarEpisodes(task: string, phase: string, limit = 5): Promise<Memory[]> {
    const query = `${phase} phase: ${task}`;

    const results = await this.store.recall({
      context: query,
      type: 'episodic',
      limit,
    });

    return results.map(r => r.memory);
  }

  /**
   * Compress an episode for storage (summarize long sequences).
   */
  private summarizeEpisode(episode: EpisodeData): string {
    const { input, actions, outcome, duration, cost } = episode;

    const actionSummary = actions.length <= 5
      ? actions.map(a => `${a.type}(${a.tool || 'N/A'})`).join(' → ')
      : `${actions.length} actions`;

    const outcomeStr = outcome.success
      ? 'succeeded'
      : `failed: ${outcome.error}`;

    return `${input.phase} phase for "${input.task}": ${actionSummary} → ${outcomeStr}. Duration: ${Math.round(duration / 1000)}s, Cost: $${cost.usd.toFixed(4)}`;
  }

  /**
   * Extract episode data from event trace.
   */
  private extractEpisodeData(events: ForgeEvent[]): EpisodeData {
    const startEvent = events[0];
    const endEvent = events[events.length - 1];

    // Extract input
    const input = {
      task: (startEvent.payload as any)?.task || 'unknown',
      phase: startEvent.phase || 'unknown',
      context: (startEvent.payload as any)?.context || {},
    };

    // Extract actions (tool executions)
    const actions = events
      .filter(e => e.type.startsWith('tool.'))
      .map(e => ({
        type: e.type,
        tool: (e.payload as any)?.tool,
        input: (e.payload as any)?.input,
        output: (e.payload as any)?.output,
        timestamp: e.timestamp,
        duration: (e.payload as any)?.duration || 0,
      }));

    // Extract outcome
    const outcome = {
      success: endEvent.type.includes('completed') || endEvent.type.includes('success'),
      result: (endEvent.payload as any)?.result,
      error: (endEvent.payload as any)?.error?.message,
    };

    // Calculate duration
    const duration = endEvent.timestamp.getTime() - startEvent.timestamp.getTime();

    // Calculate cost
    const cost = {
      tokens: events.reduce((sum, e) => sum + (e.cost?.tokens || 0), 0),
      usd: events.reduce((sum, e) => sum + (e.cost?.usd || 0), 0),
    };

    // Count bounces (phase loop iterations)
    const bounces = events.filter(e => e.type === 'loop.phase_bounce').length;

    return { input, actions, outcome, duration, cost, bounces };
  }

  /**
   * Extract tags from episode data.
   */
  private extractTags(episode: EpisodeData): string[] {
    const tags = new Set<string>();

    // Add phase
    tags.add(episode.input.phase);

    // Add tool names
    for (const action of episode.actions) {
      if (action.tool) {
        tags.add(action.tool);
      }
    }

    // Add error categories if failed
    if (!episode.outcome.success && episode.outcome.error) {
      // Simple tag extraction from error message
      if (episode.outcome.error.includes('timeout')) tags.add('timeout');
      if (episode.outcome.error.includes('rate limit')) tags.add('rate-limit');
      if (episode.outcome.error.includes('auth')) tags.add('auth-error');
    }

    return Array.from(tags);
  }
}

---

5. Semantic Memory (Pattern Extraction)

Generalizes from multiple episodes.

typescript
// ─── src/memory/patterns.ts ───────────────────────────────────

import { db } from '@/db';
import { memories, patterns } from '@/db/schema';
import { eq, and, gte, isNull } from 'drizzle-orm';
import { ulid } from 'ulid';
import type { Memory, Pattern } from './types';
import { MemoryStore } from './store';
import { THRESHOLDS } from './types';
import type { LLMProvider } from '@/tools/llm';

export class SemanticMemory {
  constructor(
    private store: MemoryStore,
    private llm: LLMProvider
  ) {}

  /**
   * Promote episodes to patterns when 3+ similar episodes exist.
   * Called by consolidation job.
   */
  async extractPatterns(): Promise<void> {
    // Find groups of similar episodic memories
    const episodicMemories = await db
      .select()
      .from(memories)
      .where(and(
        eq(memories.type, 'episodic'),
        isNull(memories.archivedAt),
        gte(memories.frequency || 0, THRESHOLDS.EPISODIC_TO_PATTERN)
      ));

    // Group by similarity (using relationships)
    const groups = await this.groupSimilarMemories(episodicMemories);

    // Extract pattern from each group
    for (const group of groups) {
      if (group.length >= THRESHOLDS.EPISODIC_TO_PATTERN) {
        await this.createPatternFromEpisodes(group);
      }
    }
  }

  /**
   * Store a new pattern or update existing.
   */
  async storePattern(pattern: Omit<Pattern, 'id' | 'lastSeen'>): Promise<string> {
    const id = ulid();

    await db.insert(patterns).values({
      id,
      type: pattern.type,
      trigger: pattern.trigger,
      pattern: pattern.pattern,
      resolution: pattern.resolution,
      frequency: pattern.frequency,
      successRate: pattern.successRate,
      confidence: pattern.confidence,
      lastSeen: Date.now(),
      geneId: pattern.geneId,
      capsuleId: pattern.capsuleId,
    });

    return id;
  }

  /**
   * Find patterns matching a trigger.
   */
  async findMatchingPatterns(trigger: string): Promise<Pattern[]> {
    const results = await db
      .select()
      .from(patterns)
      .where(
        // Simple substring match for now (could use embedding similarity)
        sql`${patterns.trigger} LIKE '%${trigger}%'`
      )
      .orderBy(desc(patterns.confidence))
      .limit(10);

    return results as Pattern[];
  }

  /**
   * Update pattern confidence and frequency.
   */
  async updatePattern(
    id: string,
    applied: boolean,
    success?: boolean
  ): Promise<void> {
    const pattern = await db
      .select()
      .from(patterns)
      .where(eq(patterns.id, id))
      .limit(1);

    if (pattern.length === 0) return;

    const p = pattern[0];

    // Update frequency
    const newFrequency = p.frequency + 1;

    // Update success rate
    let newSuccessRate = p.successRate || 0;
    if (success !== undefined) {
      const totalSuccesses = (p.successRate || 0) * p.frequency + (success ? 1 : 0);
      newSuccessRate = totalSuccesses / newFrequency;
    }

    // Update confidence
    let confidenceDelta = 0;
    if (applied && success) {
      confidenceDelta = 0.1;
    } else if (applied && success === false) {
      confidenceDelta = -0.2;
    }
    const newConfidence = Math.max(0, Math.min(1, p.confidence + confidenceDelta));

    await db
      .update(patterns)
      .set({
        frequency: newFrequency,
        successRate: newSuccessRate,
        confidence: newConfidence,
        lastSeen: Date.now(),
      })
      .where(eq(patterns.id, id));
  }

  /**
   * Create a pattern from a group of similar episodes using LLM.
   */
  private async createPatternFromEpisodes(episodes: any[]): Promise<void> {
    // Ask LLM to extract pattern
    const prompt = `
You are analyzing ${episodes.length} similar execution episodes to extract a reusable pattern.

Episodes:
${episodes.map((e, i) => `${i + 1}. ${e.content}`).join('\n')}

Extract a pattern in this format:
{
  "type": "success" | "failure" | "approach",
  "trigger": "When [situation description]",
  "pattern": "We observe [common pattern]",
  "resolution": "Do [recommended action]"
}
`.trim();

    const response = await this.llm.chat({
      system: 'You are a pattern extraction assistant. Extract actionable patterns from execution traces.',
      messages: [{ role: 'user', content: prompt }],
      temperature: 0.3,
    });

    try {
      const extracted = JSON.parse(response.content);

      // Calculate success rate from episodes
      const successCount = episodes.filter(e =>
        e.episodeData && JSON.parse(e.episodeData).outcome.success
      ).length;
      const successRate = successCount / episodes.length;

      await this.storePattern({
        type: extracted.type,
        trigger: extracted.trigger,
        pattern: extracted.pattern,
        resolution: extracted.resolution,
        frequency: episodes.length,
        successRate,
        confidence: 0.6, // Medium confidence for LLM-extracted patterns
      });
    } catch (error) {
      console.error('Failed to extract pattern from episodes:', error);
    }
  }

  /**
   * Group memories by similarity using relationships table.
   */
  private async groupSimilarMemories(memories: any[]): Promise<any[][]> {
    // Simple grouping: if two memories are similar (>0.9), put in same group
    const groups: any[][] = [];
    const seen = new Set<string>();

    for (const memory of memories) {
      if (seen.has(memory.id)) continue;

      const group = [memory];
      seen.add(memory.id);

      // Find all similar memories
      const similar = memories.filter(m =>
        !seen.has(m.id) && this.areSimilar(memory, m)
      );

      for (const sim of similar) {
        group.push(sim);
        seen.add(sim.id);
      }

      if (group.length > 1) {
        groups.push(group);
      }
    }

    return groups;
  }

  private areSimilar(a: any, b: any): boolean {
    // Simple heuristic: same phase and same tags
    const aData = a.episodeData ? JSON.parse(a.episodeData) : null;
    const bData = b.episodeData ? JSON.parse(b.episodeData) : null;

    if (!aData || !bData) return false;

    return aData.input.phase === bData.input.phase;
  }
}

---

6. Procedural Memory (Strategies)

Stores how-to knowledge.

typescript
// ─── src/memory/procedures.ts ─────────────────────────────────

import type { Memory, Procedure, ProcedureStep } from './types';
import { MemoryStore } from './store';
import { CONFIDENCE_SOURCES } from './types';

export class ProceduralMemory {
  constructor(private store: MemoryStore) {}

  /**
   * Learn a procedure from a successful execution.
   */
  async learnProcedure(
    trigger: string,
    steps: ProcedureStep[],
    context: string,
    tags: string[],
    successRate = 1.0
  ): Promise<string> {
    const content = this.formatProcedure(trigger, steps);

    return this.store.store({
      type: 'procedural',
      content,
      context,
      confidence: CONFIDENCE_SOURCES.LLM_EXTRACTED,
      tags,
      trigger,
      steps,
      successRate,
    });
  }

  /**
   * Find procedures matching a trigger.
   */
  async findProcedures(trigger: string, limit = 5): Promise<Memory[]> {
    const results = await this.store.recall({
      context: trigger,
      type: 'procedural',
      limit,
    });

    return results.map(r => r.memory);
  }

  /**
   * Record procedure application and outcome.
   */
  async recordApplication(
    procedureId: string,
    success: boolean,
    duration?: number
  ): Promise<void> {
    const memory = await this.store.get(procedureId);
    if (!memory || memory.type !== 'procedural') return;

    // Update success rate
    const currentRate = memory.successRate || 0.5;
    const newRate = (currentRate + (success ? 1 : 0)) / 2; // Simple moving average

    await this.store.update(procedureId, {
      successRate: newRate,
    });

    // Adjust confidence
    if (success) {
      await this.store.adjustConfidence(procedureId, 0.1);
    } else {
      await this.store.adjustConfidence(procedureId, -0.2);
    }
  }

  /**
   * Format procedure as human-readable text.
   */
  private formatProcedure(trigger: string, steps: ProcedureStep[]): string {
    const stepText = steps
      .sort((a, b) => a.order - b.order)
      .map(s => `${s.order}. ${s.description}${s.tool ? ` (using ${s.tool})` : ''}`)
      .join('\n');

    return `When ${trigger}:\n${stepText}`;
  }
}

---

7. Embedding Generation

Converts text to vectors for similarity search.

typescript
// ─── src/memory/embeddings.ts ─────────────────────────────────

import type { LLMProvider } from '@/tools/llm';

export class EmbeddingService {
  private cache = new Map<string, Float32Array>();
  private batchQueue: Array<{ text: string; resolve: (emb: Float32Array) => void }> = [];
  private batchTimer?: NodeJS.Timeout;

  constructor(private llm: LLMProvider) {}

  /**
   * Embed a single text (with caching).
   */
  async embed(text: string): Promise<Float32Array> {
    // Check cache
    const cached = this.cache.get(text);
    if (cached) return cached;

    // Call LLM embedding API
    const embedding = await this.llm.embed(text);

    // Cache
    this.cache.set(text, embedding);

    return embedding;
  }

  /**
   * Batch embed multiple texts (more efficient).
   */
  async batchEmbed(texts: string[]): Promise<Float32Array[]> {
    // For now, simple sequential embedding
    // TODO: Implement actual batch API call if provider supports it
    return Promise.all(texts.map(t => this.embed(t)));
  }

  /**
   * Clear cache (useful for testing or memory management).
   */
  clearCache(): void {
    this.cache.clear();
  }
}

---

8. Similarity Search

Cosine similarity without a vector DB.

typescript
// ─── src/memory/similarity.ts ─────────────────────────────────

/**
 * Calculate cosine similarity between two vectors.
 * Returns value between -1 (opposite) and 1 (identical).
 */
export function calculateCosineSimilarity(
  a: Float32Array,
  b: Float32Array
): number {
  if (a.length !== b.length) {
    throw new Error('Vectors must have same dimension');
  }

  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  for (let i = 0; i < a.length; i++) {
    dotProduct += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }

  const denominator = Math.sqrt(normA) * Math.sqrt(normB);

  if (denominator === 0) return 0;

  return dotProduct / denominator;
}

/**
 * Find top-K most similar vectors from a set.
 */
export function topKSimilar(
  query: Float32Array,
  candidates: Array<{ id: string; embedding: Float32Array }>,
  k: number
): Array<{ id: string; similarity: number }> {
  // Calculate similarity for all candidates
  const scores = candidates.map(c => ({
    id: c.id,
    similarity: calculateCosineSimilarity(query, c.embedding),
  }));

  // Sort by similarity (descending)
  scores.sort((a, b) => b.similarity - a.similarity);

  // Return top K
  return scores.slice(0, k);
}

/**
 * Filter by similarity threshold.
 */
export function filterBySimilarity(
  query: Float32Array,
  candidates: Array<{ id: string; embedding: Float32Array }>,
  threshold: number
): Array<{ id: string; similarity: number }> {
  return candidates
    .map(c => ({
      id: c.id,
      similarity: calculateCosineSimilarity(query, c.embedding),
    }))
    .filter(s => s.similarity >= threshold);
}

Performance Note: Brute-force cosine similarity is O(n*d) where n=memories, d=dimensions. For SQLite with <100K memories and 768-1536 dimensions, this is ~10-100ms per query on modern hardware. Only migrate to a vector DB (pgvector, Qdrant) if query time exceeds 500ms or memory count > 100K.

---

9. Confidence System (Complete)

typescript
// ─── src/memory/confidence.ts ─────────────────────────────────

import type { Memory } from './types';
import { CONFIDENCE_SOURCES, CONFIDENCE_ADJUSTMENTS, THRESHOLDS } from './types';

export class ConfidenceManager {
  /**
   * Calculate initial confidence based on source.
   */
  static initialConfidence(source: string): number {
    if (source === 'human') return CONFIDENCE_SOURCES.HUMAN_CONFIRMED;
    if (source === 'production') return CONFIDENCE_SOURCES.PRODUCTION_VALIDATED;
    return CONFIDENCE_SOURCES.LLM_EXTRACTED;
  }

  /**
   * Apply confidence adjustment.
   */
  static adjust(
    current: number,
    event: 'applied' | 'confirmed' | 'dismissed'
  ): number {
    let delta = 0;

    switch (event) {
      case 'applied':
        delta = CONFIDENCE_ADJUSTMENTS.APPLIED;
        break;
      case 'confirmed':
        delta = CONFIDENCE_ADJUSTMENTS.CONFIRMED;
        break;
      case 'dismissed':
        delta = CONFIDENCE_ADJUSTMENTS.DISMISSED;
        break;
    }

    return Math.max(0, Math.min(1, current + delta));
  }

  /**
   * Calculate weekly decay.
   */
  static weeklyDecay(
    confidence: number,
    lastAccessedDate: Date,
    now = new Date()
  ): number {
    const msPerWeek = 7 * 24 * 60 * 60 * 1000;
    const weeksSinceAccess = (now.getTime() - lastAccessedDate.getTime()) / msPerWeek;

    // Exponential decay
    const decayAmount = CONFIDENCE_ADJUSTMENTS.DECAY_WEEKLY * weeksSinceAccess;

    return Math.max(0, confidence - decayAmount);
  }

  /**
   * Should this memory be auto-applied?
   */
  static shouldAutoApply(confidence: number): boolean {
    return confidence >= THRESHOLDS.AUTO_APPLICATION;
  }

  /**
   * Should this memory be archived?
   */
  static shouldArchive(memory: Memory): boolean {
    // Low confidence
    if (memory.confidence < THRESHOLDS.ARCHIVE) return true;

    // Not accessed in 90 days
    const daysSinceAccess = (Date.now() - memory.lastAccessed.getTime()) / (1000 * 60 * 60 * 24);
    if (daysSinceAccess > THRESHOLDS.ACCESS_STALENESS_DAYS) return true;

    return false;
  }
}

---

10. Consolidation Job

Periodic maintenance: merge, promote, prune.

typescript
// ─── src/memory/consolidate.ts ────────────────────────────────

import { db } from '@/db';
import { memories, patterns, memoryRelationships } from '@/db/schema';
import { and, isNull, lt, gte } from 'drizzle-orm';
import { MemoryStore } from './store';
import { SemanticMemory } from './patterns';
import { ConfidenceManager } from './confidence';
import { THRESHOLDS } from './types';
import type { LLMProvider } from '@/tools/llm';
import { EmbeddingService } from './embeddings';

export class ConsolidationJob {
  private store: MemoryStore;
  private semantic: SemanticMemory;

  constructor(llm: LLMProvider) {
    const embeddings = new EmbeddingService(llm);
    this.store = new MemoryStore(embeddings);
    this.semantic = new SemanticMemory(this.store, llm);
  }

  /**
   * Run full consolidation cycle.
   * Schedule: daily at 2 AM or on-demand.
   */
  async consolidate(): Promise<ConsolidationReport> {
    console.log('[Consolidation] Starting memory consolidation...');

    const report: ConsolidationReport = {
      startTime: new Date(),
      merged: 0,
      promoted: 0,
      pruned: 0,
      decayed: 0,
      conflicts: 0,
    };

    // 1. Merge similar memories
    report.merged = await this.mergeSimilarMemories();

    // 2. Promote frequent episodes to patterns
    report.promoted = await this.promoteToPatterns();

    // 3. Apply decay to stale memories
    report.decayed = await this.applyDecay();

    // 4. Prune low-confidence and old memories
    report.pruned = await this.pruneMemories();

    // 5. Resolve conflicts
    report.conflicts = await this.resolveConflicts();

    report.endTime = new Date();
    report.duration = report.endTime.getTime() - report.startTime.getTime();

    console.log('[Consolidation] Complete:', report);

    return report;
  }

  /**
   * Merge memories with similarity > 0.9.
   */
  private async mergeSimilarMemories(): Promise<number> {
    // Find all similar relationships above merge threshold
    const similarPairs = await db
      .select()
      .from(memoryRelationships)
      .where(
        and(
          gte(memoryRelationships.similarity || 0, THRESHOLDS.SIMILARITY_MERGE),
        )
      );

    let merged = 0;

    for (const pair of similarPairs) {
      const memA = await this.store.get(pair.memoryAId);
      const memB = await this.store.get(pair.memoryBId);

      if (!memA || !memB || memA.archivedAt || memB.archivedAt) continue;

      // Merge: keep higher confidence, combine tags, archive the other
      const keep = memA.confidence >= memB.confidence ? memA : memB;
      const discard = memA.confidence >= memB.confidence ? memB : memA;

      // Combine tags
      const combinedTags = Array.from(new Set([...keep.tags, ...discard.tags]));

      // Update kept memory
      await this.store.update(keep.id, {
        tags: combinedTags,
        confidence: Math.min(1, keep.confidence + 0.05), // Small boost for redundancy
      });

      // Archive discarded memory
      await this.store.archive(discard.id);

      merged++;
    }

    return merged;
  }

  /**
   * Promote episodes to patterns when frequency >= 3.
   */
  private async promoteToPatterns(): Promise<number> {
    await this.semantic.extractPatterns();

    // Count newly created patterns (simplified)
    const newPatterns = await db
      .select()
      .from(patterns)
      .where(
        gte(patterns.lastSeen, Date.now() - 60 * 60 * 1000) // Created in last hour
      );

    return newPatterns.length;
  }

  /**
   * Apply weekly decay to all memories.
   */
  private async applyDecay(): Promise<number> {
    const allMemories = await db
      .select()
      .from(memories)
      .where(isNull(memories.archivedAt));

    let decayed = 0;

    for (const mem of allMemories) {
      const newConfidence = ConfidenceManager.weeklyDecay(
        mem.confidence,
        new Date(mem.lastAccessed)
      );

      if (newConfidence !== mem.confidence) {
        await db
          .update(memories)
          .set({ confidence: newConfidence })
          .where(eq(memories.id, mem.id));

        decayed++;
      }
    }

    return decayed;
  }

  /**
   * Prune memories that should be archived.
   */
  private async pruneMemories(): Promise<number> {
    const allMemories = await db
      .select()
      .from(memories)
      .where(isNull(memories.archivedAt));

    let pruned = 0;

    for (const mem of allMemories) {
      const memory = await this.store.get(mem.id);
      if (!memory) continue;

      if (ConfidenceManager.shouldArchive(memory)) {
        await this.store.archive(memory.id);
        pruned++;
      }
    }

    return pruned;
  }

  /**
   * Resolve conflicting memories (keep highest confidence).
   */
  private async resolveConflicts(): Promise<number> {
    // Find contradictory relationships
    const contradictions = await db
      .select()
      .from(memoryRelationships)
      .where(eq(memoryRelationships.relationship, 'contradicts'));

    let resolved = 0;

    for (const contradiction of contradictions) {
      const memA = await this.store.get(contradiction.memoryAId);
      const memB = await this.store.get(contradiction.memoryBId);

      if (!memA || !memB) continue;

      // Archive lower confidence
      if (memA.confidence > memB.confidence) {
        await this.store.archive(memB.id);
      } else {
        await this.store.archive(memA.id);
      }

      resolved++;
    }

    return resolved;
  }
}

interface ConsolidationReport {
  startTime: Date;
  endTime?: Date;
  duration?: number;
  merged: number;
  promoted: number;
  pruned: number;
  decayed: number;
  conflicts: number;
}

---

11. Smart Context Builder

Assembles memories into agent prompts.

typescript
// ─── src/memory/context-builder.ts ────────────────────────────

import type { Memory, MemoryRecallResult } from './types';
import { MemoryStore } from './store';

export class SmartContextBuilder {
  constructor(private store: MemoryStore) {}

  /**
   * Build context for an agent prompt.
   *
   * Budget allocation:
   * - 60% memories
   * - 30% conversation history
   * - 10% environment state
   */
  async buildContext(
    task: string,
    conversationHistory: string,
    environmentState: string,
    maxTokens: number
  ): Promise<string> {
    const memoryBudget = Math.floor(maxTokens * 0.6);
    const conversationBudget = Math.floor(maxTokens * 0.3);
    const environmentBudget = Math.floor(maxTokens * 0.1);

    // Recall relevant memories
    const memories = await this.store.recall({
      context: task,
      limit: 20,
    });

    // Select top memories within budget
    const selectedMemories = this.selectMemoriesWithinBudget(memories, memoryBudget);

    // Truncate conversation if needed
    const truncatedConversation = this.truncateToTokens(conversationHistory, conversationBudget);

    // Truncate environment state if needed
    const truncatedEnvironment = this.truncateToTokens(environmentState, environmentBudget);

    // Assemble context
    return this.assembleContext(selectedMemories, truncatedConversation, truncatedEnvironment);
  }

  /**
   * Select memories that fit within token budget.
   */
  private selectMemoriesWithinBudget(
    memories: MemoryRecallResult[],
    budget: number
  ): Memory[] {
    const selected: Memory[] = [];
    let tokensUsed = 0;

    // Sort by relevance (already done by recall)
    for (const result of memories) {
      const estimatedTokens = this.estimateTokens(result.memory.content);

      if (tokensUsed + estimatedTokens <= budget) {
        selected.push(result.memory);
        tokensUsed += estimatedTokens;
      } else {
        // Try to summarize and fit
        const summary = this.summarizeMemory(result.memory);
        const summaryTokens = this.estimateTokens(summary);

        if (tokensUsed + summaryTokens <= budget) {
          // Store summary as temporary memory
          selected.push({
            ...result.memory,
            content: summary,
          });
          tokensUsed += summaryTokens;
        } else {
          // Can't fit, stop
          break;
        }
      }
    }

    return selected;
  }

  /**
   * Assemble final context string.
   */
  private assembleContext(
    memories: Memory[],
    conversation: string,
    environment: string
  ): string {
    const parts: string[] = [];

    // Environment state
    if (environment) {
      parts.push('## Environment\n' + environment);
    }

    // Relevant memories
    if (memories.length > 0) {
      parts.push('## Relevant Past Experiences\n');
      for (const mem of memories) {
        parts.push(`- [${mem.type}] ${mem.content} (confidence: ${mem.confidence.toFixed(2)})`);
      }
    }

    // Conversation history
    if (conversation) {
      parts.push('## Recent Conversation\n' + conversation);
    }

    return parts.join('\n\n');
  }

  /**
   * Estimate token count (rough approximation).
   * Real implementation should use tiktoken or similar.
   */
  private estimateTokens(text: string): number {
    // Rough estimate: 1 token ≈ 4 characters
    return Math.ceil(text.length / 4);
  }

  /**
   * Truncate text to fit token budget.
   */
  private truncateToTokens(text: string, budget: number): string {
    const estimatedTokens = this.estimateTokens(text);

    if (estimatedTokens <= budget) return text;

    // Truncate to budget (rough)
    const targetLength = budget * 4;
    return text.slice(0, targetLength) + '...';
  }

  /**
   * Summarize a memory to save tokens.
   */
  private summarizeMemory(memory: Memory): string {
    // Simple summarization: first sentence + outcome
    const firstSentence = memory.content.split('.')[0];
    return `${firstSentence}. (${memory.type}, confidence ${memory.confidence.toFixed(2)})`;
  }
}

---

12. GEP Protocol Integration

Maps the Genome Evolution Protocol to our memory system.

typescript
// ─── src/memory/gep.ts ────────────────────────────────────────

import type { Pattern } from './types';
import { SemanticMemory } from './patterns';

/**
 * GEP Protocol: Genome Evolution Protocol
 *
 * Maps patterns to "genes" and "capsules":
 * - Gene: Trigger → behavioral rule
 * - Capsule: Context → solution mapping
 */
export class GEPProtocol {
  constructor(private semantic: SemanticMemory) {}

  /**
   * Create a gene from a pattern.
   * A gene is a trigger-based behavioral rule.
   */
  async createGene(
    trigger: string,
    pattern: string,
    priority = 1
  ): Promise<string> {
    return this.semantic.storePattern({
      type: 'approach',
      trigger,
      pattern,
      resolution: pattern,
      frequency: 1,
      confidence: 0.5,
      geneId: `gene-${Date.now()}`,
    });
  }

  /**
   * Create a capsule from a pattern.
   * A capsule is a context → solution mapping.
   */
  async createCapsule(
    context: string,
    solution: string
  ): Promise<string> {
    return this.semantic.storePattern({
      type: 'success',
      trigger: context,
      pattern: solution,
      resolution: solution,
      frequency: 1,
      confidence: 0.6,
      capsuleId: `capsule-${Date.now()}`,
    });
  }

  /**
   * Record gene application event.
   */
  async recordGeneApplication(
    geneId: string,
    signals: Array<{ type: string; severity: string }>,
    success: boolean
  ): Promise<void> {
    // Find pattern with this gene ID
    const patterns = await this.semantic.findMatchingPatterns(geneId);

    for (const pattern of patterns) {
      if (pattern.geneId === geneId) {
        await this.semantic.updatePattern(pattern.id, true, success);
      }
    }
  }

  /**
   * Mutate a gene based on outcomes.
   * If a gene keeps failing, reduce its confidence or modify its trigger.
   */
  async mutateGene(geneId: string, outcomes: boolean[]): Promise<void> {
    const successRate = outcomes.filter(Boolean).length / outcomes.length;

    const patterns = await this.semantic.findMatchingPatterns(geneId);

    for (const pattern of patterns) {
      if (pattern.geneId === geneId) {
        // If success rate < 30%, mutate (reduce confidence significantly)
        if (successRate < 0.3) {
          await this.semantic.updatePattern(pattern.id, false, false);
        }
      }
    }
  }
}

---

13. Integration with Agent Loop

How agents use the memory system.

typescript
// ─── Example: Integration in BaseAgent ────────────────────────

import { SmartContextBuilder } from '@/memory/context-builder';
import { ProceduralMemory } from '@/memory/procedures';
import { EpisodicMemory } from '@/memory/episodes';

abstract class BaseAgent {
  private contextBuilder: SmartContextBuilder;
  private procedures: ProceduralMemory;
  private episodes: EpisodicMemory;

  async execute(input: PhaseInput, ctx: AgentContext): Promise<PhaseOutput> {
    let iteration = 0;

    // ── PERCEIVE: Build context with memories ──
    const context = await this.contextBuilder.buildContext(
      input.task,
      this.formatConversationHistory(ctx),
      this.formatEnvironment(ctx),
      ctx.config.maxContextTokens || 100_000
    );

    // Find relevant procedures
    const procedures = await this.procedures.findProcedures(input.task);

    let workingMemory = {
      context,
      procedures,
      messages: [{ role: 'user', content: this.buildPrompt(input, context, procedures) }],
    };

    while (true) {
      iteration++;

      // ... rest of agent loop ...

      // ── LEARN: After completion ──
      if (decision.done) {
        // Record episode
        const events = await ctx.bus.replay(ctx.traceId);
        await this.episodes.recordEpisode(ctx.traceId, events);

        // If we used a procedure, record outcome
        if (procedureUsed) {
          await this.procedures.recordApplication(procedureUsed.id, true);
        }

        return decision.result;
      }
    }
  }
}

---

14. Testing Strategy

typescript
// ─── tests/memory/store.test.ts ───────────────────────────────

import { describe, it, expect, beforeEach } from 'bun:test';
import { MemoryStore } from '@/memory/store';
import { EmbeddingService } from '@/memory/embeddings';
import { MockLLMProvider } from '@/tests/mocks/llm';

describe('MemoryStore', () => {
  let store: MemoryStore;

  beforeEach(() => {
    const llm = new MockLLMProvider();
    const embeddings = new EmbeddingService(llm);
    store = new MemoryStore(embeddings);
  });

  it('should store and retrieve a memory', async () => {
    const id = await store.store({
      type: 'episodic',
      content: 'Test memory',
      context: 'Testing context',
      confidence: 0.8,
      tags: ['test'],
    });

    const memory = await store.get(id);
    expect(memory).toBeTruthy();
    expect(memory?.content).toBe('Test memory');
  });

  it('should recall similar memories', async () => {
    // Store multiple memories
    await store.store({
      type: 'semantic',
      content: 'Authentication uses JWT tokens',
      context: 'auth implementation',
      confidence: 0.9,
      tags: ['auth', 'jwt'],
    });

    await store.store({
      type: 'semantic',
      content: 'Database uses PostgreSQL',
      context: 'database setup',
      confidence: 0.8,
      tags: ['database', 'postgres'],
    });

    // Recall auth-related memories
    const results = await store.recall({
      context: 'implementing authentication',
      limit: 5,
    });

    expect(results.length).toBeGreaterThan(0);
    expect(results[0].memory.tags).toContain('auth');
  });

  it('should adjust confidence', async () => {
    const id = await store.store({
      type: 'procedural',
      content: 'Test procedure',
      context: 'test',
      confidence: 0.5,
      tags: [],
    });

    await store.adjustConfidence(id, 0.2);

    const memory = await store.get(id);
    expect(memory?.confidence).toBe(0.7);
  });

  it('should archive low-confidence memories', async () => {
    const id = await store.store({
      type: 'episodic',
      content: 'Low confidence memory',
      context: 'test',
      confidence: 0.1,
      tags: [],
    });

    await store.archive(id);

    const memory = await store.get(id);
    expect(memory?.archivedAt).toBeTruthy();
  });
});

---

15. Configuration

typescript
// ─── forge.config.ts (memory section) ─────────────────────────

export default defineConfig({
  // ... other config ...

  memory: {
    // Database
    dbPath: '.forge/memory.db',

    // Embedding model
    embeddingModel: 'text-embedding-3-small', // OpenAI
    embeddingDimensions: 1536,

    // Recall settings
    maxRecallResults: 10,
    minRelevanceScore: 0.5,

    // Confidence thresholds
    autoApplicationThreshold: 0.8,
    archiveThreshold: 0.2,

    // Consolidation
    consolidateInterval: '1d',        // Run daily
    consolidateTime: '02:00',         // At 2 AM
    maxMemories: 100_000,

    // Context building
    memoryBudgetPercent: 60,          // 60% of context for memories
    conversationBudgetPercent: 30,
    environmentBudgetPercent: 10,
  },
});

---

16. CLI Commands

typescript
// ─── src/cli/commands/memory.ts ───────────────────────────────

import { Command } from 'commander';
import { MemoryStore } from '@/memory/store';
import { ConsolidationJob } from '@/memory/consolidate';

export const memoryCommand = new Command('memory')
  .description('Memory management commands');

memoryCommand
  .command('list')
  .option('-t, --type <type>', 'Filter by type (episodic|semantic|procedural)')
  .option('-l, --limit <number>', 'Max results', '10')
  .description('List memories')
  .action(async (options) => {
    // Implementation
  });

memoryCommand
  .command('search <query>')
  .option('-l, --limit <number>', 'Max results', '10')
  .description('Search memories by context')
  .action(async (query, options) => {
    // Implementation
  });

memoryCommand
  .command('consolidate')
  .description('Run memory consolidation')
  .action(async () => {
    const job = new ConsolidationJob(llmProvider);
    const report = await job.consolidate();
    console.log('Consolidation complete:', report);
  });

memoryCommand
  .command('stats')
  .description('Show memory statistics')
  .action(async () => {
    // Show:
    // - Total memories by type
    // - Average confidence
    // - Top tags
    // - Database size
  });

---

17. Observability & Metrics

typescript
// ─── src/memory/metrics.ts ────────────────────────────────────

export interface MemoryMetrics {
  // Capacity
  totalMemories: number;
  memoriesByType: Record<MemoryType, number>;
  databaseSizeBytes: number;

  // Quality
  averageConfidence: number;
  confidenceDistribution: Record<string, number>; // Histogram
  topTags: Array<{ tag: string; count: number }>;

  // Usage
  recallsPerDay: number;
  averageRecallTime: number; // ms
  memoryHitRate: number;     // % of recalls that returned results

  // Consolidation
  lastConsolidation: Date;
  consolidationDuration: number;
  memoriesMerged: number;
  memoriesPruned: number;

  // Learning
  patternsExtracted: number;
  proceduresLearned: number;
  episodesRecorded: number;
}

export class MemoryMetricsCollector {
  async collect(): Promise<MemoryMetrics> {
    // Query database for metrics
    // Return aggregated stats
  }
}

---

18. Build Checklist

Week 1: Foundation

• Database schema with indexes
• Core types and interfaces
• MemoryStore class (store, recall, update, archive)
• EmbeddingService with caching
• Similarity search utilities
• Unit tests for core operations

Week 2: Memory Types & Integration

• EpisodicMemory implementation
• SemanticMemory (pattern extraction)
• ProceduralMemory
• ConfidenceManager
• SmartContextBuilder
• Integration tests with mock LLM

Week 3: Consolidation & CLI

• ConsolidationJob (merge, promote, prune, decay)
• GEP Protocol integration
• CLI commands (list, search, consolidate, stats)
• Cron job setup for daily consolidation
• End-to-end tests

Week 4: Polish & Optimization

• Performance profiling (recall speed)
• Memory metrics dashboard
• Documentation
• Edge case handling
• Production readiness review

---

19. Success Criteria

The memory system is done when:

1. Agents can recall relevant memories: Given a task, recall returns applicable past experiences within 100ms
2. Learning accumulates: After 10 executions of similar tasks, semantic patterns emerge
3. Confidence calibrates: Dismissed memories decay, applied memories strengthen
4. Consolidation runs cleanly: Daily job completes in <5 minutes, reduces redundancy
5. Context fits budget: SmartContextBuilder never exceeds token limits
6. Tests pass: 90%+ coverage on core memory operations

---

20. Future Enhancements (Post-MVP)

• Vector database migration: Switch to pgvector or Qdrant when memory > 100K
• Cross-session knowledge sharing: Share learnings across multiple Forge instances
• Memory visualization: UI to browse, search, and curate memories
• Meta-learning: Learn which memory retrieval strategies work best
• Adaptive consolidation: Schedule based on memory growth rate, not fixed time
• Semantic memory graphs: Link related patterns and procedures
• Personalization: User-specific memory spaces with different confidence models

---

Conclusion

This implementation plan provides a complete, buildable specification for Forge's Memory & Learning system. It covers:

• All three memory types (episodic, semantic, procedural)
• Complete CRUD operations with similarity search
• Confidence scoring and decay
• Knowledge consolidation (merge, promote, prune)
• Smart context building for agent prompts
• GEP Protocol integration for evolving patterns

The system is designed to start simple (SQLite + brute-force similarity) and scale up only when needed (vector DB at 100K+ memories). Every feedback loop (1-5) ultimately lands here, making this the foundational knowledge layer for the entire Forge system.

Build priority: Week 1-2 (parallel with Tools layer), because agents need memory from day one to be effective.