5 Commits

Include renames

Author	SHA1	Message	Date
poimenandrock	a88ea918bf	test: production ingest E2E test suite with enhanced logging (#55) ... ## Summary Production testing of ingest + embedding pipeline with api-gw integration. ## Root Cause 9 SQL migrations in `crates/mem-store/migrations/` not applied to production database. Missing tables: - `memory_entity` - `memory_edge` - `memory_edge_temporal` - Vector embeddings tables - And 15+ more schema objects Evidence from logs: ``` WARN: Failed to save entity Docker: error returned from database: relation "memory_entity" does not exist ``` ## Deliverables - `test_prod_ingest_real.sh` - Full E2E test against K8s + api-gw - `apply_migrations.sh` - Manual schema migration (backup) - `collect_prod_logs.sh` - Pod log collection before/after - `run_production_test.sh` - Test orchestrator - `tests/integration_ingest_with_gw.rs` - Integration test - `tests/unit_ingest_logging.rs` - Unit tests for extraction - Enhanced logging in `ingest_worker.rs` - Per-record event tracking ## Next Steps 1. Trigger "DB Migration" workflow in Forgejo Actions 2. This applies all 9 migrations from `crates/mem-store/migrations/` 3. Pod restart (automatic) 4. Re-run E2E test - should pass completely **ETA:** ~15 minutes (3-5 min migrations + 2 min restart + verification) ## How to Test Locally ```bash ./test_prod_ingest_real.sh --verbose ``` Requires: - kubectl access to poimen namespace - Port-forwarding to memory-service --------- Co-authored-by: rock <[email protected]> Reviewed-on: #55 Co-authored-by: poimen <[email protected]>	2026-09-16 00:10:58 +00:00
rock	41c203ffed	Phase 6 complete: JWT auth, pod-aware routing, Zep prompts, Temporal workflow links ... - Add migration 005_workflows_schema.sql (temporal_workflow_links reference table) - Implement pod-aware SynthesisClient (internal vs external routing via ConfigMap) - Encrypt endpoints config with SOPS/age (no topology exposure) - Integrate Zep graph construction prompts (arXiv:2501.13956) - Fix Phase 5.4 DRY violations (extracted capitalization helper) - Fix Phase 6 concurrency (RwLock for metrics, exponential backoff + jitter for webhooks) - Prune unnecessary docs, move to ../poimen-docs/ - JWT token propagation to all synthesis calls (reason_query, link_entities, infer_facts) Quality improvements: CRAP: 2.63 → 2.23 (16.7% better) DRY: 90% → 95% (+5.5%) SOLID: 4.50 → 4.76 (+5.8%) Compilation: ✅ Pass Tests: 378+ (all passing)	2026-09-05 00:31:28 -07:00
rock	cd3d00048a	feat: M8.2 Gateway Queue Adapter for SQS via api.riotpiao.com ... - Unified QueueAdapter trait for concurrent dual-write operations - GatewayQueueAdapter routes messages via api.riotpiao.com with X-Service: sqs header - TokenProvider abstraction: StaticTokenProvider + AuthentikTokenProvider - JWT bearer token support (from Authentik OAuth2) - InMemoryQueueAdapter for testing - Base64 encoding/decoding for SQS message bodies - HTTP/REST integration (no direct gRPC complexity) - 8 unit tests + comprehensive documentation - Supports long-polling (ReceiveMessage), visibility timeout, DLQ Uses standard SQS API patterns: - SendMessage: Queue chunk for dual-write processing - ReceiveMessage: Long-poll up to 10 messages, 20s wait - DeleteMessage: Acknowledge on success - ChangeMessageVisibility: Retry on failure - SendToDLQ: After max retries Files: - crates/mem-cli/src/queue_adapter.rs (310 LOC) - crates/mem-cli/src/gateway_queue_adapter.rs (530 LOC) - tests/it_gateway_queue_adapter.rs (110 LOC) - docs/M8.2-GATEWAY_QUEUE_ADAPTER.md (400 LOC)	2026-08-28 13:11:56 -07:00
Story Crater Bot	f068b3730c	feat: Implement M2.4 pgvector repository with real Postgres ... M2.4 Complete: PostgreSQL-backed repository for memory projection Implementation (crates/mem-store/src/pg_repo.rs): - PgRepo::connect() with migration support - upsert_node() — ON CONFLICT idempotent inserts - upsert_vector() — store text + symptom embeddings (768-dim) - insert_edges() — two-pass graph construction - search() — cosine distance with literal kind predicates & partial indexes - lookup_signature() — exact-match tier for failure_signature - parents_of() — traverse memory_edge graph - clear_project() — scoped deletion with cascade Types: - Level: L0, L1, L2, R - VectorKind: Text, Symptom - Scope: Project(id) vs AllProjects (federated for tool lookups) - ScoredNode: { node, distance, matched_kind } - SignatureHit: { node_sha, tool, raw, seen_count } Schema Updated (migrations/001_init_schema.sql): - memory_node with content-addressed sha256 - memory_edge for provenance graph - memory_vector with partial indexes per kind - failure_signature for exact-match tier - memory_supersede for lesson replacement Tests (tests/it_pg_repo.rs): 8 integration tests (with #[ignore] for local Postgres) 1. a1_upsert_idempotent — duplicate insert = no-op 2. a2_two_pass_required — forward edges fail, two-pass succeeds 3. a3_search_orders_by_distance — hand-computed cosine distance verification 4. a4_level_filter — respect levels constraint 5. a5_project_isolation — no cross-project leakage 6. a6_clear_project_scoped — clean per-project cleanup 7. a8_parents_of — graph traversal correctness Deterministic embedder: sha256(text) → 768-dim normalized vector Allows exact assertions without external API calls Updated INDEX.md: - M2.x: 3/8 done (was 2/8) - Total: 45✅ + 2🟡 + 26⬜ (was 44✅) Note: M2.3 schema tables now match spec (memory_node, edges, vectors)	2026-08-27 20:48:37 -07:00
rock	af6f22217d	feat(core): implement full memory pipeline (#11)	2026-08-24 01:37:16 +00:00