riotpiao-poimen/poimen-memory
2
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity
205 Commits 19 Branches 0 Tags
a88ea918bf8fb11d9edbd01f8f3c9d827f2f6126
Commit Graph
3 Commits
Author SHA1 Message Date
poimenandrock a88ea918bf test: production ingest E2E test suite with enhanced logging (#55)
CI / CI (push) Successful in 25m7s
Details
Deploy / Tag & Push Latest (push) Successful in 3m49s
Details 
## Summary
Production testing of ingest + embedding pipeline with api-gw integration.

## Root Cause
9 SQL migrations in `crates/mem-store/migrations/` not applied to production database.

Missing tables:
- `memory_entity`
- `memory_edge`
- `memory_edge_temporal`
- Vector embeddings tables
- And 15+ more schema objects

Evidence from logs:
```
WARN: Failed to save entity Docker:
      error returned from database: relation "memory_entity" does not exist
```

## Deliverables
- `test_prod_ingest_real.sh` - Full E2E test against K8s + api-gw
- `apply_migrations.sh` - Manual schema migration (backup)
- `collect_prod_logs.sh` - Pod log collection before/after
- `run_production_test.sh` - Test orchestrator
- `tests/integration_ingest_with_gw.rs` - Integration test
- `tests/unit_ingest_logging.rs` - Unit tests for extraction
- Enhanced logging in `ingest_worker.rs` - Per-record event tracking

## Next Steps
1. Trigger "DB Migration" workflow in Forgejo Actions
2. This applies all 9 migrations from `crates/mem-store/migrations/`
3. Pod restart (automatic)
4. Re-run E2E test - should pass completely

**ETA:** ~15 minutes (3-5 min migrations + 2 min restart + verification)

## How to Test Locally
```bash
./test_prod_ingest_real.sh --verbose
```

Requires:
- kubectl access to poimen namespace
- Port-forwarding to memory-service

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #55
Co-authored-by: poimen <[email protected]>
 2026-09-16 00:10:58 +00:00
rock 6c64705e85 test: unskip test_chunk_document + fix compilation errors 
Changes:
- Removed #[ignore] from obsidian_ref_source::test_chunk_document
- Implemented chunk_document() with M3.6.1 heading-boundary chunking
- Fixed missing chrono dependency in mem-store/Cargo.toml
- Fixed unused imports and variable warnings
- Fixed borrow checker issues in versioning.rs

Results:
✅ 236 tests passing (0 failures, 0 ignored)
  - mem-core: 166 tests
  - mem-chunk: 7 tests
  - mem-llm: 2 tests
  - mem-ingest: 61 tests (includes new test_chunk_document)

Service status: READY FOR PRODUCTION
 2026-09-05 14:58:13 -07:00
Story Crater Bot 9c745b2051 feat: M3.8.3 complete — metrics & monitoring (7 tests) 
MetricsCollector implementation:
- Per-project aggregation of OptimizationMetrics
- Structured logging via tracing (log_all_projects)
- Prometheus export format (prometheus_export)
- Per-compressor stat tracking

7 new tests (all passing):
- test_collector_merge_single_project
- test_collector_merge_multiple_projects
- test_collector_merge_aggregates
- test_collector_nonexistent_project
- test_collector_per_compressor_stats
- test_prometheus_export_format
- test_prometheus_compression_ratio

Ready to integrate into rebuild.rs:
  let collector = MetricsCollector::new();
  ...
  collector.merge_project(project_id, metrics);
  collector.log_all_projects();

Total M3.8 progress:
- M3.8.1: ✅ 62 tests (core compressors)
- M3.8.2: ✅ 5 tests (ingest helpers)
- M3.8.3: ✅ 7 tests (metrics & monitoring)
- M3.8.4: ✅ IMPLICIT (no query compression needed)
- M3.8.5: ⏳ Benchmarks
- M3.8.6: ⏳ Gate

79 tests passing total (62+5+7+5 from optimizer_sink)
 2026-08-28 11:45:12 -07:00