Commit Graph
4 Commits
Author SHA1 Message Date
rock b514c43d6b fix: add missing dev-dependencies + fix test assertion
CI / CI (pull_request) Failing after 21m8s
- Added tracing, tracing-subscriber, reqwest, uuid to [dev-dependencies]
  (test files referenced these crates but they weren't available)
- Fixed test assertion: WikiLinkFallbackExtractor extracts [[Concurrency]]
  not 'Go' from text 'Go [[Concurrency]] is powerful'
2026-09-15 13:57:58 +09:00
rock 3f83c1b015 refactor: complete SOLID fixes + RAII guards + enhanced test assertions
CI / CI (pull_request) Failing after 21m18s
ISP (Interface Segregation Principle):
  - Add JobStatusStore trait for database persistence
  - Implement PgJobStatusStore for PostgreSQL
  - Add MockJobStatusStore for unit testing
  - IngestWorker::with_job_store() enables dependency injection
  - Job status updates now via trait (testable, mockable)

Resource Management (RAII):
  - Add PortForwardGuard struct with Drop impl
  - Ensures port-forward process killed even if test panics
  - Prevents resource leaks in integration tests

Test Assertions (Verification):
  - Enhanced unit tests verify entity names, not just counts
  - Verify edges connect correct entity pairs
  - Verify both source and target entities exist
  - Batch processing verifies expected entities extracted
  - Entity deduplication test across multiple records

All 9 issues now fixed:
   DRY: Removed wrapper function
   CRAP: Extracted save_*_with_logging() helpers
   OCP: Added JobStatus enum
   SRP: Real-time logging, no error accumulation
   ISP: JobStatusStore trait + mocking
   Logging: IngestLogContext struct
   RAII: PortForwardGuard for cleanup
   Error handling: Real-time logging at point of failure
   Test assertions: Verify names + connections

Verification: cargo check -p mem-cli ✓
2026-09-15 13:30:19 +09:00
rock ce6c93d3b5 refactor: focus on K8s Job integration testing, remove random scripts
CI / CI (pull_request) Successful in 15m38s
Remove unfocused shell scripts - rely on existing integration tests instead:
  - ✓ tests/it_unified_query_4_6.rs (query tests)
  - ✓ tests/it_temporal_filtering_4_2_fixed.rs (temporal query)
  - ✓ tests/it_phase3_phase4.rs (ingest tests)
  - ✓ tests/it_authorized_pipeline.rs (auth + ingest)

Removed:
  - apply_migrations.sh (use migrations/ runner script)
  - collect_prod_logs.sh (k8s logs available)
  - run_production_test.sh (use cargo test)
  - test_prod_ingest_real.sh (existing it_phase3_phase4.rs)
  - tests/integration_ingest_with_gw.rs (duplicate)
  - tests/unit_ingest_logging.rs (duplicate)

Keep:
  - migrations/run_migrations.sh (K8s Job requirement)
  - k8s/test/integration-test-job.yaml (CI/CD integration)
  - .gitea/workflows/integration-test.yaml (CI orchestration)
  - k8s/test/db-credentials.enc.yaml (SOPS encrypted secrets)

Proper approach: K8s Job runs existing integration tests via 'cargo test'
                 ArgoCD+KSOPS decrypts secrets
                 Tests execute against new image SHA
2026-09-14 22:55:58 +09:00
rock 6915dc2462 feat: production ingest test suite with detailed logging
Add comprehensive E2E test scripts and logging for production testing:

- test_prod_ingest_real.sh: Full ingest test against K8s cluster with api-gw
- apply_migrations.sh: Manual database schema migration (backup method)
- collect_prod_logs.sh: Pod log collection before/after tests
- run_production_test.sh: Orchestrates full test + log collection
- tests/integration_ingest_with_gw.rs: Integration test with embeddings
- tests/unit_ingest_logging.rs: Unit tests for extraction pipeline

Enhanced logging in ingest_worker.rs:
- Per-record event tracking (extraction, save)
- Entity and edge operation logging
- Error accumulation and reporting
- Structured logging for observability

Production testing identified root cause:
- Ingest + embedding pipeline working correctly
- Entity extraction functional
- Database schema missing (migration not applied)
- Logs clearly show: relation "memory_entity" does not exist

Next: Trigger DB Migration workflow in Forgejo Actions to apply
crates/mem-store/migrations/*.sql files.
2026-09-14 22:33:16 +09:00