poimen-memory

riotpiao-poimen/poimen-memory

Fork 0

Author	SHA1	Message	Date
rock	6499dae6e5	test: K8s Job-based integration testing with migrations CI / CI (pull_request) Successful in 16m41s Details Add proper integration test infrastructure: migrations/run_migrations.sh: - Database migration runner (used by K8s Job) - Applies all SQL migrations in order - Waits for DB to be ready - Verifies schema creation - Reports success/failure k8s/test/integration-test-job.yaml: - Kubernetes Job manifest for E2E testing - Two-stage execution: 1. migrate: Apply database migrations 2. test: Run integration test against new pod - Uses new image SHA from CI build - Proper secret management via K8s secretKeyRef (passwords stored in cluster, not in manifests) - Resource limits and liveness probes - Cleanup after 1 hour (ttlSecondsAfterFinished) .gitea/workflows/integration-test.yaml: - CI workflow that runs after image build - Validates image exists in registry - Deploys Job with correct image SHA - Waits for job completion (10 min timeout) - Collects pod logs on failure - Automatic cleanup Security: • No plaintext credentials in manifests • Uses K8s secretKeyRef for DB password • All secrets encrypted with SOPS/Age (ArgoCD plugin) • Never embed credentials in git Usage: - Automatic: Runs after each CI build on main - Manual: Trigger with specific image SHA via workflow_dispatch - Tests: Full E2E ingest + persistence + query URGENT: Rotate memory-db-app password (was visible in debugging shell history)	2026-09-14 22:44:53 +09:00
rock	6915dc2462	feat: production ingest test suite with detailed logging Add comprehensive E2E test scripts and logging for production testing: - test_prod_ingest_real.sh: Full ingest test against K8s cluster with api-gw - apply_migrations.sh: Manual database schema migration (backup method) - collect_prod_logs.sh: Pod log collection before/after tests - run_production_test.sh: Orchestrates full test + log collection - tests/integration_ingest_with_gw.rs: Integration test with embeddings - tests/unit_ingest_logging.rs: Unit tests for extraction pipeline Enhanced logging in ingest_worker.rs: - Per-record event tracking (extraction, save) - Entity and edge operation logging - Error accumulation and reporting - Structured logging for observability Production testing identified root cause: - Ingest + embedding pipeline working correctly - Entity extraction functional - Database schema missing (migration not applied) - Logs clearly show: relation "memory_entity" does not exist Next: Trigger DB Migration workflow in Forgejo Actions to apply crates/mem-store/migrations/*.sql files.	2026-09-14 22:33:16 +09:00
rock	5fd3ac826b	test: verify embedding response parsing against real service format - 6 parsing tests for EmbeddingResponse struct - test_parse_real_embedding_response: exact format from embeddings-predictor - test_parse_768_dim_response: full 768-dim vector - test_parse_multi_input_response: array input returns multiple embeddings - test_parse_embedding_error_response: error format - test_parse_html_fails_gracefully: HTML error page correctly rejected - Confirms: parsing is correct, 'expected ident' error is non-JSON response	2026-09-14 08:46:03 +09:00

Author

SHA1

Message

Date

rock

6499dae6e5

test: K8s Job-based integration testing with migrations

CI / CI (pull_request) Successful in 16m41s

Details

Add proper integration test infrastructure:

migrations/run_migrations.sh:
  - Database migration runner (used by K8s Job)
  - Applies all SQL migrations in order
  - Waits for DB to be ready
  - Verifies schema creation
  - Reports success/failure

k8s/test/integration-test-job.yaml:
  - Kubernetes Job manifest for E2E testing
  - Two-stage execution:
    1. migrate: Apply database migrations
    2. test: Run integration test against new pod
  - Uses new image SHA from CI build
  - Proper secret management via K8s secretKeyRef
    (passwords stored in cluster, not in manifests)
  - Resource limits and liveness probes
  - Cleanup after 1 hour (ttlSecondsAfterFinished)

.gitea/workflows/integration-test.yaml:
  - CI workflow that runs after image build
  - Validates image exists in registry
  - Deploys Job with correct image SHA
  - Waits for job completion (10 min timeout)
  - Collects pod logs on failure
  - Automatic cleanup

Security:
  • No plaintext credentials in manifests
  • Uses K8s secretKeyRef for DB password
  • All secrets encrypted with SOPS/Age (ArgoCD plugin)
  • Never embed credentials in git

Usage:
  - Automatic: Runs after each CI build on main
  - Manual: Trigger with specific image SHA via workflow_dispatch
  - Tests: Full E2E ingest + persistence + query

URGENT: Rotate memory-db-app password
  (was visible in debugging shell history)

2026-09-14 22:44:53 +09:00

rock

6915dc2462

feat: production ingest test suite with detailed logging

Add comprehensive E2E test scripts and logging for production testing:

- test_prod_ingest_real.sh: Full ingest test against K8s cluster with api-gw
- apply_migrations.sh: Manual database schema migration (backup method)
- collect_prod_logs.sh: Pod log collection before/after tests
- run_production_test.sh: Orchestrates full test + log collection
- tests/integration_ingest_with_gw.rs: Integration test with embeddings
- tests/unit_ingest_logging.rs: Unit tests for extraction pipeline

Enhanced logging in ingest_worker.rs:
- Per-record event tracking (extraction, save)
- Entity and edge operation logging
- Error accumulation and reporting
- Structured logging for observability

Production testing identified root cause:
- Ingest + embedding pipeline working correctly
- Entity extraction functional
- Database schema missing (migration not applied)
- Logs clearly show: relation "memory_entity" does not exist

Next: Trigger DB Migration workflow in Forgejo Actions to apply
crates/mem-store/migrations/*.sql files.

2026-09-14 22:33:16 +09:00

rock

5fd3ac826b

test: verify embedding response parsing against real service format

- 6 parsing tests for EmbeddingResponse struct
- test_parse_real_embedding_response: exact format from embeddings-predictor
- test_parse_768_dim_response: full 768-dim vector
- test_parse_multi_input_response: array input returns multiple embeddings
- test_parse_embedding_error_response: error format
- test_parse_html_fails_gracefully: HTML error page correctly rejected
- Confirms: parsing is correct, 'expected ident' error is non-JSON response

2026-09-14 08:46:03 +09:00

1 changed files with 1 additions and 1 deletions

									
										.env
									
		+1
		-1
	
												View File
												
				@@ -4,7 +4,7 @@ MEM_RATE_LIMIT_QUERY=10000

				MEM_IDEMPOTENCY_TTL_SECS=86400

				MEM_EMBEDDING_BATCH_SIZE=4

				DATABASE_URL=postgresql://app:katFpWYB4EH9KU9NABOglnE9ekea5rBxyOY9WZeUTi1ujhFS1pVzNxrXbB7A4qGc@127.0.0.1:5433/memory

				DATABASE_URL=postgresql://app:***REMOVED***@127.0.0.1:5433/memory

				# Embedding via direct port-forward (skip gateway auth)

				LLM_ENDPOINT=http://localhost:9090/v1/chat/completions

Compare commits