riotpiao-poimen/poimen-memory

Fork 0

Files

T

poimenandrock d7a36ce9e8

CI / CI (push) Successful in 12m12s

Details

Deploy / Tag & Push Latest (push) Successful in 54s

Details

ci: optimize build + deploy + migrate workflows (#51 )

## Optimize CI/CD Workflows

### Changes

#### build.yaml
- **Merge 3 cargo steps → 1 compile pass**: `cargo build`, `cargo test`, `cargo clippy` now run in single invocation, reusing compiled artifacts
- **Remove `cargo clean`**: Eliminated wasteful step that deleted artifacts before Docker build
- **Add secret validation**: Registry credentials checked before login (fail-fast)

#### deploy.yaml
- **Skip checkout**: Removed unnecessary git clone
- **Fetch SHA via Gitea API**: Query latest commit directly instead of cloning
- **Reuse existing token**: Use `FORGEJO_REGISTRY_TOKEN` for Gitea API auth (already has privileges)
- **Validate image exists**: Check SHA image exists before tagging as latest (prevents tagging non-existent images)
- **Add secret validation**: Registry credentials checked before login (fail-fast)

#### migrate.yaml
- **Merge schema verification**: Schema inspect result reused in both changed + manual paths
- **Fix manual trigger errors**: Manual mode now fails on first migration error (was silently masking with `|| true`)
- **Track failures**: Explicit FAILED flag tracks migration errors across loop

### Benefits

- **Speed**: Fewer compiles, no unnecessary clones, reuse artifacts
- **Reliability**: Secret validation catches configuration issues early
- **Safety**: Image existence check prevents tagging phantom images
- **Clarity**: Merged steps have descriptive names, explicit error handling

### Testing

- Branch: `ci/optimize-workflows`
- Ready to merge to `main` after review

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #51
Co-authored-by: poimen <[email protected]>

2026-09-13 05:42:01 +00:00

4.5 KiB

Raw Blame History

Poimen Memory - Environment Configuration Guide

All downstream service URIs are read from environment variables, sourced from ConfigMap.

How It Works

ConfigMap provides URIs: k8s/app/config.yaml (production, SOPS-encrypted)
Deployment injects via envFrom: envFrom: configMapRef: poimen-memory-config
Application reads from ENV: Code parses LLM_ENDPOINT, OPENSEARCH_HOST, AUTHENTIK_ISSUER, etc.

# deployment.yaml
envFrom:
  - configMapRef:
      name: poimen-memory-config  # All vars injected as ENV

Environment Variables

LLM Service (Entity & Fact Extraction)

LLM_ENDPOINT — full URL to chat/completions endpoint
LLM_API_BASE — base API URL (used for client initialization)
LLM_MODEL — model identifier (ornith:35b, qwen:7b, etc.)
LLM_TIMEOUT_SECS — timeout for LLM requests
ENABLE_LLM_EXTRACTION — enable/disable LLM extraction (true/false)

OpenSearch (Vector Store, BM25)

OPENSEARCH_HOST — hostname:port
OPENSEARCH_SCHEME — http or https
OPENSEARCH_VERIFY_CERTS — SSL certificate verification (true/false)

Authentik (OIDC)

AUTHENTIK_ISSUER — OIDC issuer URL
AUTHENTIK_VERIFY_SSL — SSL certificate verification (true/false)
MEM_AUTH_MODE — auth mode: jwt | apikey | none

Temporal (Workflow Orchestration - Future)

TEMPORAL_ENDPOINT — temporal frontend hostname:port
TEMPORAL_NAMESPACE — temporal namespace

API Gateway (Route Optimization - Future)

GATEWAY_URL — gateway base URL

Memory Service Config

MEM_AUTH_MODE — jwt | apikey | none
MEM_RATE_LIMIT_INGEST — ingest requests per second
MEM_RATE_LIMIT_QUERY — query requests per second
MEM_EMBEDDING_BATCH_SIZE — batch size for embeddings

Deployment Scenarios

Production (SOPS-Encrypted ConfigMap)

File: k8s/app/config.yaml

Services use cluster-internal DNS:

LLM_ENDPOINT: http://reasoning-predictor.llm-serving.svc.cluster.local:8000/v1/chat/completions
OPENSEARCH_HOST: opensearch.poimen.svc.cluster.local:9200
AUTHENTIK_ISSUER: https://authentik.auth.svc.cluster.local:9443/application/o/poimen/
TEMPORAL_ENDPOINT: temporal-frontend.temporal.svc.cluster.local:7233
GATEWAY_URL: http://api-gw.poimen.svc.cluster.local:8080
MEM_AUTH_MODE: jwt

Deploy:

# SOPS auto-decrypts based on .sops.yaml age key
kubectl apply -f k8s/app/config.yaml -k k8s/app/

Local/Development (Plaintext ConfigMap)

File: k8s/app/config.local.yaml

Services via external URLs (ingress):

LLM_ENDPOINT: https://api.riotpiao.com/v1/chat/completions
OPENSEARCH_HOST: opensearch.riotpiao.com:443
AUTHENTIK_ISSUER: https://authentik.riotpiao.com/application/o/poimen/
TEMPORAL_ENDPOINT: temporal.riotpiao.com:443
GATEWAY_URL: https://api.riotpiao.com
MEM_AUTH_MODE: none

Deploy (override production config):

# Delete prod config, apply local
kubectl delete configmap poimen-memory-config -n poimen
kubectl apply -f k8s/app/config.local.yaml

Encrypting with SOPS

Production config.yaml is encrypted with SOPS (Age-based).

Encrypt:

sops -e k8s/app/config.yaml > k8s/app/config.yaml.enc
mv k8s/app/config.yaml.enc k8s/app/config.yaml

Decrypt for editing (SOPS auto-handles with $EDITOR):

sops k8s/app/config.yaml

View decrypted (without editing):

sops -d k8s/app/config.yaml

.sops.yaml defines encryption key:

creation_rules:
  - path_regex: k8s/app/config.yaml
    key_groups:
      - age:
          - <age-public-key>

Application Code Pattern

Example: Application should read URIs from ENV at startup.

// Pseudocode
let llm_endpoint = env::var("LLM_ENDPOINT")
    .unwrap_or("http://localhost:11434/v1/chat/completions".to_string());
let opensearch_host = env::var("OPENSEARCH_HOST")
    .unwrap_or("localhost:9200".to_string());
let auth_mode = env::var("MEM_AUTH_MODE")
    .unwrap_or("none".to_string());

// Initialize clients with these URIs
let llm_client = LlmClient::new(llm_endpoint)?;
let search_client = OpenSearchClient::new(opensearch_host)?;

Summary

Aspect	Production	Local
Config File	`config.yaml`	`config.local.yaml`
Encryption	SOPS (Age)	Plaintext
Service URIs	Cluster-internal DNS	External HTTPS
Auth Mode	JWT (Authentik)	None (disabled)
Rate Limits	100/1000	1000/10000
Deploy	`kubectl apply -k k8s/app/`	`kubectl apply -f config.local.yaml`

4.5 KiB Raw Blame History