Compare commits
271
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4fb1c6feeb | ||
|
|
2d7127b37e | ||
|
|
2ba89f2ec0 | ||
|
|
0b282ba1f8 | ||
|
|
f10f0a8a26 | ||
|
|
34288b0b95 | ||
|
|
1fb0b62d7d | ||
|
|
09873aa275 | ||
|
|
63f2eaddd6 | ||
|
|
555b4b4050 | ||
|
|
8cf342b27c | ||
|
|
44f9bc25c4 | ||
|
|
5e97c5cf64 | ||
|
|
b7b1f15084 | ||
|
|
d51713ad6a | ||
|
|
fa239972a7 | ||
|
|
f53d54cba9 | ||
|
|
e4bbec95fb | ||
|
|
beb3cb21a0 | ||
|
|
9e84fb3386 | ||
|
|
2333310c38 | ||
|
|
be16020878 | ||
|
|
e5c371ed39 | ||
|
|
8b88e13762 | ||
|
|
e691a91df1 | ||
|
|
3fd930c518 | ||
|
|
88e73a885a | ||
|
|
f06cefabc4 | ||
|
|
36dfaa4ddd | ||
|
|
9188be39c6 | ||
|
|
d4508afc07 | ||
|
|
3f5d44d6fa | ||
|
|
209df7558e | ||
|
|
d81c57f860 | ||
|
|
e7b526b1d0 | ||
|
|
8589c40b44 | ||
|
|
feb7b25aba | ||
|
|
9d0ffcd29f | ||
|
|
accdfb11d7 | ||
|
|
47e7a2b1d6 | ||
|
|
8d63db9f3b | ||
|
|
bf67d2d9de | ||
|
|
95ae933489 | ||
|
|
f1d5c71a6c | ||
|
|
e6f2ab1423 | ||
|
|
1c7395d9e1 | ||
|
|
eba9f2144c | ||
|
|
4ad4df7965 | ||
|
|
8fda8c50d3 | ||
|
|
2e835510d6 | ||
|
|
8c32c16f79 | ||
|
|
18602759c0 | ||
|
|
751ae733d5 | ||
|
|
68107ba962 | ||
|
|
c2bcda58d9 | ||
|
|
14a17b4542 | ||
|
|
ce019f5f3a | ||
|
|
a55918e1f0 | ||
|
|
2499cc241f | ||
|
|
1c98628417 | ||
|
|
ec046cccde | ||
|
|
9d485d1238 | ||
|
|
48aac4998b | ||
|
|
70fcf111b9 | ||
|
|
cee29b8cb8 | ||
|
|
423e40200a | ||
|
|
4e7a7b065e | ||
|
|
a831c4d3db | ||
|
|
dafccd5d72 | ||
|
|
e963ceb90e | ||
|
|
ed9cf4d1e6 | ||
|
|
7b0f9171e2 | ||
|
|
2d9a23c4db | ||
|
|
7854e4557e | ||
|
|
162d95c314 | ||
|
|
cb2e44691e | ||
|
|
ce7a8bad81 | ||
|
|
e2c246cfcd | ||
|
|
acf8184680 | ||
|
|
cd3abede11 | ||
|
|
21fb58e1d3 | ||
|
|
0cbd5c16dd | ||
|
|
af30c189fe | ||
|
|
17dbe32a10 | ||
|
|
4b3f664502 | ||
|
|
d1b3c0e53d | ||
|
|
84aefc8db2 | ||
|
|
2b94114310 | ||
|
|
4faf8115c3 | ||
|
|
9836d20b06 | ||
|
|
cd6760bdca | ||
|
|
24892544b7 | ||
|
|
1685bca027 | ||
|
|
e76ad914d2 | ||
|
|
30c5197228 | ||
|
|
6d1c05574a | ||
|
|
be7881d6f0 | ||
|
|
dde4b602c4 | ||
|
|
be2a56ccf5 | ||
|
|
3d8a965718 | ||
|
|
246196407a | ||
|
|
1361c9bd13 | ||
|
|
01a310d13f | ||
|
|
46ec0caf5d | ||
|
|
616660cebe | ||
|
|
f9b9fbce95 | ||
|
|
301c661a46 | ||
|
|
d22842ca33 | ||
|
|
af00467b2b | ||
|
|
0a323fc039 | ||
|
|
566dcafbf6 | ||
|
|
261fa6faa8 | ||
|
|
b8c3528848 | ||
|
|
f08fb2bb75 | ||
|
|
d602ed8c78 | ||
|
|
1dd261bb25 | ||
|
|
c64b437bca | ||
|
|
64ee19c822 | ||
|
|
32cb01388c | ||
|
|
875b87cea2 | ||
|
|
d55e7ff31e | ||
|
|
eda152015c | ||
|
|
c9bf9f7dce | ||
|
|
5170921eea | ||
|
|
b3017c525a | ||
|
|
f101b3381e | ||
|
|
ef348d23f4 | ||
|
|
04ec157c19 | ||
|
|
ded98329e5 | ||
|
|
d51056c684 | ||
|
|
c661d7eb77 | ||
|
|
edc12c388f | ||
|
|
f0178b3bc5 | ||
|
|
e82c4b36a4 | ||
|
|
4ea25620dd | ||
|
|
0588cb91b4 | ||
|
|
4bb99ef24f | ||
|
|
fd07b3cff2 | ||
|
|
dc0bb63a01 | ||
|
|
b2191509fb | ||
|
|
5635482e0d | ||
|
|
328a713f4f | ||
|
|
cc5325d905 | ||
|
|
26f7da3610 | ||
|
|
f2f4a2580f | ||
|
|
21e3987b11 | ||
|
|
3e7238f71c | ||
|
|
88f8a764de | ||
|
|
34e996475f | ||
|
|
3f4653ac56 | ||
|
|
1dc6a2025f | ||
|
|
da925f3101 | ||
|
|
2443708abb | ||
|
|
9a34c12068 | ||
|
|
f646bb06fd | ||
|
|
4363739d59 | ||
|
|
2bf543bba1 | ||
|
|
6a2aacc4e6 | ||
|
|
0471177250 | ||
|
|
7fb73d6a4c | ||
|
|
e0b24c83d0 | ||
|
|
9117fd777a | ||
|
|
5ce0b92186 | ||
|
|
89fa87f7c1 | ||
|
|
42cd0204fa | ||
|
|
3a95f57b8f | ||
|
|
2ec6eba9d2 | ||
|
|
d6f5b9ed69 | ||
|
|
ce1fc4e296 | ||
|
|
d841bbdb95 | ||
|
|
d9d2e34558 | ||
|
|
4cfac71a73 | ||
|
|
7728f20d2b | ||
|
|
cea1a78a37 | ||
|
|
abaea8823b | ||
|
|
d282ae1aa0 | ||
|
|
063308308f | ||
|
|
a207c56637 | ||
|
|
c759481ea6 | ||
|
|
15b1ec6ad4 | ||
|
|
f7a8df0514 | ||
|
|
32281ee923 | ||
|
|
578a707867 | ||
|
|
c6493f14ae | ||
|
|
7437078f23 | ||
|
|
f8b19d9f55 | ||
|
|
c000ddb402 | ||
|
|
888c4f5493 | ||
|
|
403e495fe9 | ||
|
|
f30771a78a | ||
|
|
e2e17ae0fb | ||
|
|
c230b3ee45 | ||
|
|
2d7330798b | ||
|
|
a81b9b6169 | ||
|
|
a860de94da | ||
|
|
94a2bd648c | ||
|
|
be2e5c321f | ||
|
|
9d1d79b774 | ||
|
|
72ab6b6973 | ||
|
|
7114fc8fc9 | ||
|
|
fe7b749951 | ||
|
|
694350634b | ||
|
|
ab103f00f0 | ||
|
|
e4d645eae9 | ||
|
|
ab76e40d05 | ||
|
|
b568c015e2 | ||
|
|
e71c7ad37e | ||
|
|
842360288d | ||
|
|
f18f96eb5b | ||
|
|
d655726eca | ||
|
|
ff22027c7a | ||
|
|
f158512261 | ||
|
|
69d2240cf4 | ||
|
|
543105bf46 | ||
|
|
7d1eb09486 | ||
|
|
0dddf15dc8 | ||
|
|
37ee3dc5b1 | ||
|
|
f864e3dc51 | ||
|
|
96e40916e3 | ||
|
|
04100232ef | ||
|
|
09980aa41d | ||
|
|
5144ab732d | ||
|
|
4379f3cb21 | ||
|
|
d2f4b3c7e4 | ||
|
|
e7f3409d0f | ||
|
|
23ec31bd6d | ||
|
|
421086f845 | ||
|
|
bd00bca9bf | ||
|
|
397edf632f | ||
|
|
13557184c6 | ||
|
|
6d554961c2 | ||
|
|
3eccf9f653 | ||
|
|
52cb895cda | ||
|
|
bca247a763 | ||
|
|
9e3781a069 | ||
|
|
ebeb4948d4 | ||
|
|
a3190abe50 | ||
|
|
dddc7a524a | ||
|
|
ed7be6f229 | ||
|
|
418ab7bfc2 | ||
|
|
ca8525c625 | ||
|
|
8a3a892cbd | ||
|
|
cdacdd8d11 | ||
|
|
263a48a22d | ||
|
|
47c0301a43 | ||
|
|
edd4ea7fe4 | ||
|
|
086ad9f9a9 | ||
|
|
3eabb847fd | ||
|
|
6109477bf9 | ||
|
|
20c634fd03 | ||
|
|
074b43e1f2 | ||
|
|
c2e084c7c2 | ||
|
|
dd608d3231 | ||
|
|
9a4d486b86 | ||
|
|
4d32e2765b | ||
|
|
0d4e98d88b | ||
|
|
ebf97f573e | ||
|
|
05b088ca48 | ||
|
|
b33fdde5b7 | ||
|
|
cd1c5691e8 | ||
|
|
e6d7626fb6 | ||
|
|
e1d0cfd70b | ||
|
|
18f2f94f8e | ||
|
|
6d5a0ba205 | ||
|
|
1c02e2b831 | ||
|
|
4ab596196e | ||
|
|
63d7256b9e | ||
|
|
674c8f0d66 | ||
|
|
36aea89e47 | ||
|
|
11c26f3f29 | ||
|
|
11898733e8 |
@@ -0,0 +1,460 @@
|
|||||||
|
# CI/CD Pipeline: GitOps Validation & Deployment
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Pure GitOps CI/CD pipeline using Forgejo Actions (self-hosted runner).
|
||||||
|
|
||||||
|
**Principle:** Validate in CI, deploy via ArgoCD (no manual steps).
|
||||||
|
|
||||||
|
```
|
||||||
|
git push
|
||||||
|
↓
|
||||||
|
[CI: Validate]
|
||||||
|
├─ yamllint (YAML syntax)
|
||||||
|
├─ kubeval (K8s manifests)
|
||||||
|
├─ kustomize build (all layers)
|
||||||
|
├─ argocd validation (app definitions)
|
||||||
|
└─ security scan (secrets, best practices)
|
||||||
|
↓
|
||||||
|
[If push to main]
|
||||||
|
└─ ArgoCD auto-syncs (if enabled)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Workflows
|
||||||
|
|
||||||
|
### 1. validate-k8s.yaml (Mandatory)
|
||||||
|
|
||||||
|
**Trigger:** Any push/PR with k8s/ changes
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
1. Lints all YAML files (`yamllint`)
|
||||||
|
2. Validates K8s manifests (`kubeval`)
|
||||||
|
3. Builds all kustomization layers
|
||||||
|
4. Validates ArgoCD applications
|
||||||
|
5. Reports results
|
||||||
|
|
||||||
|
**Duration:** ~2-3 minutes
|
||||||
|
|
||||||
|
**Status:**
|
||||||
|
- ✅ PASS: All layers build, manifests valid → OK to merge
|
||||||
|
- ❌ FAIL: Syntax error, invalid resource, build failed → Fix & push again
|
||||||
|
|
||||||
|
**Example output:**
|
||||||
|
```
|
||||||
|
=== Building k8s/infrastructure/ ===
|
||||||
|
✓ Infrastructure built successfully
|
||||||
|
Resources: 47
|
||||||
|
|
||||||
|
=== Building k8s/bootstrap/ ===
|
||||||
|
✓ Bootstrap built successfully
|
||||||
|
Resources: 23
|
||||||
|
```
|
||||||
|
|
||||||
|
**When to check:**
|
||||||
|
- After every commit
|
||||||
|
- Before merging PRs
|
||||||
|
- On every branch
|
||||||
|
|
||||||
|
### 2. argocd-sync.yaml (Recommended)
|
||||||
|
|
||||||
|
**Trigger:** Push to main only (k8s/ changed)
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
1. Authenticates with ArgoCD
|
||||||
|
2. Syncs `homelab-root` application
|
||||||
|
3. Waits for sync to complete (5 min timeout)
|
||||||
|
4. Verifies all applications healthy
|
||||||
|
|
||||||
|
**Duration:** 1-5 minutes (depends on resources)
|
||||||
|
|
||||||
|
**Status:**
|
||||||
|
- ✅ SYNCED: All resources deployed to cluster
|
||||||
|
- ❌ FAILED: Sync error, pod crashes, etc. → Check ArgoCD UI for details
|
||||||
|
|
||||||
|
**When it runs:**
|
||||||
|
- Automatically after merge to main
|
||||||
|
- Only on k8s/ changes (not on docs)
|
||||||
|
|
||||||
|
**Manual trigger (if needed):**
|
||||||
|
```bash
|
||||||
|
# SSH to runner or use Forgejo UI
|
||||||
|
# Re-run failed workflow
|
||||||
|
# Or manually sync: argocd app sync homelab-root
|
||||||
|
```
|
||||||
|
|
||||||
|
**Requires secrets:**
|
||||||
|
- `ARGOCD_SERVER`: ArgoCD server URL (https://argocd.riotpiao.com)
|
||||||
|
- `ARGOCD_AUTH_TOKEN`: ArgoCD API token (generate via ArgoCD UI)
|
||||||
|
|
||||||
|
### 3. security-scan.yaml (Optional)
|
||||||
|
|
||||||
|
**Trigger:** Any push/PR with k8s/ changes
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
1. Scans Dockerfiles for vulnerabilities (`trivy`)
|
||||||
|
2. Scans Helm charts for security issues
|
||||||
|
3. Audits K8s manifests (`polaris`)
|
||||||
|
4. Checks for hardcoded secrets
|
||||||
|
5. Verifies security best practices
|
||||||
|
|
||||||
|
**Duration:** ~3-5 minutes
|
||||||
|
|
||||||
|
**Status:**
|
||||||
|
- ✅ PASS: No critical issues
|
||||||
|
- ⚠️ WARNING: Best practice recommendations (non-blocking)
|
||||||
|
- ❌ FAIL: Hardcoded secrets found (must fix)
|
||||||
|
|
||||||
|
**Common issues:**
|
||||||
|
- Missing resource limits (warning)
|
||||||
|
- Privileged containers (warning)
|
||||||
|
- Hardcoded passwords (ERROR)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.forgejo/
|
||||||
|
├── workflows/ # CI/CD workflows
|
||||||
|
│ ├── validate-k8s.yaml # Validate manifests (required)
|
||||||
|
│ ├── argocd-sync.yaml # Sync to cluster (auto on main)
|
||||||
|
│ └── security-scan.yaml # Security checks (optional)
|
||||||
|
└── CI-CD.md # This file
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Setup Instructions
|
||||||
|
|
||||||
|
### 1. Install Forgejo Runner
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On runner machine (inside cluster or external)
|
||||||
|
forgejo-runner register \
|
||||||
|
--instance https://forgejo.riotpiao.com \
|
||||||
|
--token <registration-token> \
|
||||||
|
--name homelab-runner \
|
||||||
|
--labels docker
|
||||||
|
|
||||||
|
forgejo-runner daemon
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Add ArgoCD Secrets to Forgejo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Go to: Forgejo → Settings → Secrets
|
||||||
|
|
||||||
|
# Add:
|
||||||
|
ARGOCD_SERVER = https://argocd.riotpiao.com
|
||||||
|
ARGOCD_AUTH_TOKEN = <token> # Generate: argocd account generate-token
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Generate ArgoCD Token
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Inside cluster
|
||||||
|
kubectl -n argocd port-forward svc/argocd-server 8080:443
|
||||||
|
|
||||||
|
# Go to: https://localhost:8080/user-info/api-tokens
|
||||||
|
# Create new token (CI/CD)
|
||||||
|
# Copy token to Forgejo secrets
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Workflow Execution
|
||||||
|
|
||||||
|
### When developer pushes to feature branch:
|
||||||
|
|
||||||
|
```
|
||||||
|
git push origin feature/new-service
|
||||||
|
|
||||||
|
↓
|
||||||
|
Forgejo Actions triggered
|
||||||
|
↓
|
||||||
|
validate-k8s.yaml runs:
|
||||||
|
✓ Lints YAML
|
||||||
|
✓ Validates manifests
|
||||||
|
✓ Builds kustomizations
|
||||||
|
✓ All pass → GitHub comment: "Ready to merge"
|
||||||
|
↓
|
||||||
|
Developer opens PR
|
||||||
|
↓
|
||||||
|
Reviewer checks:
|
||||||
|
- Code changes (YAML)
|
||||||
|
- Workflow results
|
||||||
|
- ArgoCD impact (diff)
|
||||||
|
↓
|
||||||
|
PR merged to main
|
||||||
|
```
|
||||||
|
|
||||||
|
### When merged to main:
|
||||||
|
|
||||||
|
```
|
||||||
|
git merge feature/new-service → main
|
||||||
|
|
||||||
|
↓
|
||||||
|
Forgejo Actions triggered
|
||||||
|
↓
|
||||||
|
validate-k8s.yaml runs:
|
||||||
|
✓ Same validation as above
|
||||||
|
↓
|
||||||
|
argocd-sync.yaml runs (if enabled):
|
||||||
|
✓ Syncs homelab-root
|
||||||
|
✓ Waits for sync
|
||||||
|
✓ Verifies health
|
||||||
|
✓ Resources deployed to cluster
|
||||||
|
↓
|
||||||
|
Cluster state = git state
|
||||||
|
(No manual kubectl apply needed!)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Debugging CI/CD Failures
|
||||||
|
|
||||||
|
### Issue: "Kustomize build failed"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run locally
|
||||||
|
cd k8s/
|
||||||
|
kustomize build bootstrap/ # See actual error
|
||||||
|
|
||||||
|
# Fix YAML/kustomization.yaml
|
||||||
|
# git push again
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue: "Kubeval validation failed"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check K8s manifest syntax
|
||||||
|
kubeval k8s/platform/minio/config.yaml
|
||||||
|
|
||||||
|
# Common issues:
|
||||||
|
# - Typos in apiVersion, kind, metadata
|
||||||
|
# - Missing required fields
|
||||||
|
# - Invalid references (namespace, service name)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue: "ArgoCD sync failed"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check ArgoCD UI
|
||||||
|
# https://argocd.riotpiao.com → homelab-root
|
||||||
|
|
||||||
|
# Or CLI
|
||||||
|
argocd app get homelab-root
|
||||||
|
argocd app logs homelab-root --follow
|
||||||
|
|
||||||
|
# Common issues:
|
||||||
|
# - Missing namespace (fixed by infrastructure layer)
|
||||||
|
# - Invalid Helm chart version
|
||||||
|
# - Secret not found
|
||||||
|
# - Network policy blocking traffic
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue: "Security scan found hardcoded secret"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Fix: Remove secret from YAML
|
||||||
|
# Add to SOPS encryption instead
|
||||||
|
|
||||||
|
# Or use ArgoCD Sealed Secrets
|
||||||
|
# (if SOPS not available)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Viewing Results
|
||||||
|
|
||||||
|
### Forgejo Actions UI
|
||||||
|
|
||||||
|
```
|
||||||
|
Repository → Actions
|
||||||
|
├─ validate-k8s
|
||||||
|
│ ├─ ✅ Success (merge safe)
|
||||||
|
│ ├─ ❌ Failed (fix required)
|
||||||
|
│ └─ Logs (click "Steps" → "Summary")
|
||||||
|
├─ argocd-sync
|
||||||
|
│ ├─ ✅ Synced (deployed)
|
||||||
|
│ └─ ❌ Failed (check ArgoCD UI)
|
||||||
|
└─ security-scan
|
||||||
|
├─ ✅ Pass (no critical issues)
|
||||||
|
└─ ⚠️ Warning (review, non-blocking)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ArgoCD UI
|
||||||
|
|
||||||
|
```
|
||||||
|
https://argocd.riotpiao.com
|
||||||
|
├─ homelab-root
|
||||||
|
│ ├─ Status: Synced ✓
|
||||||
|
│ ├─ Health: Healthy ✓
|
||||||
|
│ └─ Details (click to see resources)
|
||||||
|
├─ layer-1-bootstrap
|
||||||
|
├─ layer-2-platform
|
||||||
|
├─ layer-3-security
|
||||||
|
├─ layer-4-applications
|
||||||
|
└─ layer-5-data
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Tasks
|
||||||
|
|
||||||
|
### Add new service to cluster
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Create directory and kustomization.yaml
|
||||||
|
mkdir -p k8s/applications/my-service
|
||||||
|
cat > k8s/applications/my-service/kustomization.yaml << EOF
|
||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
namespace: my-namespace
|
||||||
|
helmCharts:
|
||||||
|
- name: my-chart
|
||||||
|
repo: https://charts.example.com
|
||||||
|
version: 1.0.0
|
||||||
|
releaseName: my-service
|
||||||
|
valuesFile: values.yaml
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 2. Add values.yaml
|
||||||
|
cp /template/values.yaml k8s/applications/my-service/
|
||||||
|
|
||||||
|
# 3. Commit and push
|
||||||
|
git add k8s/applications/my-service/
|
||||||
|
git commit -m "feat(apps): add my-service"
|
||||||
|
git push
|
||||||
|
|
||||||
|
# 4. CI validates
|
||||||
|
# 5. Merge to main
|
||||||
|
# 6. ArgoCD syncs automatically
|
||||||
|
# ✓ Service deployed to cluster
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rollback a deployment
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Find broken commit
|
||||||
|
git log --oneline k8s/ # Identify bad commit
|
||||||
|
|
||||||
|
# 2. Revert
|
||||||
|
git revert <commit-hash>
|
||||||
|
git push
|
||||||
|
|
||||||
|
# 3. CI validates (should pass)
|
||||||
|
# 4. Merge to main
|
||||||
|
# 5. ArgoCD syncs back to previous version
|
||||||
|
# ✓ Cluster state reverted
|
||||||
|
```
|
||||||
|
|
||||||
|
### Emergency: Disable ArgoCD auto-sync
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# If production broken and need time to debug:
|
||||||
|
argocd app set homelab-root --sync-policy none
|
||||||
|
|
||||||
|
# Fix issue in git
|
||||||
|
# Test locally: kustomize build k8s/
|
||||||
|
|
||||||
|
# Re-enable
|
||||||
|
argocd app set homelab-root --sync-policy automated
|
||||||
|
argocd app sync homelab-root
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Monitoring & Alerts
|
||||||
|
|
||||||
|
### Check workflow status in Forgejo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dashboard shows:
|
||||||
|
✅ All green → Safe to merge
|
||||||
|
❌ Red → Fix required before merge
|
||||||
|
⏳ Yellow → Still running (wait)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check ArgoCD status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
argocd app list
|
||||||
|
# Shows: Synced, OutOfSync, Unknown status
|
||||||
|
|
||||||
|
argocd app get homelab-root
|
||||||
|
# Shows: health, sync status, resources
|
||||||
|
|
||||||
|
argocd app logs homelab-root --follow
|
||||||
|
# Real-time logs during sync
|
||||||
|
```
|
||||||
|
|
||||||
|
### Alerts (optional, future)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Could add Forgejo webhooks → Slack/email
|
||||||
|
# When CI/CD fails → Alert ops team
|
||||||
|
# When ArgoCD goes OutOfSync → Alert ops team
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Workflow doesn't trigger
|
||||||
|
|
||||||
|
**Check:**
|
||||||
|
- Is Forgejo runner running? `forgejo-runner daemon`
|
||||||
|
- Did you push to correct branch? (validate runs on all, argocd-sync only on main)
|
||||||
|
- Did path match filter? (must change k8s/ or .forgejo/workflows/)
|
||||||
|
|
||||||
|
### Workflow hangs/times out
|
||||||
|
|
||||||
|
**Check:**
|
||||||
|
- kustomize build → Check for dependency cycles
|
||||||
|
- argocd sync → Check cluster resources (storage full? network down?)
|
||||||
|
- security scan → Large image scan → Takes time
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
- Increase timeout in workflow
|
||||||
|
- Optimize kustomization (remove unused resources)
|
||||||
|
- Add resource limits to pods
|
||||||
|
|
||||||
|
### ArgoCD token invalid
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
```bash
|
||||||
|
# Regenerate token
|
||||||
|
argocd account generate-token
|
||||||
|
|
||||||
|
# Update Forgejo secret
|
||||||
|
# Settings → Secrets → ARGOCD_AUTH_TOKEN = <new-token>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
✅ **DO:**
|
||||||
|
- Commit all K8s changes to git (no manual kubectl apply)
|
||||||
|
- Run validate-k8s locally before push
|
||||||
|
- Write descriptive commit messages (why this change?)
|
||||||
|
- Review workflow logs before merging
|
||||||
|
- Monitor ArgoCD sync after merge
|
||||||
|
|
||||||
|
❌ **DON'T:**
|
||||||
|
- Push directly to main (always use PR)
|
||||||
|
- Skip workflow validation (it catches errors early)
|
||||||
|
- Ignore security scan warnings
|
||||||
|
- Manually `kubectl apply` (breaks GitOps)
|
||||||
|
- Edit resources in cluster (they revert via ArgoCD)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
1. **Setup Forgejo runner** (if not already running)
|
||||||
|
2. **Add ArgoCD secrets** to Forgejo
|
||||||
|
3. **Test workflows** on feature branch
|
||||||
|
4. **Merge to main** → Watch ArgoCD sync
|
||||||
|
5. **Celebrate:** Full GitOps pipeline working! 🎉
|
||||||
@@ -156,44 +156,22 @@ jobs:
|
|||||||
- name: Check for Secrets in Code
|
- name: Check for Secrets in Code
|
||||||
run: |
|
run: |
|
||||||
echo "=== Scanning for hardcoded secrets ==="
|
echo "=== Scanning for hardcoded secrets ==="
|
||||||
# BLOCKING. This step used to only count findings and then exit 0, so a
|
SECRETS_FOUND=0
|
||||||
# plaintext deploy key rode through it into a public remote. Two failure
|
|
||||||
# modes fixed: it now fails the build, and it matches key material by
|
|
||||||
# PEM header rather than only `private_key:`-style YAML field names.
|
|
||||||
# Findings are captured into variables and tested for emptiness rather than
|
|
||||||
# branching on grep's exit status: implementations disagree on the rc of a
|
|
||||||
# `-v` filter fed empty input, and a wrong rc here fails open.
|
|
||||||
# NOTE: --include must precede `--`; after `--` grep treats it as a filename
|
|
||||||
# and silently scans nothing.
|
|
||||||
FAILED=0
|
|
||||||
|
|
||||||
# Any private key block is fatal, regardless of the field name carrying it.
|
for pattern in "password:" "secret:" "token:" "api_key:" "apikey:" "private_key:" "privatekey:"; do
|
||||||
KEYS=$(grep -rIE --include="*.yaml" --include="*.yml" \
|
if grep -r "$pattern" k8s/ --include="*.yaml" --include="*.yml" | grep -v "^Binary"; then
|
||||||
-- "-----BEGIN ([A-Z]+ )?PRIVATE KEY-----" k8s/ \
|
echo "⚠️ Found potential secret pattern: $pattern"
|
||||||
| grep -v "\.enc\.yaml" || true)
|
SECRETS_FOUND=$((SECRETS_FOUND + 1))
|
||||||
if [ -n "$KEYS" ]; then
|
fi
|
||||||
echo "❌ Unencrypted private key material found:"
|
done
|
||||||
echo "$KEYS"
|
|
||||||
FAILED=1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Plaintext values in secret-ish YAML fields. SOPS output is ENC[...],
|
if [ $SECRETS_FOUND -gt 0 ]; then
|
||||||
# so encrypted files never trip this.
|
echo "⚠️ Warning: Found $SECRETS_FOUND potential secrets"
|
||||||
VALS=$(grep -rInE --include="*.yaml" --include="*.yml" \
|
echo "Secrets should be encrypted with SOPS or stored in ArgoCD Sealed Secrets"
|
||||||
-- "^[[:space:]]*(password|token|apiKey|api_key|sshPrivateKey|client_secret):[[:space:]]*[\"']?[^\"'[:space:]{\$]{8,}" k8s/ \
|
else
|
||||||
| grep -v "ENC\[" | grep -v "\.enc\.yaml" || true)
|
echo "✓ No hardcoded secrets found"
|
||||||
if [ -n "$VALS" ]; then
|
|
||||||
echo "❌ Plaintext secret value found:"
|
|
||||||
echo "$VALS"
|
|
||||||
FAILED=1
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [ "$FAILED" -ne 0 ]; then
|
|
||||||
echo "Encrypt with SOPS (see .sops.yaml) — *.enc.yaml files are exempt."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
echo "✓ No hardcoded secrets found"
|
|
||||||
|
|
||||||
# === Check K8s Security Best Practices ===
|
# === Check K8s Security Best Practices ===
|
||||||
- name: Check K8s Security Best Practices
|
- name: Check K8s Security Best Practices
|
||||||
run: |
|
run: |
|
||||||
|
|||||||
+2
-11
@@ -50,19 +50,10 @@ terraform/*.tfstate.*
|
|||||||
terraform.tfvars.local
|
terraform.tfvars.local
|
||||||
skills-lock.json
|
skills-lock.json
|
||||||
secrets-plaintext.yaml
|
secrets-plaintext.yaml
|
||||||
|
skills-lock.json
|
||||||
# Saved plan files — binary, environment-specific, may embed resource attributes
|
|
||||||
terraform/tfplan
|
|
||||||
terraform/tfplan-*
|
|
||||||
|
|
||||||
.DS_Store
|
.DS_Store
|
||||||
CLAUDE.md
|
CLAUDE.md
|
||||||
docs/
|
docs/
|
||||||
|
|
||||||
bootstrap-argocd.log
|
k8s/argocd/seed-repo-secret.yamlbootstrap-argocd.log
|
||||||
|
|
||||||
# Any plaintext (non-SOPS) secret manifest. Encrypted ones are *.enc.yaml and
|
|
||||||
# ARE committed — see .sops.yaml. A missing newline once merged two patterns on
|
|
||||||
# one line here, which is how a plaintext deploy key reached a public remote.
|
|
||||||
k8s/**/*-secret.yaml
|
|
||||||
!k8s/**/*.enc.yaml
|
|
||||||
|
|||||||
+1
-3
@@ -1,5 +1,3 @@
|
|||||||
creation_rules:
|
creation_rules:
|
||||||
# `secrets?` — singular too. A `seed-repo-secret.yaml` once slipped this regex
|
- path_regex: k8s/.*secrets.*\.ya?ml
|
||||||
# and was committed in plaintext to a public remote.
|
|
||||||
- path_regex: k8s/.*secrets?.*\.ya?ml
|
|
||||||
age: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
age: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
||||||
|
|||||||
@@ -1,206 +0,0 @@
|
|||||||
# Authentik Auth Integration for NextJS
|
|
||||||
|
|
||||||
## Current State
|
|
||||||
|
|
||||||
### Gateway Auth Status
|
|
||||||
|
|
||||||
| Endpoint | Auth Status | Notes |
|
|
||||||
|----------|-------------|-------|
|
|
||||||
| `/v1/chat/completions` | ❌ **OFF** | LLM routes have no auth middleware |
|
|
||||||
| `/v1/embeddings` | ❌ **OFF** | Same - no auth |
|
|
||||||
| `/v1/rerank` | ❌ **OFF** | Same - no auth |
|
|
||||||
| `X-Service: sqs` | ✅ **ON** | JWT validated via `internal/auth/jwt.go` |
|
|
||||||
| `/workflow` | ❌ **OFF** | Pass-through to Temporal |
|
|
||||||
|
|
||||||
**Auth module exists** at `homelab-frontend/internal/auth/jwt.go` but only wired for SQS.
|
|
||||||
LLM routes in `internal/proxy/proxy.go` have no auth middleware.
|
|
||||||
|
|
||||||
### Authentik App
|
|
||||||
|
|
||||||
Authentik app `local-llm` exists for LLM API auth:
|
|
||||||
- **Client ID**: `local-llm`
|
|
||||||
- **Client Secret**: `kubectl -n llm-serving get secret local-llm-jwt -o jsonpath='{.data.client-secret}' | base64 -d`
|
|
||||||
- **Token endpoint**: `https://authentik.riotpiao.com/application/o/token/`
|
|
||||||
- **Userinfo endpoint**: `https://authentik.riotpiao.com/application/o/userinfo/`
|
|
||||||
- **OIDC discovery**: `https://authentik.riotpiao.com/application/o/local-llm/.well-known/openid-configuration`
|
|
||||||
|
|
||||||
## Sign-in Methods
|
|
||||||
|
|
||||||
### 1. Resource Owner Password Credentials (ROPC)
|
|
||||||
|
|
||||||
Direct username/password login. Server-side only (needs client_secret).
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// API Route: app/api/auth/login/route.ts
|
|
||||||
const response = await fetch('https://authentik.riotpiao.com/application/o/token/', {
|
|
||||||
method: 'POST',
|
|
||||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
|
||||||
body: new URLSearchParams({
|
|
||||||
grant_type: 'password',
|
|
||||||
client_id: 'local-llm',
|
|
||||||
client_secret: process.env.AUTHENTIK_CLIENT_SECRET,
|
|
||||||
username: '[email protected]',
|
|
||||||
password: 'userpassword',
|
|
||||||
scope: 'openid email profile groups',
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
|
|
||||||
const tokens = await response.json();
|
|
||||||
// { access_token, refresh_token, expires_in, token_type }
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Authorization Code Flow (Browser Redirect)
|
|
||||||
|
|
||||||
Requires adding redirect URIs to `local-llm` Authentik app:
|
|
||||||
|
|
||||||
```python
|
|
||||||
# In k8s/infra/iam/scripts/authentik-provision.py, update:
|
|
||||||
"local-llm": {
|
|
||||||
...
|
|
||||||
"redirect_uris": [
|
|
||||||
"http://localhost:3000/api/auth/callback", # dev
|
|
||||||
"https://your-nextjs-app.com/api/auth/callback", # prod
|
|
||||||
],
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Then standard OIDC flow:
|
|
||||||
1. Redirect to `https://authentik.riotpiao.com/application/o/authorize/?client_id=local-llm&redirect_uri=...&response_type=code&scope=openid email profile groups`
|
|
||||||
2. User logs in via Authentik UI
|
|
||||||
3. Callback receives `code`, exchange for tokens
|
|
||||||
|
|
||||||
## JWT Token Persistence
|
|
||||||
|
|
||||||
### Browser (localStorage)
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
const TOKEN_KEY = 'llm_auth_token';
|
|
||||||
|
|
||||||
// Save
|
|
||||||
localStorage.setItem(TOKEN_KEY, JSON.stringify({
|
|
||||||
access_token: tokens.access_token,
|
|
||||||
refresh_token: tokens.refresh_token,
|
|
||||||
expires_at: Date.now() + tokens.expires_in * 1000,
|
|
||||||
}));
|
|
||||||
|
|
||||||
// Load
|
|
||||||
const stored = JSON.parse(localStorage.getItem(TOKEN_KEY) || 'null');
|
|
||||||
if (stored && stored.expires_at > Date.now()) {
|
|
||||||
// Token valid
|
|
||||||
}
|
|
||||||
|
|
||||||
// Clear (logout)
|
|
||||||
localStorage.removeItem(TOKEN_KEY);
|
|
||||||
```
|
|
||||||
|
|
||||||
### Server-side (HTTP-only cookies)
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// app/api/auth/login/route.ts
|
|
||||||
import { cookies } from 'next/headers';
|
|
||||||
|
|
||||||
// After successful login
|
|
||||||
cookies().set('llm_auth_token', JSON.stringify(tokens), {
|
|
||||||
httpOnly: true,
|
|
||||||
secure: process.env.NODE_ENV === 'production',
|
|
||||||
sameSite: 'lax',
|
|
||||||
maxAge: tokens.expires_in,
|
|
||||||
path: '/',
|
|
||||||
});
|
|
||||||
|
|
||||||
// Read in middleware or API routes
|
|
||||||
const tokenCookie = cookies().get('llm_auth_token');
|
|
||||||
const tokens = JSON.parse(tokenCookie?.value || 'null');
|
|
||||||
```
|
|
||||||
|
|
||||||
## Token Refresh
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
async function refreshAccessToken(refresh_token: string) {
|
|
||||||
const response = await fetch('https://authentik.riotpiao.com/application/o/token/', {
|
|
||||||
method: 'POST',
|
|
||||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
|
||||||
body: new URLSearchParams({
|
|
||||||
grant_type: 'refresh_token',
|
|
||||||
client_id: 'local-llm',
|
|
||||||
client_secret: process.env.AUTHENTIK_CLIENT_SECRET,
|
|
||||||
refresh_token,
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
return response.json();
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# .env.local
|
|
||||||
AUTHENTIK_URL=https://authentik.riotpiao.com
|
|
||||||
AUTHENTIK_CLIENT_ID=local-llm
|
|
||||||
AUTHENTIK_CLIENT_SECRET=<from-secret>
|
|
||||||
|
|
||||||
# For client-side (public)
|
|
||||||
NEXT_PUBLIC_AUTHENTIK_URL=https://authentik.riotpiao.com
|
|
||||||
NEXT_PUBLIC_AUTHENTIK_CLIENT_ID=local-llm
|
|
||||||
```
|
|
||||||
|
|
||||||
## Using Token with LLM API
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
const token = await getValidToken(); // from localStorage or cookie
|
|
||||||
|
|
||||||
const response = await fetch('https://api.riotpiao.com/v1/chat/completions', {
|
|
||||||
method: 'POST',
|
|
||||||
headers: {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'Authorization': `Bearer ${token}`, // JWT from Authentik
|
|
||||||
},
|
|
||||||
body: JSON.stringify({
|
|
||||||
model: 'reasoning',
|
|
||||||
messages: [{ role: 'user', content: 'Hello' }],
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## TODO
|
|
||||||
|
|
||||||
### Gateway-side (homelab-frontend)
|
|
||||||
|
|
||||||
- [ ] Wire `internal/auth/jwt.go` into LLM proxy handler (`internal/proxy/proxy.go`)
|
|
||||||
- [ ] Add `authRequired: true` to model config or create LLM-specific middleware
|
|
||||||
- [ ] Example pattern from SQS (in `internal/serviceadapter/router.go`):
|
|
||||||
|
|
||||||
```go
|
|
||||||
// In proxy.go ServeHTTP, before dispatching to LLM upstream:
|
|
||||||
if strings.HasPrefix(r.URL.Path, "/v1/") {
|
|
||||||
authHeader := r.Header.Get("Authorization")
|
|
||||||
claims, err := llmJWTAuth.ValidateBearerToken(authHeader)
|
|
||||||
if err != nil {
|
|
||||||
// Return 401/403
|
|
||||||
}
|
|
||||||
if !llmJWTAuth.CheckPermissions(claims, "llm:inference", "*") {
|
|
||||||
// Return 403 insufficient permissions
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Authentik-side
|
|
||||||
|
|
||||||
- [ ] Enable ROPC grant in Authentik provider settings (if not already)
|
|
||||||
- [ ] Add redirect URIs to `local-llm` app if browser OAuth flow needed:
|
|
||||||
|
|
||||||
```python
|
|
||||||
# k8s/infra/iam/scripts/authentik-provision.py
|
|
||||||
"local-llm": {
|
|
||||||
...
|
|
||||||
"redirect_uris": [
|
|
||||||
"http://localhost:3000/api/auth/callback",
|
|
||||||
"https://your-app.com/api/auth/callback",
|
|
||||||
],
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### NextJS-side
|
|
||||||
|
|
||||||
- [ ] Until gateway auth is wired, LLM API works without token
|
|
||||||
- [ ] Once wired, add `Authorization: Bearer <token>` to all LLM requests
|
|
||||||
@@ -42,86 +42,6 @@ All logs + metrics centralized in Grafana for debugging
|
|||||||
- **Secrets at rest** — Vault + encrypted etcd; credentials never in logs or ConfigMaps
|
- **Secrets at rest** — Vault + encrypted etcd; credentials never in logs or ConfigMaps
|
||||||
- **Infrastructure-as-code** — Every service deployed via Helmfile; one `helmfile apply` recovers from total failure
|
- **Infrastructure-as-code** — Every service deployed via Helmfile; one `helmfile apply` recovers from total failure
|
||||||
|
|
||||||
## ArgoCD — GitOps Deployment Flow
|
|
||||||
|
|
||||||
**ArgoCD** pulls infrastructure changes from git and syncs the cluster automatically.
|
|
||||||
No manual `kubectl apply` — push to git, ArgoCD detects the change, and deploys within ~3 minutes.
|
|
||||||
|
|
||||||
```
|
|
||||||
Developer pushes to git
|
|
||||||
↓
|
|
||||||
ArgoCD detects change (every 3 min or webhook)
|
|
||||||
↓
|
|
||||||
Syncs manifests to cluster
|
|
||||||
↓
|
|
||||||
Workloads reconcile automatically
|
|
||||||
```
|
|
||||||
|
|
||||||
Applications are deployed in waves (numbered 00, 10, 20, 30, ...) to respect dependencies —
|
|
||||||
storage deploys before databases, databases before applications.
|
|
||||||
|
|
||||||
### Tracked Git Repositories
|
|
||||||
|
|
||||||
ArgoCD monitors these repos for changes:
|
|
||||||
|
|
||||||
| Repository | Purpose |
|
|
||||||
|------------|----------|
|
|
||||||
| `https://github.com/Riotpiaole/riotpiao.homelab.com` | Main infrastructure repo (all manifests in `k8s/argocd/apps/`) |
|
|
||||||
| `https://forgejo.riotpiao.com/rock/*` | Any `rock/*` repo in in-cluster Forgejo (apps + configs) |
|
|
||||||
| `https://github.com/Riotpiaole/Poimen-*` | External Poimen services (memory, workflows) |
|
|
||||||
|
|
||||||
To deploy a new application: create a git repo, add an Application manifest to the homelab repo's
|
|
||||||
`k8s/argocd/apps/`, commit + push, and ArgoCD syncs within 3 minutes.
|
|
||||||
|
|
||||||
## Management Planes — Talos vs Kubernetes
|
|
||||||
|
|
||||||
This cluster has **two separate management planes**, each with different workflows:
|
|
||||||
|
|
||||||
| Plane | What it manages | Workflow | Tool |
|
|
||||||
|-------|-----------------|----------|------|
|
|
||||||
| **Talos (OS)** | Node configuration, kernel params, networking, CoreDNS, machine state | Edit `terraform/` → `terraform apply` → `make apply-cp` | `terraform` + `talosctl` |
|
|
||||||
| **Kubernetes (workloads)** | All pods, services, deployments, ingresses, databases | Edit `k8s/argocd/apps/` → `git push` → ArgoCD syncs | `git` + ArgoCD |
|
|
||||||
|
|
||||||
**Critical distinction:**
|
|
||||||
- **Kubernetes resources** (`k8s/**`) flow through **git → ArgoCD** — never use `kubectl apply`
|
|
||||||
- **Talos machine config** (`terraform/**`) uses **local `terraform apply`** (sanctioned exception — CI can't hold node credentials)
|
|
||||||
|
|
||||||
Example: To add a CoreDNS hostname rewrite, you edit `terraform/files/coredns/Corefile`, then:
|
|
||||||
```bash
|
|
||||||
cd terraform && terraform apply -var-file=terraform.tfvars.local
|
|
||||||
cd .. && make apply-cp # talosctl apply-config to all 3 control planes
|
|
||||||
```
|
|
||||||
|
|
||||||
But to add a new Kubernetes Deployment or update an Ingress, you only `git push` — **never `kubectl apply`**.
|
|
||||||
|
|
||||||
### CoreDNS ConfigMap Ownership — Critical
|
|
||||||
|
|
||||||
⚠️ **Warning:** The `coredns` ConfigMap in `kube-system` namespace is **owned by Talos**, not ArgoCD or kubectl.
|
|
||||||
It is rendered from `terraform/files/coredns/Corefile` into Talos's machine config at bootstrap time.
|
|
||||||
|
|
||||||
**Do not `kubectl apply` or `kubectl edit` this ConfigMap directly.** Doing so transfers field ownership to kubectl's
|
|
||||||
client-side-apply mechanism, and Talos's inline-manifest controller will silently no-op on every future reconcile
|
|
||||||
(server-side-apply conflict, no error surfaced).
|
|
||||||
|
|
||||||
**To update CoreDNS (e.g., add a hostname rewrite):**
|
|
||||||
1. Edit `terraform/files/coredns/Corefile`
|
|
||||||
2. Commit + push
|
|
||||||
3. Run `cd terraform && terraform apply -var-file=terraform.tfvars.local`
|
|
||||||
4. Run `make apply-cp` to push config to all control planes
|
|
||||||
5. CoreDNS picks up changes via its `reload` plugin — no pod restart needed
|
|
||||||
|
|
||||||
**If you accidentally edited the ConfigMap directly and broke Talos's ownership:**
|
|
||||||
```bash
|
|
||||||
kubectl delete configmap coredns -n kube-system
|
|
||||||
# Wait ~30s for Talos's k8s.ManifestApplyController to recreate it
|
|
||||||
kubectl get configmap coredns -n kube-system -w
|
|
||||||
```
|
|
||||||
|
|
||||||
Or as a stopgap, apply the correct content yourself:
|
|
||||||
```bash
|
|
||||||
kubectl apply --server-side -f <(terraform output coredns_config)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Quick Start — Deploying the Cluster
|
## Quick Start — Deploying the Cluster
|
||||||
|
|
||||||
### 1. Bootstrap Talos Nodes
|
### 1. Bootstrap Talos Nodes
|
||||||
|
|||||||
+11
-6
@@ -12,15 +12,16 @@
|
|||||||
# - Talos cluster up; kubectl context points at it
|
# - Talos cluster up; kubectl context points at it
|
||||||
# - helm 3, kubectl
|
# - helm 3, kubectl
|
||||||
# - SOPS age key at $SOPS_KEY (for the ArgoCD SOPS CMP plugin)
|
# - SOPS age key at $SOPS_KEY (for the ArgoCD SOPS CMP plugin)
|
||||||
#
|
# - GitHub read-only deploy key private half at $DEPLOY_KEY (public half added
|
||||||
# The GitHub seed repo is public, so it is cloned anonymously over HTTPS — no
|
# to the GitHub repo's Deploy keys)
|
||||||
# deploy key, no repository Secret, one less thing to bootstrap before ArgoCD.
|
|
||||||
#
|
#
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
BOOT="$SCRIPT_DIR/k8s/bootstrap"
|
BOOT="$SCRIPT_DIR/k8s/bootstrap"
|
||||||
SOPS_KEY="${SOPS_KEY:-$HOME/.sops/key.txt}"
|
SOPS_KEY="${SOPS_KEY:-$HOME/.sops/key.txt}"
|
||||||
|
DEPLOY_KEY="${DEPLOY_KEY:-$HOME/.ssh/argocd_seed}"
|
||||||
|
GITHUB_SSH="[email protected]:Riotpiaole/riotpiao.homelab.com.git"
|
||||||
|
|
||||||
log() { echo "[$(date +%H:%M:%S)] $*"; }
|
log() { echo "[$(date +%H:%M:%S)] $*"; }
|
||||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||||
@@ -38,6 +39,7 @@ preflight() {
|
|||||||
kubectl cluster-info >/dev/null || die "kubectl not configured / cluster unreachable"
|
kubectl cluster-info >/dev/null || die "kubectl not configured / cluster unreachable"
|
||||||
command -v helm >/dev/null || die "helm 3 not found"
|
command -v helm >/dev/null || die "helm 3 not found"
|
||||||
[[ -f "$SOPS_KEY" ]] || die "SOPS age key missing at $SOPS_KEY"
|
[[ -f "$SOPS_KEY" ]] || die "SOPS age key missing at $SOPS_KEY"
|
||||||
|
[[ -f "$DEPLOY_KEY" ]] || die "GitHub deploy key missing at $DEPLOY_KEY (see phase4-argocd/seed-repo-secret.example.yaml)"
|
||||||
log "✅ preflight ok"
|
log "✅ preflight ok"
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -192,10 +194,13 @@ p3_forgejo() {
|
|||||||
p4_argocd() {
|
p4_argocd() {
|
||||||
phase "PHASE 4: ArgoCD (seeded from GitHub)"
|
phase "PHASE 4: ArgoCD (seeded from GitHub)"
|
||||||
|
|
||||||
# Always ensure namespace (idempotent). The seed repo is public — ArgoCD clones
|
# Always ensure namespace + repository secret (idempotent)
|
||||||
# it anonymously over HTTPS, so there is no repository Secret to create.
|
|
||||||
kubectl create ns argocd --dry-run=client -o yaml | kubectl apply -f -
|
kubectl create ns argocd --dry-run=client -o yaml | kubectl apply -f -
|
||||||
|
kubectl -n argocd create secret generic seed-github-repo \
|
||||||
|
--from-literal=type=git --from-literal=url="$GITHUB_SSH" \
|
||||||
|
--from-file=sshPrivateKey="$DEPLOY_KEY" --dry-run=client -o yaml | kubectl apply -f -
|
||||||
|
kubectl -n argocd label secret seed-github-repo argocd.argoproj.io/secret-type=repository --overwrite 2>/dev/null || true
|
||||||
|
|
||||||
# Decrypt and apply any encrypted secrets from bootstrap dir (local SOPS)
|
# Decrypt and apply any encrypted secrets from bootstrap dir (local SOPS)
|
||||||
if command -v sops &> /dev/null; then
|
if command -v sops &> /dev/null; then
|
||||||
export SOPS_AGE_KEY_FILE="$SOPS_KEY"
|
export SOPS_AGE_KEY_FILE="$SOPS_KEY"
|
||||||
|
|||||||
@@ -1,18 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
settings.json: |
|
|
||||||
{
|
|
||||||
"defaultProvider": "homelab-ornith",
|
|
||||||
"defaultModel": "ornith:35b",
|
|
||||||
"defaultThinkingLevel": "medium",
|
|
||||||
"theme": "light",
|
|
||||||
"compaction": {
|
|
||||||
"enabled": true,
|
|
||||||
"reserveTokens": 16000,
|
|
||||||
"keepRecentTokens": 6000
|
|
||||||
}
|
|
||||||
}
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: pi-config
|
|
||||||
namespace: agent-pod
|
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
# Exposes agent-hub at api.riotpiao.com/console (WebSocket) and /run
|
|
||||||
# (trigger a new session) -- both are routes on the same hub.js service.
|
|
||||||
#
|
|
||||||
# Was ingressClassName: kong until Kong was retired on 2026-08-19. Pointed
|
|
||||||
# straight at nginx rather than through the replacement Go gateway because that
|
|
||||||
# gateway has no WebSocket upgrade support yet -- routing /console through it
|
|
||||||
# would break the console outright. nginx handles the upgrade natively.
|
|
||||||
#
|
|
||||||
# Path precedence: the nginx Ingress api/api catch-alls `/` on this same host
|
|
||||||
# to the gateway. nginx matches longest prefix first, so these three paths win
|
|
||||||
# over `/` and the rest of the host still reaches the gateway.
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: Ingress
|
|
||||||
metadata:
|
|
||||||
name: console
|
|
||||||
namespace: agent-pod
|
|
||||||
annotations:
|
|
||||||
# A console WebSocket stays open across a whole agent session; nginx's 60s
|
|
||||||
# default read timeout would drop it mid-run.
|
|
||||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
|
||||||
spec:
|
|
||||||
ingressClassName: nginx
|
|
||||||
rules:
|
|
||||||
- host: api.riotpiao.com
|
|
||||||
http:
|
|
||||||
paths:
|
|
||||||
- path: /console
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: agent-hub
|
|
||||||
port:
|
|
||||||
number: 9090
|
|
||||||
- path: /run
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: agent-hub
|
|
||||||
port:
|
|
||||||
number: 9090
|
|
||||||
- path: /sessions
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: agent-hub
|
|
||||||
port:
|
|
||||||
number: 9090
|
|
||||||
@@ -1,982 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: coordinator-src
|
|
||||||
namespace: agent-pod
|
|
||||||
data:
|
|
||||||
coordinator.js: |
|
|
||||||
#!/usr/bin/env node
|
|
||||||
// coordinator: local CLI that drives a multi-phase, multi-task pipeline of
|
|
||||||
// planner/investigator/implementer/judge stages, same state machine as
|
|
||||||
// hub.js's old /pipeline handler. Every interactive stage runs through the
|
|
||||||
// patched agent-manager fork's `spawn` subcommand, so it is a real tracked
|
|
||||||
// session (tmux pane on agent-manager's private socket + a state.db row)
|
|
||||||
// from the moment it exists -- attachable and visible in agent-manager's
|
|
||||||
// own TUI the whole time it runs.
|
|
||||||
//
|
|
||||||
// Per repo, each role (planner/investigator/implementer/judge) is ONE
|
|
||||||
// persistent agent-manager session, not a fresh spawn per task: the first
|
|
||||||
// task to need a role spawns it, every later task for that role reuses the
|
|
||||||
// same tmux pane via `tmux send-keys` (see runOnPool) -- the same nudge
|
|
||||||
// mechanism that used to only fire on a stall now doubles as "give this
|
|
||||||
// agent its next task." Every role reads everything it needs fresh off disk
|
|
||||||
// each call, so every pane gets a `/new` before every reuse instead of
|
|
||||||
// accumulating history that degrades and eventually errors out task after
|
|
||||||
// task -- same pane, same agent-manager session, zero memory of the last
|
|
||||||
// task it handled.
|
|
||||||
// Because only one implementer/judge/etc. exists per repo, tasks within a
|
|
||||||
// phase run strictly sequentially against the pool -- no per-task worktree,
|
|
||||||
// no per-task branch, no merge-back step; every task commits straight onto
|
|
||||||
// the phase branch in the repo's one shared clone.
|
|
||||||
//
|
|
||||||
// The unit of concurrency is now the REPO, not the task: runCoordinator
|
|
||||||
// takes a list of repos and runs up to REPO_CONCURRENCY of them at once,
|
|
||||||
// each with its own clone (under WORK_DIR/<repoId>) and its own 4-agent
|
|
||||||
// pool. The coordinator never kills a role's session; it rests at an idle
|
|
||||||
// prompt between tasks, and agent-manager's session list becomes the audit
|
|
||||||
// trail of everything every repo's pipeline ran. Completion is signaled by
|
|
||||||
// sentinel files under the repo's clone (unchanged convention), waited on
|
|
||||||
// with fs.watch instead of polling.
|
|
||||||
const fs = require("node:fs");
|
|
||||||
const path = require("node:path");
|
|
||||||
const { spawn } = require("node:child_process");
|
|
||||||
|
|
||||||
const WORK_DIR = process.env.HUB_WORK_DIR || path.join(require("node:os").tmpdir(), "agent-harness-work");
|
|
||||||
|
|
||||||
// Never rely on a bare `pi`/`agent-manager` on $PATH -- see PI_BIN's own
|
|
||||||
// comment below; the same collision risk applies to any CLI name. Always
|
|
||||||
// invoke explicit pinned paths.
|
|
||||||
const PI_BIN =
|
|
||||||
process.env.PI_BIN ||
|
|
||||||
path.join(__dirname, "..", ".pi-cli", "node_modules", "@earendil-works", "pi-coding-agent", "dist", "cli.js");
|
|
||||||
|
|
||||||
const AGENT_MANAGER_BIN = process.env.AGENT_MANAGER_BIN || path.join(__dirname, "..", ".bin", "agent-manager-fork");
|
|
||||||
|
|
||||||
// agent-manager's own session-state DB -- used to detect a session that has
|
|
||||||
// actually died (process crashed/exited, status flips to "errored"/"dead")
|
|
||||||
// instead of one that's merely slow. Read-only introspection plus the one
|
|
||||||
// UPDATE in killDeadSession below, same class of operation as the tmux
|
|
||||||
// nudges already done directly against agent-manager's internals.
|
|
||||||
const AGENT_MANAGER_DB =
|
|
||||||
process.env.AGENT_MANAGER_DB || path.join(require("node:os").homedir(), ".config", "agent-manager", "state.db");
|
|
||||||
|
|
||||||
// Empty means "let pi fall back to ~/.pi/agent/settings.json's default"
|
|
||||||
// (currently anthropic/claude-sonnet-4-5, real paid usage). Set both to
|
|
||||||
// route every stage -- headless (spawnPi) and interactive (runOnPool) --
|
|
||||||
// at the homelab model instead, e.g. AGENT_PROVIDER=homelab-ornith
|
|
||||||
// AGENT_MODEL=ornith:35b.
|
|
||||||
const AGENT_PROVIDER = process.env.AGENT_PROVIDER || "";
|
|
||||||
const AGENT_MODEL = process.env.AGENT_MODEL || "";
|
|
||||||
|
|
||||||
// judge can run a different model than the rest of the chain, e.g.
|
|
||||||
// homelab-reasoning instead of homelab-ornith now that verifier/PRM is
|
|
||||||
// retired. Falls back to AGENT_PROVIDER/AGENT_MODEL when unset, so a run
|
|
||||||
// that doesn't care keeps one uniform model everywhere.
|
|
||||||
const JUDGE_PROVIDER = process.env.JUDGE_PROVIDER || AGENT_PROVIDER;
|
|
||||||
const JUDGE_MODEL = process.env.JUDGE_MODEL || AGENT_MODEL;
|
|
||||||
|
|
||||||
function providerModelFor(role) {
|
|
||||||
return role === "judge" ? { provider: JUDGE_PROVIDER, model: JUDGE_MODEL } : { provider: AGENT_PROVIDER, model: AGENT_MODEL };
|
|
||||||
}
|
|
||||||
|
|
||||||
// agent-manager's private tmux server and session-naming scheme
|
|
||||||
// (internal/tmux/tmux.go: defaultSocket = "agentmgr", sessionName(id) =
|
|
||||||
// "am_"+id) -- stable, documented internals of the fork, used here only
|
|
||||||
// for read-only introspection (pane capture) and role nudges, exactly the
|
|
||||||
// class of operation hub.js already ran directly against its own sessions
|
|
||||||
// rather than asking a model to do it.
|
|
||||||
const AM_SOCKET = "agentmgr";
|
|
||||||
function amSessionName(id) {
|
|
||||||
return `am_${id}`;
|
|
||||||
}
|
|
||||||
function runAmTmux(args) {
|
|
||||||
return runCmd("tmux", ["-L", AM_SOCKET, ...args]);
|
|
||||||
}
|
|
||||||
|
|
||||||
const ROLE_SKILLS = new Set(["planner", "investigator", "info-collector", "implementer", "judge", "resolver"]);
|
|
||||||
|
|
||||||
// Every role reads everything it needs fresh off disk each call -- PLAN.md,
|
|
||||||
// the task spec, judge's verdict file, `git diff` against baseBranch --
|
|
||||||
// nothing depends on remembering earlier tasks. Left to accumulate, a
|
|
||||||
// pooled session's conversation grows without bound across every task in a
|
|
||||||
// repo and both correctness and reliability degrade hard once it does
|
|
||||||
// (observed: a planner session at ~1.5M cumulative tokens started erroring
|
|
||||||
// out every call, an investigator session that far gone started narrating a
|
|
||||||
// different codebase entirely). So every role gets reset to a clean
|
|
||||||
// conversation before every reuse instead of just being nudged with the
|
|
||||||
// next prompt -- same pane, same agent-manager session (still
|
|
||||||
// visible/attachable), zero history carried between tasks.
|
|
||||||
|
|
||||||
function sleep(ms) {
|
|
||||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
|
||||||
}
|
|
||||||
|
|
||||||
const HARD_RULES =
|
|
||||||
"Read and follow ~/.pi/agent/skills/karpathy-guidelines/SKILL.md and " +
|
|
||||||
"~/.pi/agent/skills/caveman/SKILL.md as hard rules for this entire task, before anything else. ";
|
|
||||||
|
|
||||||
// judge (routed to homelab-reasoning) has been observed narrating an
|
|
||||||
// entire review in prose -- "I should run git diff, then check X..." --
|
|
||||||
// and then writing a verdict based on that narration without ever calling
|
|
||||||
// a real tool. Live example: a phase-judge call produced a page of
|
|
||||||
// "I would check..." reasoning, declared VERDICT: PASS, and showed the
|
|
||||||
// touch command as a fenced code block IN ITS OWN TEXT rather than
|
|
||||||
// executing it. Coordinator just timed out waiting on a sentinel that was
|
|
||||||
// never going to appear, since nothing was ever actually run. Spelled out
|
|
||||||
// explicitly since "use the judge skill" alone apparently isn't enough to
|
|
||||||
// rule this out.
|
|
||||||
const REQUIRE_REAL_TOOL_CALLS =
|
|
||||||
"Do not narrate what you would check -- actually run the commands via a real tool call and read their real " +
|
|
||||||
"output before writing anything. A verdict based on describing checks instead of executing them is invalid. " +
|
|
||||||
"Writing the verdict file and touching the sentinel are themselves tool calls you must execute, not text to " +
|
|
||||||
"display in your response. ";
|
|
||||||
|
|
||||||
function parseVerdictLine(text, label) {
|
|
||||||
if (!text) return null;
|
|
||||||
const re = new RegExp(`${label}:\\s*(\\w+)`, "i");
|
|
||||||
const m = text.match(re);
|
|
||||||
return m ? m[1].toUpperCase() : null;
|
|
||||||
}
|
|
||||||
|
|
||||||
function runCmd(bin, args, cwd) {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
const child = spawn(bin, args, { cwd, stdio: ["ignore", "pipe", "pipe"] });
|
|
||||||
let out = "";
|
|
||||||
child.stdout.on("data", (c) => (out += c));
|
|
||||||
child.stderr.on("data", (c) => (out += c));
|
|
||||||
child.on("close", (code) => resolve({ code, out: out.trim() }));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function runGit(cwd, args) {
|
|
||||||
return runCmd("git", args, cwd);
|
|
||||||
}
|
|
||||||
|
|
||||||
// A stage saying "commit" in its prompt is a request, not a guarantee -- seen
|
|
||||||
// in practice: a stage writes a real file and simply never runs `git add`/
|
|
||||||
// `git commit`, leaving it untracked and invisible to every later `git diff`.
|
|
||||||
// Sweep and commit anything left dirty after every stage, deterministically.
|
|
||||||
async function commitPending(cwd, message) {
|
|
||||||
await runGit(cwd, ["add", "-A"]);
|
|
||||||
const status = await runGit(cwd, ["status", "--porcelain"]);
|
|
||||||
if (!status.out) return { committed: false };
|
|
||||||
const commit = await runGit(cwd, ["commit", "-m", message]);
|
|
||||||
return { committed: commit.code === 0, error: commit.code !== 0 ? commit.out : undefined };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Headless one-shot pi call (`pi -p --mode json <prompt>`), used only for
|
|
||||||
// quick diagnostic/mechanical calls that don't need to be a watchable
|
|
||||||
// session: resolver's crash/stall diagnosis, and the initial clone. Kept
|
|
||||||
// exactly as before -- only the interactive per-role stages (runOnPool,
|
|
||||||
// below) go through agent-manager.
|
|
||||||
//
|
|
||||||
// Bounded by SPAWN_PI_TIMEOUT_MS -- unlike runOnPool's pooled sessions
|
|
||||||
// (which now have status polling to catch a dead session fast, see
|
|
||||||
// waitForSentinel/killDeadSession), this is a raw child_process with no
|
|
||||||
// equivalent escape hatch. Observed live: a resolver call shared the
|
|
||||||
// default backend with a concurrently-busy repo's implementer and sat for
|
|
||||||
// 6+ minutes producing nothing -- with no timeout here, that blocks the
|
|
||||||
// entire calling repo's pipeline forever, since askResolver is always
|
|
||||||
// awaited before the next stage can run.
|
|
||||||
const SPAWN_PI_TIMEOUT_MS = 5 * 60 * 1000;
|
|
||||||
|
|
||||||
function spawnPi({ agent, prompt, cwd }) {
|
|
||||||
const finalPrompt = ROLE_SKILLS.has(agent) ? `/skill:${agent} ${HARD_RULES}${prompt}` : prompt;
|
|
||||||
const args = ["-p", "--mode", "json"];
|
|
||||||
if (AGENT_PROVIDER) args.push("--provider", AGENT_PROVIDER);
|
|
||||||
if (AGENT_MODEL) args.push("--model", AGENT_MODEL);
|
|
||||||
args.push(finalPrompt);
|
|
||||||
const child = spawn(PI_BIN, args, { stdio: ["ignore", "pipe", "pipe"], cwd });
|
|
||||||
let lastText = "";
|
|
||||||
let stderrTail = "";
|
|
||||||
let buf = "";
|
|
||||||
child.stdout.on("data", (chunk) => {
|
|
||||||
buf += chunk;
|
|
||||||
let idx;
|
|
||||||
while ((idx = buf.indexOf("\n")) !== -1) {
|
|
||||||
const line = buf.slice(0, idx);
|
|
||||||
buf = buf.slice(idx + 1);
|
|
||||||
if (!line.trim()) continue;
|
|
||||||
try {
|
|
||||||
const event = JSON.parse(line);
|
|
||||||
if (event.type === "message_end" && event.message && Array.isArray(event.message.content)) {
|
|
||||||
const text = event.message.content
|
|
||||||
.filter((c) => c.type === "text")
|
|
||||||
.map((c) => c.text)
|
|
||||||
.join("\n");
|
|
||||||
if (text) lastText = text;
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
// non-JSON stdout noise, ignore
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
child.stderr.on("data", (chunk) => {
|
|
||||||
process.stderr.write(chunk);
|
|
||||||
stderrTail = (stderrTail + chunk.toString()).slice(-4000);
|
|
||||||
});
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
let settled = false;
|
|
||||||
const timer = setTimeout(() => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
child.kill("SIGKILL");
|
|
||||||
resolve({ code: null, lastText, stderrTail, timedOut: true });
|
|
||||||
}, SPAWN_PI_TIMEOUT_MS);
|
|
||||||
child.on("close", (code) => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
clearTimeout(timer);
|
|
||||||
resolve({ code, lastText, stderrTail });
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async function askResolver(cwd, repoId, diagnosticPrompt) {
|
|
||||||
const result = await spawnPi({ agent: "resolver", prompt: diagnosticPrompt, cwd });
|
|
||||||
return parseVerdictLine(result.lastText, "RESOLUTION");
|
|
||||||
}
|
|
||||||
|
|
||||||
const STAGE_TIMEOUT_MS = 10 * 60 * 1000;
|
|
||||||
const NUDGE_TIMEOUT_MS = 5 * 60 * 1000;
|
|
||||||
|
|
||||||
// Resolves as soon as filePath appears (fs.watch on its directory, same as
|
|
||||||
// before), as soon as target's agent-manager status flips to "errored" or
|
|
||||||
// "dead" (polled -- state.db has no watch mechanism), or after limitMs with
|
|
||||||
// neither. A session that has actually crashed will never touch the
|
|
||||||
// sentinel, so without the status poll this just burns the full STAGE_
|
|
||||||
// TIMEOUT_MS waiting on a file that was never coming, same as a genuine
|
|
||||||
// stall -- polling status catches that in ~pollMs instead.
|
|
||||||
function waitForSentinel(filePath, target, limitMs, pollMs = 5000) {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
if (fs.existsSync(filePath)) return resolve({ ok: true });
|
|
||||||
const dir = path.dirname(filePath);
|
|
||||||
const id = target.replace(/^am_/, "");
|
|
||||||
let settled = false;
|
|
||||||
let watcher;
|
|
||||||
let poller;
|
|
||||||
let timer;
|
|
||||||
const finish = (result) => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
clearTimeout(timer);
|
|
||||||
clearInterval(poller);
|
|
||||||
if (watcher) {
|
|
||||||
try {
|
|
||||||
watcher.close();
|
|
||||||
} catch {
|
|
||||||
// already closed
|
|
||||||
}
|
|
||||||
}
|
|
||||||
resolve(result);
|
|
||||||
};
|
|
||||||
try {
|
|
||||||
watcher = fs.watch(dir, () => {
|
|
||||||
if (fs.existsSync(filePath)) finish({ ok: true });
|
|
||||||
});
|
|
||||||
} catch {
|
|
||||||
// dir missing at watch time is a real bug elsewhere (cwd should
|
|
||||||
// already exist); surface it as a timeout rather than hang forever.
|
|
||||||
return finish({ timedOut: true });
|
|
||||||
}
|
|
||||||
// Closes the race between the existsSync check above and the watcher
|
|
||||||
// actually being attached.
|
|
||||||
if (fs.existsSync(filePath)) return finish({ ok: true });
|
|
||||||
poller = setInterval(async () => {
|
|
||||||
const { out } = await runCmd("sqlite3", [AGENT_MANAGER_DB, `SELECT status FROM sessions WHERE id='${id}'`]);
|
|
||||||
const status = out.trim();
|
|
||||||
if (status === "errored" || status === "dead") finish({ dead: true, status });
|
|
||||||
}, pollMs);
|
|
||||||
timer = setTimeout(() => finish({ timedOut: true }), limitMs);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Kills a session that's actually crashed (not just slow) and archives it
|
|
||||||
// in agent-manager's own DB so it stops showing up as a live, unattended
|
|
||||||
// pane -- otherwise every crash leaves an orphaned tmux session + state.db
|
|
||||||
// row behind permanently, identical to the manually-cleaned-up poiman-
|
|
||||||
// planner ghost session found earlier this same run.
|
|
||||||
async function killDeadSession(target) {
|
|
||||||
await runAmTmux(["kill-session", "-t", target]);
|
|
||||||
const id = target.replace(/^am_/, "");
|
|
||||||
await runCmd("sqlite3", [AGENT_MANAGER_DB, `UPDATE sessions SET archived=1 WHERE id='${id}'`]);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Runs one task's worth of work on a persistent per-role agent: spawns the
|
|
||||||
// role's session the first time it's ever needed for this repo, sends every
|
|
||||||
// later prompt into that same tmux pane via send-keys -- prefixed with a
|
|
||||||
// `/new` first, so the pane and agent-manager session stay the same but the
|
|
||||||
// model starts that prompt with a clean conversation, no history carried
|
|
||||||
// over from whatever task this role last handled. pool is a plain object
|
|
||||||
// keyed by role name ("planner"/"investigator"/"implementer"/"judge"),
|
|
||||||
// shared across every task in a repo's pipeline (see runRepoPipeline) -- it
|
|
||||||
// IS the 4-agent pool, one entry per role, filled in lazily as each role
|
|
||||||
// gets its first task.
|
|
||||||
// A dead/errored session gets one respawn-and-retry (same prompt, fresh
|
|
||||||
// session) before this stage is abandoned -- matches resolver-SKILL.md's
|
|
||||||
// own documented contract of retrying a failed stage at most once.
|
|
||||||
const DEAD_SESSION_RETRIES = 1;
|
|
||||||
|
|
||||||
async function runOnPool(pool, cwd, repoId, role, prompt, sentinelFile) {
|
|
||||||
fs.rmSync(sentinelFile, { force: true });
|
|
||||||
const label = `${repoId}-${role}`;
|
|
||||||
|
|
||||||
// A pooled session's shell cwd drifts as it explores the repo (e.g. cd
|
|
||||||
// into a Rust workspace subdirectory to read source) and nothing resets
|
|
||||||
// it back between turns. Seen in practice: a repo whose own internal
|
|
||||||
// workspace folder is one letter off from the repo's own directory name
|
|
||||||
// ("poiman" the repo vs. "poimen" the crate workspace inside it) was
|
|
||||||
// enough for the agent to touch its sentinel one level off from where
|
|
||||||
// this function is watching for it -- coordinator waits out the full
|
|
||||||
// STAGE_TIMEOUT_MS for a file that already exists, just in the wrong
|
|
||||||
// place. State the absolute target directory and use absolute paths for
|
|
||||||
// every filesystem instruction, so there's nothing for the agent to get
|
|
||||||
// wrong by reasoning about a relative "current directory."
|
|
||||||
const cwdReminder = `Your working directory for this task is ${cwd} -- if your shell isn't already there, run: cd ${cwd}\n\n`;
|
|
||||||
|
|
||||||
const spawnFresh = async () => {
|
|
||||||
const spawnArgs = ["spawn", "--tool", "pi", "--cwd", cwd, "--name", label, "--group", repoId, "--prompt", cwdReminder + HARD_RULES + prompt];
|
|
||||||
const { provider, model } = providerModelFor(role);
|
|
||||||
if (provider) spawnArgs.push("--provider", provider);
|
|
||||||
if (model) spawnArgs.push("--model", model);
|
|
||||||
const spawned = await runCmd(AGENT_MANAGER_BIN, spawnArgs);
|
|
||||||
return spawned.code === 0 ? amSessionName(spawned.out) : null;
|
|
||||||
};
|
|
||||||
|
|
||||||
let target = pool[role];
|
|
||||||
if (!target) {
|
|
||||||
target = await spawnFresh();
|
|
||||||
if (!target) return { ok: false, crashed: true, error: "spawn failed", sessionName: label };
|
|
||||||
pool[role] = target;
|
|
||||||
} else {
|
|
||||||
await runAmTmux(["send-keys", "-t", target, "/new", "Enter"]);
|
|
||||||
await sleep(1000);
|
|
||||||
await runAmTmux(["send-keys", "-t", target, cwdReminder + HARD_RULES + prompt, "Enter"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
for (let deadRetries = 0; ; deadRetries++) {
|
|
||||||
const outcome = await waitForSentinel(sentinelFile, target, STAGE_TIMEOUT_MS);
|
|
||||||
if (outcome.ok) return { ok: true, sessionName: label };
|
|
||||||
|
|
||||||
if (outcome.dead) {
|
|
||||||
await killDeadSession(target);
|
|
||||||
if (pool[role] === target) delete pool[role];
|
|
||||||
if (deadRetries >= DEAD_SESSION_RETRIES) {
|
|
||||||
return { ok: false, crashed: true, error: `session died (status: ${outcome.status})`, sessionName: label };
|
|
||||||
}
|
|
||||||
target = await spawnFresh();
|
|
||||||
if (!target) return { ok: false, crashed: true, error: "respawn after death failed", sessionName: label };
|
|
||||||
pool[role] = target;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Plain stall -- session still alive, just slow. Ask resolver once,
|
|
||||||
// nudge if it says worth it, and stop here either way (this is not
|
|
||||||
// the death path, so no respawn/retry loop).
|
|
||||||
const pane = await runAmTmux(["capture-pane", "-t", target, "-p", "-S", "-200"]);
|
|
||||||
const resolution = await askResolver(
|
|
||||||
cwd,
|
|
||||||
repoId,
|
|
||||||
`Repo ${repoId}'s "${role}" agent hasn't finished its current task after 10 minutes. Its pane tail:\n${pane.out.slice(-3000)}\n\n` +
|
|
||||||
`Decide: is it still making real progress and worth nudging to wrap up, or stuck and worth abandoning?`
|
|
||||||
);
|
|
||||||
let ok = false;
|
|
||||||
if (resolution === "RETRY") {
|
|
||||||
await runAmTmux(["send-keys", "-t", target, `Please wrap up now and run: touch ${sentinelFile}`, "Enter"]);
|
|
||||||
const nudged = await waitForSentinel(sentinelFile, target, NUDGE_TIMEOUT_MS);
|
|
||||||
ok = nudged.ok === true;
|
|
||||||
if (nudged.dead) {
|
|
||||||
await killDeadSession(target);
|
|
||||||
if (pool[role] === target) delete pool[role];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return { ok, sessionName: label };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function plannerPrompt(task, specHint, judgeOnly, cwd) {
|
|
||||||
// judgeOnly (auto-discovered tasks only, see parseTaskBoard): planner
|
|
||||||
// itself decides whether the task is already done before planning it,
|
|
||||||
// reading tasks/INDEX.md's own status notes plus git log/current code --
|
|
||||||
// replaces what used to be a separate judge pre-check call. One LLM round
|
|
||||||
// trip instead of two, and the same agent that's about to plan the task
|
|
||||||
// is the one deciding whether planning it is even necessary.
|
|
||||||
const resultFile = path.join(cwd, `.task-result-${task}`);
|
|
||||||
const decideStep = judgeOnly
|
|
||||||
? `First, decide whether task ${task} is already fully implemented on this branch: check ` +
|
|
||||||
`\`git log --oneline --grep '${task}'\`, tasks/INDEX.md's own status notes for this task, and the current ` +
|
|
||||||
`code directly against its spec (${specHint})'s acceptance criteria. Write your decision to ` +
|
|
||||||
`${resultFile} as a single "VERDICT: PASS" (already done, no further work needed) or ` +
|
|
||||||
`"VERDICT: FAIL" (needs work) line plus one line of rationale. If VERDICT is FAIL, continue below and ` +
|
|
||||||
`draft the plan in this same turn; if VERDICT is PASS, skip the rest and go straight to the touch step.\n\n`
|
|
||||||
: "";
|
|
||||||
return (
|
|
||||||
`${decideStep}Use the planner skill to draft PLAN.md for task ${task}, reading its spec (${specHint}). ` +
|
|
||||||
`PLAN.md is scratch state for this harness, not a deliverable -- do NOT commit it or add it to git. ` +
|
|
||||||
`Then run: touch ${path.join(cwd, `.stage-done-${task}-planner`)}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function investigatorPrompt(task, cwd) {
|
|
||||||
return (
|
|
||||||
`Use the investigator skill to confirm PLAN.md against real sources for task ${task}, append findings. ` +
|
|
||||||
`PLAN.md is scratch state for this harness, not a deliverable -- do NOT commit it or add it to git. ` +
|
|
||||||
`Then run: touch ${path.join(cwd, `.stage-done-${task}-investigator`)}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function implementerPrompt(task, attempt, feedbackHint, cwd) {
|
|
||||||
return (
|
|
||||||
`Use the implementer skill to implement what the current PLAN.md specifies for task ${task} (commit as you go). ` +
|
|
||||||
`${feedbackHint} Then run: touch ${path.join(cwd, `.stage-done-${task}-implementer-${attempt}`)}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function judgePrompt(task, baseBranch, attempt, cwd) {
|
|
||||||
return (
|
|
||||||
`${REQUIRE_REAL_TOOL_CALLS}Use the judge skill to review the diff against ${baseBranch}...HEAD for task ${task}. ` +
|
|
||||||
`Write your verdict to ${path.join(cwd, `.task-result-${task}`)} as a single "VERDICT: PASS" or "VERDICT: FAIL" ` +
|
|
||||||
`line plus one line of rationale, then run: touch ${path.join(cwd, `.stage-done-${task}-judge-${attempt}`)}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
const MAX_IMPLEMENT_ATTEMPTS = 5;
|
|
||||||
const MAX_PLAN_REVISIONS = 3;
|
|
||||||
|
|
||||||
// Runs one task against the repo's shared role pool: planner drafts
|
|
||||||
// PLAN.md (for auto-discovered tasks, first deciding off tasks/INDEX.md and
|
|
||||||
// the repo's own state whether the task is already done -- see
|
|
||||||
// plannerPrompt's judgeOnly branch; judge never does this pre-check),
|
|
||||||
// investigator confirms it, then implementer and judge go back and
|
|
||||||
// forth -- judge's FAIL rationale lands in .task-result-<task>, which the
|
|
||||||
// next implementer attempt is told to read and address. After
|
|
||||||
// MAX_IMPLEMENT_ATTEMPTS straight fails, the planner role is asked to judge
|
|
||||||
// whether the plan itself is wrong -- fresh conversation, same as any other
|
|
||||||
// planner call, reading PLAN.md/the judge feedback/the
|
|
||||||
// diff off disk rather than remembering having drafted the original plan.
|
|
||||||
// If it decides the approach is wrong it revises PLAN.md and the implementer
|
|
||||||
// gets a fresh attempt budget.
|
|
||||||
// MAX_PLAN_REVISIONS caps this from looping forever on a task that's
|
|
||||||
// genuinely stuck. All work happens directly in cwd (the repo's one shared
|
|
||||||
// clone, currently checked out to the phase branch) -- no worktree, since
|
|
||||||
// only one implementer/judge exist per repo and tasks run strictly one at a
|
|
||||||
// time (see runPhase).
|
|
||||||
async function runTaskOnPool(cwd, baseBranch, task, pool, repoId, pipelineSession, judgeOnly) {
|
|
||||||
const resultFile = path.join(cwd, `.task-result-${task}`);
|
|
||||||
fs.rmSync(resultFile, { force: true });
|
|
||||||
|
|
||||||
const specHint = `the file under tasks/ starting with "${task}-"`;
|
|
||||||
|
|
||||||
const stage = async (role, prompt, sentinel, displayLabel) => {
|
|
||||||
const label = displayLabel || role;
|
|
||||||
pipelineSession.activeTasks[task] = { stage: label, startedAt: new Date().toISOString() };
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
const result = await runOnPool(pool, cwd, repoId, role, prompt, sentinel);
|
|
||||||
await commitPending(cwd, `task: ${task} (${label})`);
|
|
||||||
return result;
|
|
||||||
};
|
|
||||||
|
|
||||||
const abandon = (stageLabel, result, attempt) => {
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
return {
|
|
||||||
task,
|
|
||||||
status: result.crashed ? "spawn-crashed" : "timed-out",
|
|
||||||
error: result.error,
|
|
||||||
stoppedAt: stageLabel,
|
|
||||||
...(attempt !== undefined ? { attempt } : {}),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
// PLAN.md is scratch state for this one task, not a deliverable (see
|
|
||||||
// plannerPrompt/investigatorPrompt -- it's gitignored too, as a backstop
|
|
||||||
// in case an agent commits it anyway). Discard it once the task is done,
|
|
||||||
// whatever the outcome, so it never bleeds into the next task's planner
|
|
||||||
// call or sits around as stale harness clutter in the shared clone.
|
|
||||||
try {
|
|
||||||
let result = await stage("planner", plannerPrompt(task, specHint, judgeOnly, cwd), path.join(cwd, `.stage-done-${task}-planner`));
|
|
||||||
if (!result.ok) return abandon("planner", result);
|
|
||||||
|
|
||||||
if (judgeOnly) {
|
|
||||||
const quickText = fs.existsSync(resultFile) ? fs.readFileSync(resultFile, "utf8") : "";
|
|
||||||
if (parseVerdictLine(quickText, "VERDICT") === "PASS") {
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
return { task, status: "done", judgeRationale: quickText, judgeOnlyPass: true };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
result = await stage("investigator", investigatorPrompt(task, cwd), path.join(cwd, `.stage-done-${task}-investigator`));
|
|
||||||
if (!result.ok) return abandon("investigator", result);
|
|
||||||
|
|
||||||
let planRevisions = 0;
|
|
||||||
let implementAttempt = 0;
|
|
||||||
let verdict = null;
|
|
||||||
let resultText = "";
|
|
||||||
let justRevisedPlan = false;
|
|
||||||
|
|
||||||
while (true) {
|
|
||||||
implementAttempt++;
|
|
||||||
const feedbackHint = fs.existsSync(resultFile)
|
|
||||||
? justRevisedPlan
|
|
||||||
? `${resultFile} holds the judge's feedback against the OLD plan, which prompted a plan revision -- ` +
|
|
||||||
`PLAN.md has since changed. Read the current PLAN.md as the source of truth, not the old feedback verbatim.`
|
|
||||||
: `A previous judge review exists at ${resultFile} -- read it and address every issue it raises.`
|
|
||||||
: "";
|
|
||||||
justRevisedPlan = false;
|
|
||||||
|
|
||||||
result = await stage(
|
|
||||||
"implementer",
|
|
||||||
implementerPrompt(task, implementAttempt, feedbackHint, cwd),
|
|
||||||
path.join(cwd, `.stage-done-${task}-implementer-${implementAttempt}`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("implementer", result, implementAttempt);
|
|
||||||
|
|
||||||
result = await stage("judge", judgePrompt(task, baseBranch, implementAttempt, cwd), path.join(cwd, `.stage-done-${task}-judge-${implementAttempt}`));
|
|
||||||
if (!result.ok) return abandon("judge", result, implementAttempt);
|
|
||||||
|
|
||||||
resultText = fs.existsSync(resultFile) ? fs.readFileSync(resultFile, "utf8") : "";
|
|
||||||
verdict = parseVerdictLine(resultText, "VERDICT");
|
|
||||||
if (verdict === "PASS") break;
|
|
||||||
|
|
||||||
if (implementAttempt >= MAX_IMPLEMENT_ATTEMPTS) {
|
|
||||||
if (planRevisions >= MAX_PLAN_REVISIONS) break;
|
|
||||||
planRevisions++;
|
|
||||||
result = await stage(
|
|
||||||
"planner",
|
|
||||||
`Implementer failed judge review ${MAX_IMPLEMENT_ATTEMPTS} times in a row for task ${task}. Read PLAN.md, ` +
|
|
||||||
`the judge's feedback in ${resultFile}, and the current diff against ${baseBranch}...HEAD. Decide ` +
|
|
||||||
`whether the plan's approach itself is wrong, not just the implementation -- if so, revise PLAN.md. If ` +
|
|
||||||
`you change the approach, also use the investigator skill to confirm the new approach against real ` +
|
|
||||||
`sources. If the plan is sound, note why in PLAN.md and leave it as-is. PLAN.md is scratch state for ` +
|
|
||||||
`this harness, not a deliverable -- do NOT commit it or add it to git. Then run: ` +
|
|
||||||
`touch ${path.join(cwd, `.stage-done-${task}-planner-revise-${planRevisions}`)}`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-planner-revise-${planRevisions}`),
|
|
||||||
"planner-revise"
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("planner-revise", result, planRevisions);
|
|
||||||
implementAttempt = 0;
|
|
||||||
justRevisedPlan = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
|
|
||||||
if (verdict !== "PASS" && planRevisions >= MAX_PLAN_REVISIONS) {
|
|
||||||
return { task, status: "unresolved", judgeRationale: resultText, implementAttempts: implementAttempt, planRevisions };
|
|
||||||
}
|
|
||||||
return { task, status: verdict === "PASS" ? "done" : "done-with-concerns", judgeRationale: resultText };
|
|
||||||
} finally {
|
|
||||||
fs.rmSync(path.join(cwd, "PLAN.md"), { force: true });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Committed (never gitignored) so it survives a resumed phase branch --
|
|
||||||
// one task id per line, appended as each task resolves. This is what lets
|
|
||||||
// a resumed run skip straight past already-resolved tasks instead of
|
|
||||||
// re-running planner's judgeOnly decision on every one of them again:
|
|
||||||
// resuming the git branch alone only recovers the CODE, not "which tasks
|
|
||||||
// are already settled," and re-deciding that from scratch for every task
|
|
||||||
// burns a full LLM call per already-done task before ever reaching the
|
|
||||||
// first one that actually needs work.
|
|
||||||
function progressLedgerPath(cwd) {
|
|
||||||
return path.join(cwd, ".agent-progress");
|
|
||||||
}
|
|
||||||
|
|
||||||
function readCompletedTasks(cwd) {
|
|
||||||
const file = progressLedgerPath(cwd);
|
|
||||||
if (!fs.existsSync(file)) return new Set();
|
|
||||||
return new Set(
|
|
||||||
fs
|
|
||||||
.readFileSync(file, "utf8")
|
|
||||||
.split("\n")
|
|
||||||
.map((line) => line.trim())
|
|
||||||
.filter(Boolean)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
async function recordTaskComplete(cwd, task) {
|
|
||||||
fs.appendFileSync(progressLedgerPath(cwd), `${task}\n`);
|
|
||||||
await runGit(cwd, ["add", path.basename(progressLedgerPath(cwd))]);
|
|
||||||
await runGit(cwd, ["commit", "-m", `chore: mark ${task} complete in progress ledger`]);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Runs every task in a phase (no declared dependency between them) strictly
|
|
||||||
// one at a time against the repo's shared role pool -- only one implementer/
|
|
||||||
// judge/etc. exists per repo, so there is no per-task concurrency to have
|
|
||||||
// here anymore (see REPO_CONCURRENCY below for where concurrency now
|
|
||||||
// lives). No worktrees: every task commits directly onto phaseBranch in the
|
|
||||||
// one shared cwd. baseBranch here is the TRUE base (e.g. "main") -- judge
|
|
||||||
// reviews `git diff baseBranch...HEAD`, not phaseBranch...HEAD, which would
|
|
||||||
// always be empty since HEAD *is* phaseBranch while it's checked out.
|
|
||||||
//
|
|
||||||
// Pushes phaseBranch after every task, not just once at full-phase-end: the
|
|
||||||
// pod is ephemeral and every restart re-clones baseBranch fresh (see
|
|
||||||
// runRepoPipeline) -- without this, a redeploy mid-phase silently discards
|
|
||||||
// every task committed so far, and the next run re-decides "is this done?"
|
|
||||||
// from a clone that never saw any of that work.
|
|
||||||
async function runPhase(cwd, baseBranch, phaseBranch, phaseTasks, pool, repoId, pipelineSession) {
|
|
||||||
const entries = phaseTasks.map((t) => (typeof t === "string" ? { id: t, judgeOnly: false } : t));
|
|
||||||
const completed = readCompletedTasks(cwd);
|
|
||||||
|
|
||||||
for (const entry of entries) {
|
|
||||||
if (completed.has(entry.id)) {
|
|
||||||
const result = { task: entry.id, status: "done", resumed: true };
|
|
||||||
pipelineSession.taskResults.push(result);
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
const result = await runTaskOnPool(cwd, baseBranch, entry.id, pool, repoId, pipelineSession, entry.judgeOnly);
|
|
||||||
pipelineSession.taskResults.push(result);
|
|
||||||
if (result.status === "done" || result.status === "done-with-concerns") {
|
|
||||||
await recordTaskComplete(cwd, entry.id);
|
|
||||||
}
|
|
||||||
await runGit(cwd, ["push", "-u", "origin", phaseBranch]);
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Discovers phases/tasks from the repo's own tasks/INDEX.md instead of
|
|
||||||
// requiring the caller to pass --tasks. Matches this convention's board
|
|
||||||
// shape (see e.g. Poimen/agent-rust's tasks/INDEX.md): a numbered phase
|
|
||||||
// heading ("## 1 — Foundations · T0.x"), followed by a markdown table
|
|
||||||
// whose rows link to each task's own spec file ("| [T0.1](T0.1-....md) |
|
|
||||||
// ... |"). Headings that aren't a numbered phase (prose sections like
|
|
||||||
// "## Ordering — declared, never derived", "## Progress") are skipped --
|
|
||||||
// only "## <digits> — ..." starts a new phase. Returns null if
|
|
||||||
// tasks/INDEX.md doesn't exist; an empty array if it exists but no phase
|
|
||||||
// yielded any task rows.
|
|
||||||
function parseTaskBoard(cwd) {
|
|
||||||
const indexPath = path.join(cwd, "tasks", "INDEX.md");
|
|
||||||
if (!fs.existsSync(indexPath)) return null;
|
|
||||||
|
|
||||||
const phaseHeaderRe = /^##\s+\d+\s+—/;
|
|
||||||
const taskRowRe = /^\|\s*\[([A-Za-z0-9.]+)\]\(/;
|
|
||||||
|
|
||||||
const phases = [];
|
|
||||||
let current = null;
|
|
||||||
for (const line of fs.readFileSync(indexPath, "utf8").split("\n")) {
|
|
||||||
if (phaseHeaderRe.test(line)) {
|
|
||||||
current = [];
|
|
||||||
phases.push(current);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const m = line.match(taskRowRe);
|
|
||||||
if (m && current) current.push(m[1]);
|
|
||||||
}
|
|
||||||
return phases.filter((phase) => phase.length > 0);
|
|
||||||
}
|
|
||||||
|
|
||||||
function phaseLabelFor(phaseTasks, index) {
|
|
||||||
const first = phaseTasks[0];
|
|
||||||
const id = typeof first === "string" ? first : first.id;
|
|
||||||
const dot = id.indexOf(".");
|
|
||||||
return dot === -1 ? `phase-${index}` : id.slice(0, dot);
|
|
||||||
}
|
|
||||||
|
|
||||||
function logProgress(pipelineSession) {
|
|
||||||
console.log(`[repo ${pipelineSession.id}] ${JSON.stringify(pipelineSession)}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Runs one repo's full pipeline: clone, then phases strictly sequentially.
|
|
||||||
// tasks: array of phases, each phase an array of task ids with no declared
|
|
||||||
// dependency on each other (e.g. [["T0.1","T0.2"], ["T1.1","T1.2","T1.3"]]).
|
|
||||||
// A flat array of ids is also accepted and treated as one single phase. If
|
|
||||||
// omitted, phases are discovered from the repo's own tasks/INDEX.md and run
|
|
||||||
// judgeOnly first (a cheap "is this already done" check against the
|
|
||||||
// board's possibly-stale checkmarks). Each phase gets its own branch
|
|
||||||
// (agent-run/<repoId>/<phaseLabel>, e.g. .../T1); once every task in that
|
|
||||||
// phase lands "done" or "done-with-concerns" AND the phase judge (the same
|
|
||||||
// pooled judge agent that reviewed each task) passes the integration
|
|
||||||
// review, the phase branch is squash-merged into baseBranch and pushed,
|
|
||||||
// then the next phase branches off that updated base. Any failure halts
|
|
||||||
// this repo's pipeline before merging -- it does not affect other repos
|
|
||||||
// running concurrently (see runCoordinator).
|
|
||||||
async function runRepoPipeline({ repoId, repo, baseBranch, tasks, branchName }, pipelineSession) {
|
|
||||||
const cwd = path.join(WORK_DIR, repoId);
|
|
||||||
// repoId is a slug derived from the repo URL now (see slugFor), not a
|
|
||||||
// fresh UUID -- reusable across separate `runCoordinator` invocations
|
|
||||||
// against the same repo, so a stale clone from a prior run has to be
|
|
||||||
// wiped before this one starts, not merged into.
|
|
||||||
fs.rmSync(cwd, { recursive: true, force: true });
|
|
||||||
fs.mkdirSync(cwd, { recursive: true });
|
|
||||||
const pool = {};
|
|
||||||
|
|
||||||
const finish = (status) => {
|
|
||||||
pipelineSession.status = status;
|
|
||||||
pipelineSession.endedAt = new Date().toISOString();
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
return pipelineSession;
|
|
||||||
};
|
|
||||||
|
|
||||||
// Deterministic, not routed through an LLM -- clone is 100% mechanical
|
|
||||||
// (same reasoning as commitPending/the squash-merge sequence below), and
|
|
||||||
// was the one place left that broke that pattern: a headless spawnPi
|
|
||||||
// call here meant a crash gave zero diagnostic output, just a silent
|
|
||||||
// exit code with nothing to debug from.
|
|
||||||
const clone = await runGit(cwd, ["clone", "--branch", baseBranch, repo, "."]);
|
|
||||||
if (clone.code !== 0) {
|
|
||||||
pipelineSession.gitError = clone.out;
|
|
||||||
return finish("clone-crashed");
|
|
||||||
}
|
|
||||||
if (!fs.existsSync(path.join(cwd, ".git"))) return finish("clone-missing");
|
|
||||||
|
|
||||||
let phases = tasks ? (Array.isArray(tasks[0]) ? tasks : [tasks]) : parseTaskBoard(cwd);
|
|
||||||
if (!phases || phases.length === 0) {
|
|
||||||
pipelineSession.gitError = "no tasks given and tasks/INDEX.md not found or empty";
|
|
||||||
return finish("no-tasks-found");
|
|
||||||
}
|
|
||||||
if (!tasks) {
|
|
||||||
phases = phases.map((phase) => phase.map((id) => ({ id, judgeOnly: true })));
|
|
||||||
}
|
|
||||||
pipelineSession.totalTasks = phases.flat().length;
|
|
||||||
|
|
||||||
for (let i = 0; i < phases.length; i++) {
|
|
||||||
const phaseTasks = phases[i];
|
|
||||||
const phaseLabel = phaseLabelFor(phaseTasks, i);
|
|
||||||
const phaseBranch = branchName ? `${branchName}/${phaseLabel}` : `agent-run/${repoId}/${phaseLabel}`;
|
|
||||||
|
|
||||||
// Resume a phase branch a prior (since-restarted) run already pushed,
|
|
||||||
// instead of always branching fresh off baseBranch -- otherwise every
|
|
||||||
// redeploy silently discards whatever tasks that prior run already
|
|
||||||
// committed and pushed (see runPhase's per-task push below).
|
|
||||||
const fetchExisting = await runGit(cwd, ["fetch", "origin", phaseBranch]);
|
|
||||||
const resuming = fetchExisting.code === 0;
|
|
||||||
const branchResult = resuming
|
|
||||||
? await runGit(cwd, ["checkout", "-b", phaseBranch, "FETCH_HEAD"])
|
|
||||||
: await runGit(cwd, ["checkout", "-b", phaseBranch]);
|
|
||||||
if (branchResult.code !== 0) {
|
|
||||||
pipelineSession.gitError = branchResult.out;
|
|
||||||
return finish("branch-crashed");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Idempotent and run every phase, NOT gated on a fresh (non-resumed)
|
|
||||||
// start -- every run this session was a resume, so the old i===0 &&
|
|
||||||
// !resuming gate meant this setup permanently never ran on poiman's
|
|
||||||
// branch, and portfolio's PLAN.md stayed tracked from before this rule
|
|
||||||
// ever existed (gitignore has no effect on an already-tracked file --
|
|
||||||
// observed live: it kept getting swept back in by every `git add -A`
|
|
||||||
// regardless of the ignore rule). Check-and-fix on every phase instead
|
|
||||||
// of once-at-genesis so a repo that's missing either self-heals on its
|
|
||||||
// very next run rather than carrying the gap forever.
|
|
||||||
const gitignorePath = path.join(cwd, ".gitignore");
|
|
||||||
const currentGitignore = fs.existsSync(gitignorePath) ? fs.readFileSync(gitignorePath, "utf8").split("\n") : [];
|
|
||||||
const requiredGitignoreLines = [
|
|
||||||
"*.tar.gz",
|
|
||||||
"*.tgz",
|
|
||||||
"*.crate",
|
|
||||||
"*.zip",
|
|
||||||
"*.bin",
|
|
||||||
"*.whl",
|
|
||||||
"vendor/",
|
|
||||||
"node_modules/",
|
|
||||||
".task-result-*",
|
|
||||||
".phase-result-*",
|
|
||||||
".stage-done-*",
|
|
||||||
"PLAN.md",
|
|
||||||
];
|
|
||||||
const missingGitignoreLines = requiredGitignoreLines.filter((line) => !currentGitignore.includes(line));
|
|
||||||
if (missingGitignoreLines.length > 0) {
|
|
||||||
fs.appendFileSync(
|
|
||||||
gitignorePath,
|
|
||||||
"\n# agent-harness: build artifacts, vendored archives, and harness bookkeeping never belong in source control\n" +
|
|
||||||
missingGitignoreLines.join("\n") +
|
|
||||||
"\n"
|
|
||||||
);
|
|
||||||
await runGit(cwd, ["add", ".gitignore"]);
|
|
||||||
await runGit(cwd, ["commit", "-m", "chore: broaden .gitignore for agent-run artifacts"]);
|
|
||||||
}
|
|
||||||
const trackedFiles = await runGit(cwd, ["ls-tree", "-r", "HEAD", "--name-only"]);
|
|
||||||
if (trackedFiles.out.split("\n").includes("PLAN.md")) {
|
|
||||||
await runGit(cwd, ["rm", "--cached", "PLAN.md"]);
|
|
||||||
await runGit(cwd, ["commit", "-m", "chore: untrack PLAN.md (already gitignored, was committed pre-rule)"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
await runPhase(cwd, baseBranch, phaseBranch, phaseTasks, pool, repoId, pipelineSession);
|
|
||||||
|
|
||||||
const phaseTaskIds = new Set(phaseTasks.map((t) => (typeof t === "string" ? t : t.id)));
|
|
||||||
const phaseResults = pipelineSession.taskResults.filter((r) => phaseTaskIds.has(r.task));
|
|
||||||
const phaseClean =
|
|
||||||
phaseResults.length === phaseTaskIds.size && phaseResults.every((r) => r.status === "done" || r.status === "done-with-concerns");
|
|
||||||
|
|
||||||
if (!phaseClean) {
|
|
||||||
pipelineSession.haltedAt = phaseLabel;
|
|
||||||
return finish("halted-phase-failed");
|
|
||||||
}
|
|
||||||
|
|
||||||
const phaseResultFile = path.join(cwd, `.phase-result-${phaseLabel}`);
|
|
||||||
fs.rmSync(phaseResultFile, { force: true });
|
|
||||||
const phaseJudge = await runOnPool(
|
|
||||||
pool,
|
|
||||||
cwd,
|
|
||||||
repoId,
|
|
||||||
"judge",
|
|
||||||
`${REQUIRE_REAL_TOOL_CALLS}Use the judge skill to review the full phase diff for phase ${phaseLabel} against ` +
|
|
||||||
`${baseBranch}...HEAD (covers every task in this phase: ${[...phaseTaskIds].join(", ")}). Every ` +
|
|
||||||
`individual task already passed its own judge review -- your job here is different: confirm the ` +
|
|
||||||
`tasks integrate correctly as one coherent narrative, and that real integration tests (not just ` +
|
|
||||||
`each task's isolated unit checks) exist and actually exercise the phase's intended use case end ` +
|
|
||||||
`to end. Write your verdict to ${phaseResultFile} as a single "VERDICT: PASS" or ` +
|
|
||||||
`"VERDICT: FAIL" line plus rationale, then run: touch ${path.join(cwd, `.stage-done-phase-${phaseLabel}-judge`)}`,
|
|
||||||
path.join(cwd, `.stage-done-phase-${phaseLabel}-judge`)
|
|
||||||
);
|
|
||||||
await commitPending(cwd, `phase: ${phaseLabel} integration review`);
|
|
||||||
|
|
||||||
if (!phaseJudge.ok) {
|
|
||||||
pipelineSession.haltedAt = phaseLabel;
|
|
||||||
pipelineSession.gitError = phaseJudge.error;
|
|
||||||
return finish("phase-judge-crashed");
|
|
||||||
}
|
|
||||||
|
|
||||||
const phaseJudgeText = fs.existsSync(phaseResultFile) ? fs.readFileSync(phaseResultFile, "utf8") : "";
|
|
||||||
if (parseVerdictLine(phaseJudgeText, "VERDICT") !== "PASS") {
|
|
||||||
pipelineSession.haltedAt = phaseLabel;
|
|
||||||
pipelineSession.phaseJudgeRationale = phaseJudgeText;
|
|
||||||
return finish("halted-phase-judge-failed");
|
|
||||||
}
|
|
||||||
|
|
||||||
const checkoutBase = await runGit(cwd, ["checkout", baseBranch]);
|
|
||||||
if (checkoutBase.code !== 0) {
|
|
||||||
pipelineSession.gitError = checkoutBase.out;
|
|
||||||
return finish("squash-crashed");
|
|
||||||
}
|
|
||||||
const squash = await runGit(cwd, ["merge", "--squash", phaseBranch]);
|
|
||||||
if (squash.code !== 0) {
|
|
||||||
await runGit(cwd, ["merge", "--abort"]);
|
|
||||||
pipelineSession.gitError = squash.out;
|
|
||||||
return finish("squash-crashed");
|
|
||||||
}
|
|
||||||
const commit = await runGit(cwd, ["commit", "-m", `feat: ${phaseLabel} (${[...phaseTaskIds].join(", ")})`]);
|
|
||||||
if (commit.code !== 0) {
|
|
||||||
pipelineSession.gitError = commit.out;
|
|
||||||
return finish("squash-crashed");
|
|
||||||
}
|
|
||||||
const push = await runGit(cwd, ["push", "origin", baseBranch]);
|
|
||||||
if (push.code !== 0) {
|
|
||||||
pipelineSession.gitError = push.out;
|
|
||||||
return finish("squash-push-crashed");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Milestone's content now lives in baseBranch as one squashed commit --
|
|
||||||
// the phase branch (and whatever a prior restart already pushed of it)
|
|
||||||
// has no further reason to exist. Delete it both places so a future run
|
|
||||||
// never tries to resume a phase that's already done, and so origin
|
|
||||||
// doesn't accumulate one dangling branch per completed phase forever.
|
|
||||||
await runGit(cwd, ["branch", "-D", phaseBranch]);
|
|
||||||
await runGit(cwd, ["push", "origin", "--delete", phaseBranch]);
|
|
||||||
|
|
||||||
logProgress(pipelineSession);
|
|
||||||
}
|
|
||||||
|
|
||||||
return finish("completed");
|
|
||||||
}
|
|
||||||
|
|
||||||
async function runConcurrent(items, limit, worker) {
|
|
||||||
const results = new Array(items.length);
|
|
||||||
let i = 0;
|
|
||||||
async function next() {
|
|
||||||
while (i < items.length) {
|
|
||||||
const idx = i++;
|
|
||||||
results[idx] = await worker(items[idx], idx);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, next));
|
|
||||||
return results;
|
|
||||||
}
|
|
||||||
|
|
||||||
// How many repos can be mid-flight at once. Each repo gets its own clone
|
|
||||||
// and its own 4-agent pool (planner/investigator/implementer/judge), so
|
|
||||||
// this is now the real concurrency knob -- tasks within one repo are
|
|
||||||
// already serialized against that repo's pool (see runPhase). The backend
|
|
||||||
// (homelab-ornith) actually runs 2 GPU replicas behind one Kubernetes
|
|
||||||
// Service, each with its own copy of the model loaded (see homelab's
|
|
||||||
// k8s/apps/llm-serving/ornith.yaml) -- so up to 2 concurrent LLM calls get
|
|
||||||
// real independent instances; a 3rd+ concurrent call queues inside
|
|
||||||
// whichever replica the Service's own load-balancing lands it on (each
|
|
||||||
// replica runs OLLAMA_NUM_PARALLEL=1). REPO_CONCURRENCY above 2 is still
|
|
||||||
// useful (more repos in flight overlaps git/file work, not just LLM calls)
|
|
||||||
// but past 2 simultaneous LLM calls, extra concurrency mostly means queueing
|
|
||||||
// rather than added throughput -- bump the backend's replica count to
|
|
||||||
// change that, not this constant.
|
|
||||||
const REPO_CONCURRENCY = Number(process.env.REPO_CONCURRENCY) || 3;
|
|
||||||
|
|
||||||
// repoId is the repo's own name, not a random id -- it's what every role
|
|
||||||
// session's --name is built from (see runOnPool: `${repoId}-${role}`), so
|
|
||||||
// agent-manager's own session list groups naturally by repo ("portfolio-
|
|
||||||
// planner", "portfolio-judge", "poiman-planner", ...) instead of by opaque
|
|
||||||
// UUID. Takes the last path segment of the URL, strips a trailing `.git`,
|
|
||||||
// and sanitizes anything that isn't safe in a tmux session name / directory
|
|
||||||
// name / git branch name. Two different repos that happen to share a
|
|
||||||
// basename (e.g. two orgs' "portfolio") would collide -- not handled, since
|
|
||||||
// nothing about this harness's usage has needed more than one org per run.
|
|
||||||
function slugFor(repoUrl) {
|
|
||||||
const last = repoUrl.replace(/\/+$/, "").split("/").pop() || repoUrl;
|
|
||||||
return last.replace(/\.git$/, "").replace(/[^a-zA-Z0-9._-]/g, "-");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Top-level entry point: runs every repo in `repos` to completion, up to
|
|
||||||
// REPO_CONCURRENCY at a time. Returns a map of repoId -> final
|
|
||||||
// pipelineSession, one per repo, independent of how the others fared.
|
|
||||||
async function runCoordinator({ repos, base, tasks, branchName }) {
|
|
||||||
const sessions = {};
|
|
||||||
await runConcurrent(repos, REPO_CONCURRENCY, async (repoUrl) => {
|
|
||||||
const repoId = slugFor(repoUrl);
|
|
||||||
const pipelineSession = {
|
|
||||||
id: repoId,
|
|
||||||
repo: repoUrl,
|
|
||||||
status: "running",
|
|
||||||
taskResults: [],
|
|
||||||
activeTasks: {},
|
|
||||||
totalTasks: 0,
|
|
||||||
startedAt: new Date().toISOString(),
|
|
||||||
};
|
|
||||||
sessions[repoId] = pipelineSession;
|
|
||||||
await runRepoPipeline({ repoId, repo: repoUrl, baseBranch: base, tasks, branchName }, pipelineSession);
|
|
||||||
});
|
|
||||||
return sessions;
|
|
||||||
}
|
|
||||||
|
|
||||||
function parseArgs(argv) {
|
|
||||||
const opts = { base: "main" };
|
|
||||||
for (let i = 0; i < argv.length; i++) {
|
|
||||||
const a = argv[i];
|
|
||||||
if (a === "--repo") opts.repo = argv[++i];
|
|
||||||
else if (a === "--repos") opts.repos = argv[++i];
|
|
||||||
else if (a === "--base") opts.base = argv[++i];
|
|
||||||
else if (a === "--tasks") opts.tasks = argv[++i];
|
|
||||||
else if (a === "--branch") opts.branch = argv[++i];
|
|
||||||
}
|
|
||||||
return opts;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function main() {
|
|
||||||
const opts = parseArgs(process.argv.slice(2));
|
|
||||||
const repos = opts.repos ? opts.repos.split(",") : opts.repo ? [opts.repo] : null;
|
|
||||||
if (!repos || repos.length === 0) {
|
|
||||||
console.error(
|
|
||||||
"usage: coordinator.js --repos <url1,url2,...> [--tasks T0.1,T0.2;T1.1,T1.2,...] [--base main] [--branch <name>]\n" +
|
|
||||||
" --repo <url> also accepted for a single repo\n" +
|
|
||||||
" --tasks applies to every repo listed; omitted: each repo discovers its own phases from tasks/INDEX.md\n" +
|
|
||||||
" REPO_CONCURRENCY env var (default 3): how many repos run at once"
|
|
||||||
);
|
|
||||||
// process.exitCode + natural exit, not process.exit() -- stdout piped
|
|
||||||
// through kubectl exec (not a TTY) can drop buffered console.log/
|
|
||||||
// console.error output if the process exits before it flushes. Setting
|
|
||||||
// exitCode and letting the event loop drain naturally is the
|
|
||||||
// documented-safe way to exit with a specific code without racing it.
|
|
||||||
process.exitCode = 1;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const phases = opts.tasks ? opts.tasks.split(";").map((phase) => phase.split(",")) : null;
|
|
||||||
const sessions = await runCoordinator({ repos, base: opts.base, tasks: phases, branchName: opts.branch });
|
|
||||||
process.exitCode = Object.values(sessions).every((s) => s.status === "completed") ? 0 : 1;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (require.main === module) {
|
|
||||||
main();
|
|
||||||
}
|
|
||||||
|
|
||||||
module.exports = { runCoordinator, runRepoPipeline, runOnPool, parseTaskBoard };
|
|
||||||
|
|
||||||
@@ -1,168 +0,0 @@
|
|||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: agent-pod
|
|
||||||
namespace: agent-pod
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: agent-pod
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: agent-pod
|
|
||||||
spec:
|
|
||||||
# api.riotpiao.com has no in-cluster DNS record (only resolves from the
|
|
||||||
# home network's own resolver) -- pin it to ingress-nginx-controller's
|
|
||||||
# ClusterIP so pi's models.json baseUrl works unchanged. TLS still
|
|
||||||
# terminates correctly since SNI/Host still say api.riotpiao.com.
|
|
||||||
hostAliases:
|
|
||||||
- ip: "10.101.128.185"
|
|
||||||
hostnames:
|
|
||||||
- "api.riotpiao.com"
|
|
||||||
containers:
|
|
||||||
# hub.js runs in the same container as pi (not a sidecar) so it can
|
|
||||||
# spawn `pi -p --mode json` directly via child_process -- a separate
|
|
||||||
# container can't exec into another container's filesystem/PATH.
|
|
||||||
# It IS the container's long-running process now; no more `sleep
|
|
||||||
# infinity` placeholder.
|
|
||||||
#
|
|
||||||
# Also builds the agent-manager fork (github.com/Riotpiaole/
|
|
||||||
# agent-manager, add-headless-spawn branch) from source and drops
|
|
||||||
# coordinator.js in beside hub.js -- neither is the container's
|
|
||||||
# foreground process. hub.js keeps that role unchanged; coordinator.js
|
|
||||||
# itself now owns multi-repo concurrency (REPO_CONCURRENCY env,
|
|
||||||
# default 3), so one invocation handles every repo:
|
|
||||||
# `kubectl exec <pod> -- node /root/coordinator.js --repos
|
|
||||||
# repoA,repoB,... --tasks ...`. Each repo gets its own clone and its
|
|
||||||
# own persistent 4-agent pool (planner/investigator/implementer/
|
|
||||||
# judge, one agent-manager session per role, reused across every
|
|
||||||
# task in that repo) on the container's local tmux server --
|
|
||||||
# `kubectl exec -it <pod> -- agent-manager` attaches its TUI live
|
|
||||||
# against those same sessions, no cross-machine visibility problem
|
|
||||||
# since spawner, tmux server, and viewer are all colocated here.
|
|
||||||
#
|
|
||||||
# No prebuilt Linux binary is shipped for agent-manager: the local
|
|
||||||
# .bin/ build is macOS arm64 (wrong OS/arch for this container
|
|
||||||
# anyway) and it's 27MB, well over a ConfigMap's ~1MiB cap. Debian's
|
|
||||||
# `apt-get golang-go` is far too old for this fork's go 1.26.5
|
|
||||||
# requirement, so the real Go toolchain is fetched directly from
|
|
||||||
# go.dev instead.
|
|
||||||
- name: pi
|
|
||||||
image: node:22-slim
|
|
||||||
command:
|
|
||||||
- sh
|
|
||||||
- -c
|
|
||||||
- |
|
|
||||||
set -e
|
|
||||||
apt-get update && apt-get install -y git curl jq openssh-client tmux python3 sqlite3 gcc build-essential
|
|
||||||
ssh-keygen -y -f /root/.ssh/id_forgejo > /root/.ssh/id_forgejo.pub
|
|
||||||
eval "$(ssh-agent -s)"
|
|
||||||
ssh-add /root/.ssh/id_forgejo
|
|
||||||
npm install -g @earendil-works/[email protected]
|
|
||||||
npm install --prefix /root ws
|
|
||||||
|
|
||||||
curl -fsSL "https://go.dev/dl/go1.26.5.linux-$(dpkg --print-architecture).tar.gz" | tar -C /usr/local -xz
|
|
||||||
export PATH="$PATH:/usr/local/go/bin"
|
|
||||||
git clone --branch add-headless-spawn --depth 1 \
|
|
||||||
https://github.com/Riotpiaole/agent-manager.git /root/agent-manager-src
|
|
||||||
(cd /root/agent-manager-src && go build -o /usr/local/bin/agent-manager .)
|
|
||||||
|
|
||||||
# Language toolchains for whatever repos the implementer/investigator/
|
|
||||||
# judge roles actually build and test -- go was already fetched above
|
|
||||||
# only for building agent-manager itself, and its PATH export above is
|
|
||||||
# local to this script, invisible to `kubectl exec` sessions into the
|
|
||||||
# already-running container. Symlinking both into /usr/local/bin (on
|
|
||||||
# PATH for every exec session, interactive or not) instead of relying
|
|
||||||
# on shell rc sourcing, which pi's non-interactive tool calls don't do.
|
|
||||||
ln -sf /usr/local/go/bin/go /usr/local/bin/go
|
|
||||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable
|
|
||||||
ln -sf /root/.cargo/bin/cargo /usr/local/bin/cargo
|
|
||||||
ln -sf /root/.cargo/bin/rustc /usr/local/bin/rustc
|
|
||||||
ln -sf /root/.cargo/bin/rustup /usr/local/bin/rustup
|
|
||||||
|
|
||||||
node /root/hub.js
|
|
||||||
env:
|
|
||||||
- name: PI_BIN
|
|
||||||
value: pi
|
|
||||||
- name: AGENT_MANAGER_BIN
|
|
||||||
value: /usr/local/bin/agent-manager
|
|
||||||
- name: HUB_WORK_DIR
|
|
||||||
value: /root/agent-harness-work
|
|
||||||
# planner/investigator/implementer stay on the default
|
|
||||||
# (homelab-ornith/ornith:35b, pi's settings.json default). Judge
|
|
||||||
# moves to the separate homelab-reasoning backend (DeepSeek-R1,
|
|
||||||
# its own 2 GPU replicas) so judge calls stop contending with the
|
|
||||||
# other 3 roles for the 2 ornith pods -- an entire role's worth
|
|
||||||
# of traffic moves onto otherwise-idle capacity instead.
|
|
||||||
- name: JUDGE_PROVIDER
|
|
||||||
value: homelab-reasoning
|
|
||||||
- name: JUDGE_MODEL
|
|
||||||
value: reasoning
|
|
||||||
ports:
|
|
||||||
- containerPort: 9090
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: "4"
|
|
||||||
memory: 8Gi
|
|
||||||
limits:
|
|
||||||
cpu: "8"
|
|
||||||
memory: 16Gi
|
|
||||||
volumeMounts:
|
|
||||||
- name: pi-config
|
|
||||||
mountPath: /root/.pi/agent/settings.json
|
|
||||||
subPath: settings.json
|
|
||||||
- name: pi-models
|
|
||||||
mountPath: /root/.pi/agent/models.json
|
|
||||||
subPath: models.json
|
|
||||||
- name: pi-skills
|
|
||||||
mountPath: /root/.pi/agent/skills
|
|
||||||
- name: hub-src
|
|
||||||
mountPath: /root/hub.js
|
|
||||||
subPath: hub.js
|
|
||||||
- name: coordinator-src
|
|
||||||
mountPath: /root/coordinator.js
|
|
||||||
subPath: coordinator.js
|
|
||||||
- name: ssh-key
|
|
||||||
mountPath: /root/.ssh/id_forgejo
|
|
||||||
subPath: id_forgejo
|
|
||||||
- name: ssh-config
|
|
||||||
mountPath: /root/.ssh/config
|
|
||||||
subPath: config
|
|
||||||
volumes:
|
|
||||||
- name: pi-config
|
|
||||||
configMap:
|
|
||||||
name: pi-config
|
|
||||||
- name: pi-models
|
|
||||||
secret:
|
|
||||||
secretName: pi-models
|
|
||||||
- name: pi-skills
|
|
||||||
configMap:
|
|
||||||
name: pi-skills
|
|
||||||
items:
|
|
||||||
- key: planner-SKILL.md
|
|
||||||
path: planner/SKILL.md
|
|
||||||
- key: investigator-SKILL.md
|
|
||||||
path: investigator/SKILL.md
|
|
||||||
- key: info-collector-SKILL.md
|
|
||||||
path: info-collector/SKILL.md
|
|
||||||
- key: implementer-SKILL.md
|
|
||||||
path: implementer/SKILL.md
|
|
||||||
- key: judge-SKILL.md
|
|
||||||
path: judge/SKILL.md
|
|
||||||
- key: resolver-SKILL.md
|
|
||||||
path: resolver/SKILL.md
|
|
||||||
- name: hub-src
|
|
||||||
configMap:
|
|
||||||
name: hub-src
|
|
||||||
- name: coordinator-src
|
|
||||||
configMap:
|
|
||||||
name: coordinator-src
|
|
||||||
- name: ssh-key
|
|
||||||
secret:
|
|
||||||
secretName: agent-pod-ssh-key
|
|
||||||
defaultMode: 0600
|
|
||||||
- name: ssh-config
|
|
||||||
configMap:
|
|
||||||
name: agent-pod-ssh-config
|
|
||||||
@@ -1,700 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
hub.js: |
|
|
||||||
#!/usr/bin/env node
|
|
||||||
// agent-hub: lives inside the pi container (not a sidecar) so it can spawn
|
|
||||||
// `pi` directly, and control the pod's own tmux server. One persistent
|
|
||||||
// in-cluster service -- POST /run to trigger a single ad-hoc headless agent
|
|
||||||
// run, POST /pipeline to run an ordered list of task phases against a
|
|
||||||
// repo/branch (phases run sequentially, up to PHASE_CONCURRENCY tasks within
|
|
||||||
// a phase run concurrently, each in its own git worktree). Within one task,
|
|
||||||
// planner/investigator/implementer/judge are separate agents in separate
|
|
||||||
// named tmux sessions (task-<id>-<role>, attachable via `kubectl exec -it --
|
|
||||||
// tmux attach -t <name>` while running), coordinating only through what's on
|
|
||||||
// disk in that task's worktree -- not one shared conversation. GET /console
|
|
||||||
// (WebSocket) watches every concurrent headless run live, relaying pi's own
|
|
||||||
// session protocol verbatim (same event shape Claude Code sessions use).
|
|
||||||
const http = require("node:http");
|
|
||||||
const crypto = require("node:crypto");
|
|
||||||
const fs = require("node:fs");
|
|
||||||
const path = require("node:path");
|
|
||||||
const { spawn } = require("node:child_process");
|
|
||||||
const readline = require("node:readline");
|
|
||||||
const { WebSocketServer } = require("ws");
|
|
||||||
|
|
||||||
const PORT = process.env.HUB_PORT || 9090;
|
|
||||||
const WORK_DIR = process.env.HUB_WORK_DIR || path.join(require("node:os").tmpdir(), "agent-harness-work");
|
|
||||||
|
|
||||||
// Never rely on a bare `pi` on $PATH -- both `pi` and `agent-console` collide
|
|
||||||
// with unrelated tools on this machine (a Rust CLI and a Datadog TUI,
|
|
||||||
// respectively, discovered the hard way this session). Always invoke the
|
|
||||||
// exact pinned @earendil-works/[email protected] installed locally under
|
|
||||||
// .pi-cli/, by explicit path.
|
|
||||||
const PI_BIN =
|
|
||||||
process.env.PI_BIN ||
|
|
||||||
path.join(
|
|
||||||
__dirname,
|
|
||||||
"..",
|
|
||||||
".pi-cli",
|
|
||||||
"node_modules",
|
|
||||||
"@earendil-works",
|
|
||||||
"pi-coding-agent",
|
|
||||||
"dist",
|
|
||||||
"cli.js"
|
|
||||||
);
|
|
||||||
|
|
||||||
// Job-type skills under pi/skills/<name>/SKILL.md (mounted at
|
|
||||||
// ~/.pi/agent/skills/<name>/ in agent-pod). When `agent` matches one of
|
|
||||||
// these, the prompt is forced through pi's `/skill:<name> <args>` mechanism
|
|
||||||
// instead of being sent bare -- see pi's skills.md docs on single-shot skill
|
|
||||||
// loading. `resolver` is never dispatched directly by a caller; only the
|
|
||||||
// pipeline driver invokes it, on stage crashes.
|
|
||||||
const ROLE_SKILLS = new Set([
|
|
||||||
"planner",
|
|
||||||
"investigator",
|
|
||||||
"info-collector",
|
|
||||||
"implementer",
|
|
||||||
"judge",
|
|
||||||
"resolver",
|
|
||||||
]);
|
|
||||||
|
|
||||||
const sessions = new Map(); // id -> {id, agent, status, events, startedAt, endedAt, pipelineId?, stage?}
|
|
||||||
const viewers = new Set(); // WebSocket connections watching /console
|
|
||||||
|
|
||||||
function broadcast(type, session) {
|
|
||||||
const msg = JSON.stringify({ type, session });
|
|
||||||
for (const ws of viewers) {
|
|
||||||
if (ws.readyState === ws.OPEN) ws.send(msg);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function startSession(agent, extra = {}) {
|
|
||||||
const id = extra.id || crypto.randomUUID();
|
|
||||||
const session = {
|
|
||||||
...extra,
|
|
||||||
id,
|
|
||||||
agent,
|
|
||||||
status: "running",
|
|
||||||
events: [],
|
|
||||||
startedAt: new Date().toISOString(),
|
|
||||||
};
|
|
||||||
sessions.set(id, session);
|
|
||||||
broadcast("start", session);
|
|
||||||
return session;
|
|
||||||
}
|
|
||||||
|
|
||||||
function addEvent(session, rawLine) {
|
|
||||||
const event = JSON.parse(rawLine);
|
|
||||||
session.events.push(event);
|
|
||||||
broadcast("event", session);
|
|
||||||
return event;
|
|
||||||
}
|
|
||||||
|
|
||||||
function endSession(session, status) {
|
|
||||||
session.status = status;
|
|
||||||
session.endedAt = new Date().toISOString();
|
|
||||||
broadcast("end", session);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Extracts the plain-text content of a message_end event, if any -- used to
|
|
||||||
// find the VERDICT:/RESOLUTION: line judge/resolver skills are required to
|
|
||||||
// end their final message with.
|
|
||||||
function textOf(event) {
|
|
||||||
if (event.type !== "message_end" || !event.message || !Array.isArray(event.message.content)) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
return event.message.content
|
|
||||||
.filter((c) => c.type === "text")
|
|
||||||
.map((c) => c.text)
|
|
||||||
.join("\n");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Core spawn primitive. Spawns `pi -p --mode json <extraArgs> <prompt>`,
|
|
||||||
// relays every line as a session event exactly like before. Returns
|
|
||||||
// { session, done } -- `session` is available synchronously (so an HTTP
|
|
||||||
// handler can respond with its id right away, same as the old runAgent),
|
|
||||||
// `done` is a Promise resolving once the process exits, for callers that
|
|
||||||
// need to wait on a stage (the pipeline driver) rather than fire-and-forget.
|
|
||||||
function spawnPi({ agent, prompt, provider, model, cwd, sessionExtra = {} }) {
|
|
||||||
const session = startSession(agent, sessionExtra);
|
|
||||||
const args = ["-p", "--mode", "json"];
|
|
||||||
if (provider) args.push("--provider", provider);
|
|
||||||
if (model) args.push("--model", model);
|
|
||||||
const finalPrompt = ROLE_SKILLS.has(agent) ? `/skill:${agent} ${prompt}` : prompt;
|
|
||||||
args.push(finalPrompt);
|
|
||||||
|
|
||||||
const child = spawn(PI_BIN, args, {
|
|
||||||
stdio: ["ignore", "pipe", "pipe"],
|
|
||||||
cwd,
|
|
||||||
});
|
|
||||||
const rl = readline.createInterface({ input: child.stdout });
|
|
||||||
let lastText = "";
|
|
||||||
let stderrTail = "";
|
|
||||||
|
|
||||||
rl.on("line", (line) => {
|
|
||||||
if (!line.trim()) return;
|
|
||||||
try {
|
|
||||||
const event = addEvent(session, line);
|
|
||||||
const text = textOf(event);
|
|
||||||
if (text) lastText = text;
|
|
||||||
} catch {
|
|
||||||
// non-JSON stdout noise, ignore
|
|
||||||
}
|
|
||||||
});
|
|
||||||
child.stderr.on("data", (chunk) => {
|
|
||||||
process.stderr.write(chunk);
|
|
||||||
stderrTail = (stderrTail + chunk.toString()).slice(-4000);
|
|
||||||
});
|
|
||||||
|
|
||||||
const done = new Promise((resolve) => {
|
|
||||||
child.on("close", (code) => {
|
|
||||||
endSession(session, code === 0 ? "done" : "error");
|
|
||||||
resolve({ code, session, lastText, stderrTail });
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
return { session, done };
|
|
||||||
}
|
|
||||||
|
|
||||||
function runAgent(agent, prompt, extraArgs = {}) {
|
|
||||||
// Fire-and-forget: caller (the /run handler) doesn't await `done`.
|
|
||||||
return spawnPi({ agent, prompt, ...extraArgs }).session;
|
|
||||||
}
|
|
||||||
|
|
||||||
function parseVerdictLine(text, label) {
|
|
||||||
if (!text) return null;
|
|
||||||
const re = new RegExp(`${label}:\\s*(\\w+)`, "i");
|
|
||||||
const m = text.match(re);
|
|
||||||
return m ? m[1].toUpperCase() : null;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deterministic git operations, run directly by hub.js rather than left to
|
|
||||||
// the model -- branch creation and pushing after each task are mechanical,
|
|
||||||
// not judgment calls, and need to happen reliably every time regardless of
|
|
||||||
// what a task's stages did or didn't remember to do.
|
|
||||||
function runCmd(bin, args, cwd) {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
const child = spawn(bin, args, { cwd, stdio: ["ignore", "pipe", "pipe"] });
|
|
||||||
let out = "";
|
|
||||||
child.stdout.on("data", (c) => (out += c));
|
|
||||||
child.stderr.on("data", (c) => (out += c));
|
|
||||||
child.on("close", (code) => resolve({ code, out: out.trim() }));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function runGit(cwd, args) {
|
|
||||||
return runCmd("git", args, cwd);
|
|
||||||
}
|
|
||||||
|
|
||||||
// A stage saying "commit" in its prompt is a request, not a guarantee -- seen
|
|
||||||
// in practice: a stage writes a real file and simply never runs `git add`/
|
|
||||||
// `git commit`, leaving it untracked and invisible to every later `git diff`.
|
|
||||||
// Sweep and commit anything left dirty after every stage, deterministically.
|
|
||||||
async function commitPending(cwd, message) {
|
|
||||||
await runGit(cwd, ["add", "-A"]);
|
|
||||||
const status = await runGit(cwd, ["status", "--porcelain"]);
|
|
||||||
if (!status.out) return { committed: false };
|
|
||||||
const commit = await runGit(cwd, ["commit", "-m", message]);
|
|
||||||
return { committed: commit.code === 0, error: commit.code !== 0 ? commit.out : undefined };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Invokes the `resolver` skill to diagnose a stuck/crashed stage and decide
|
|
||||||
// RETRY vs ABORT. Shared by both crash-recovery paths below (headless
|
|
||||||
// exit-code failures and interactive sentinel-file timeouts) -- the
|
|
||||||
// diagnostic prompt differs per caller, but "ask resolver, parse the
|
|
||||||
// RESOLUTION: line" is identical either way.
|
|
||||||
async function askResolver(pipelineId, cwd, task, diagnosticPrompt) {
|
|
||||||
const resolverResult = await spawnPi({
|
|
||||||
agent: "resolver",
|
|
||||||
prompt: diagnosticPrompt,
|
|
||||||
cwd,
|
|
||||||
sessionExtra: { pipelineId, stage: "resolver", task },
|
|
||||||
}).done;
|
|
||||||
return parseVerdictLine(resolverResult.lastText, "RESOLUTION");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Runs one pipeline stage, and if it crashes (nonzero exit -- not a semantic
|
|
||||||
// judge FAIL, which is handled separately), asks the resolver to diagnose
|
|
||||||
// and decide RETRY vs ABORT. Retries the failed stage at most once,
|
|
||||||
// regardless of what resolver recommends a second time -- a hard cap, not
|
|
||||||
// indefinite trust in the model's judgment.
|
|
||||||
async function runStageWithResolver(pipelineId, cwd, stage, prompt, task) {
|
|
||||||
let result = await spawnPi({
|
|
||||||
agent: stage,
|
|
||||||
prompt,
|
|
||||||
cwd,
|
|
||||||
sessionExtra: { pipelineId, stage, task },
|
|
||||||
}).done;
|
|
||||||
if (result.code === 0) return result;
|
|
||||||
|
|
||||||
const resolution = await askResolver(
|
|
||||||
pipelineId,
|
|
||||||
cwd,
|
|
||||||
task,
|
|
||||||
`Stage "${stage}" exited with code ${result.code}. Its stderr tail:\n${result.stderrTail}`
|
|
||||||
);
|
|
||||||
|
|
||||||
if (resolution === "RETRY") {
|
|
||||||
result = await spawnPi({
|
|
||||||
agent: stage,
|
|
||||||
prompt,
|
|
||||||
cwd,
|
|
||||||
sessionExtra: { pipelineId, stage, task },
|
|
||||||
}).done;
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deterministic tmux operations -- same rationale as runGit: mechanical,
|
|
||||||
// not a judgment call, run directly rather than trusted to a prompt.
|
|
||||||
function runTmux(args) {
|
|
||||||
return runCmd("tmux", args);
|
|
||||||
}
|
|
||||||
|
|
||||||
function tmuxSessionName(task) {
|
|
||||||
return `task-${task.replace(/[^a-zA-Z0-9]/g, "-")}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Bounded-concurrency pool -- runs `worker` over `items`, at most `limit` in
|
|
||||||
// flight at once. No external dep; a plain in-order index cursor shared by
|
|
||||||
// `limit` runner loops.
|
|
||||||
async function runConcurrent(items, limit, worker) {
|
|
||||||
const results = new Array(items.length);
|
|
||||||
let i = 0;
|
|
||||||
async function next() {
|
|
||||||
while (i < items.length) {
|
|
||||||
const idx = i++;
|
|
||||||
results[idx] = await worker(items[idx], idx);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, next));
|
|
||||||
return results;
|
|
||||||
}
|
|
||||||
|
|
||||||
const STAGE_TIMEOUT_MS = 10 * 60 * 1000;
|
|
||||||
const NUDGE_TIMEOUT_MS = 5 * 60 * 1000;
|
|
||||||
const POLL_MS = 10 * 1000;
|
|
||||||
|
|
||||||
async function waitForFile(filePath, limitMs) {
|
|
||||||
const start = Date.now();
|
|
||||||
while (!fs.existsSync(filePath)) {
|
|
||||||
if (Date.now() - start > limitMs) return false;
|
|
||||||
await new Promise((r) => setTimeout(r, POLL_MS));
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
const MAX_IMPLEMENT_ATTEMPTS = 5;
|
|
||||||
const MAX_PLAN_REVISIONS = 3;
|
|
||||||
|
|
||||||
// Runs one role as its own fresh interactive pi session in its own named
|
|
||||||
// tmux session -- planner, investigator, implementer, and judge are
|
|
||||||
// separate agents with separate context, not turns in one shared
|
|
||||||
// conversation. They coordinate only through what's on disk in the task's
|
|
||||||
// worktree: PLAN.md, committed code, judge's result file. Each session is
|
|
||||||
// attachable while it runs (kubectl exec -it -- tmux attach -t <name>) and
|
|
||||||
// killed once its sentinel file lands or it's abandoned after resolver
|
|
||||||
// escalation.
|
|
||||||
async function runStage(pipelineId, cwd, task, stageLabel, stagePrompt, sentinelFile) {
|
|
||||||
const sessionName = `${tmuxSessionName(task)}-${stageLabel}`;
|
|
||||||
fs.rmSync(sentinelFile, { force: true });
|
|
||||||
|
|
||||||
const spawned = await runTmux(["new-session", "-d", "-s", sessionName, "-c", cwd, PI_BIN, stagePrompt]);
|
|
||||||
if (spawned.code !== 0) return { ok: false, crashed: true, error: spawned.out, sessionName };
|
|
||||||
|
|
||||||
let ok = await waitForFile(sentinelFile, STAGE_TIMEOUT_MS);
|
|
||||||
if (!ok) {
|
|
||||||
const pane = await runTmux(["capture-pane", "-t", sessionName, "-p", "-S", "-200"]);
|
|
||||||
const resolution = await askResolver(
|
|
||||||
pipelineId,
|
|
||||||
cwd,
|
|
||||||
task,
|
|
||||||
`Task ${task}'s "${stageLabel}" stage hasn't finished after 10 minutes. ` +
|
|
||||||
`Its pane tail:\n${pane.out.slice(-3000)}\n\nDecide: is it still making ` +
|
|
||||||
`real progress and worth nudging to wrap up, or stuck and worth abandoning?`
|
|
||||||
);
|
|
||||||
if (resolution === "RETRY") {
|
|
||||||
await runTmux([
|
|
||||||
"send-keys",
|
|
||||||
"-t",
|
|
||||||
sessionName,
|
|
||||||
`Please wrap up the "${stageLabel}" stage now and touch ${path.basename(sentinelFile)} when done.`,
|
|
||||||
"Enter",
|
|
||||||
]);
|
|
||||||
ok = await waitForFile(sentinelFile, NUDGE_TIMEOUT_MS);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
await runTmux(["kill-session", "-t", sessionName]);
|
|
||||||
return { ok, sessionName };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Runs one task in its own git worktree (see runPhase): planner drafts
|
|
||||||
// PLAN.md, investigator confirms it, then implementer and judge go back and
|
|
||||||
// forth -- judge's FAIL rationale lands in .task-result-<task>, which the
|
|
||||||
// next implementer attempt is told to read and address. After
|
|
||||||
// MAX_IMPLEMENT_ATTEMPTS straight fails, planner is brought back in to
|
|
||||||
// judge whether the *plan* itself is wrong, not just the implementation; if
|
|
||||||
// so it revises PLAN.md and the implementer gets a fresh attempt budget
|
|
||||||
// against the new plan. MAX_PLAN_REVISIONS caps this from looping forever
|
|
||||||
// on a task that's genuinely stuck.
|
|
||||||
async function runTaskInteractive(pipelineId, cwd, baseBranch, task, pipelineSession, judgeOnly) {
|
|
||||||
const resultFile = path.join(cwd, `.task-result-${task}`);
|
|
||||||
fs.rmSync(resultFile, { force: true });
|
|
||||||
|
|
||||||
const specHint = `the file under tasks/ starting with "${task}-"`;
|
|
||||||
|
|
||||||
const runRole = async (stageLabel, prompt, sentinel) => {
|
|
||||||
pipelineSession.activeTasks[task] = {
|
|
||||||
stage: stageLabel,
|
|
||||||
sessionName: `${tmuxSessionName(task)}-${stageLabel}`,
|
|
||||||
startedAt: new Date().toISOString(),
|
|
||||||
};
|
|
||||||
broadcast("event", pipelineSession);
|
|
||||||
const result = await runStage(pipelineId, cwd, task, stageLabel, prompt, sentinel);
|
|
||||||
await commitPending(cwd, `task: ${task} (${stageLabel})`);
|
|
||||||
return result;
|
|
||||||
};
|
|
||||||
|
|
||||||
const abandon = (stageLabel, result, attempt) => {
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
broadcast("event", pipelineSession);
|
|
||||||
return {
|
|
||||||
task,
|
|
||||||
status: result.crashed ? "spawn-crashed" : "timed-out",
|
|
||||||
error: result.error,
|
|
||||||
stoppedAt: stageLabel,
|
|
||||||
...(attempt !== undefined ? { attempt } : {}),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
// Task's implementation is inherited already-committed (e.g. from a base
|
|
||||||
// branch of prior work) -- try one judge pass against the spec directly
|
|
||||||
// (no PLAN.md exists yet) before paying for a full planner/investigator
|
|
||||||
// redo. PASS ends the task here; FAIL falls through into the normal flow
|
|
||||||
// below, so planner/implementer pick up with the judge's real feedback.
|
|
||||||
if (judgeOnly) {
|
|
||||||
const quick = await runRole(
|
|
||||||
"judge",
|
|
||||||
`Task ${task} may already be implemented on this branch -- check ` +
|
|
||||||
`\`git log --oneline --grep '${task}'\` and the current code directly against its spec ` +
|
|
||||||
`(${specHint})'s acceptance criteria (no PLAN.md exists for this task yet). Write your ` +
|
|
||||||
`verdict to .task-result-${task} as a single "VERDICT: PASS" or "VERDICT: FAIL" line plus ` +
|
|
||||||
`one line of rationale, then run: touch .stage-done-${task}-judge-0`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-judge-0`)
|
|
||||||
);
|
|
||||||
if (!quick.ok) return abandon("judge", quick, 0);
|
|
||||||
|
|
||||||
const quickText = fs.existsSync(resultFile) ? fs.readFileSync(resultFile, "utf8") : "";
|
|
||||||
if (parseVerdictLine(quickText, "VERDICT") === "PASS") {
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
broadcast("event", pipelineSession);
|
|
||||||
return { task, status: "done", judgeRationale: quickText, judgeOnlyPass: true };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let result = await runRole(
|
|
||||||
"planner",
|
|
||||||
`Use the planner skill to draft PLAN.md for task ${task}, reading its spec (${specHint}). Commit PLAN.md, then run: touch .stage-done-${task}-planner`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-planner`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("planner", result);
|
|
||||||
|
|
||||||
result = await runRole(
|
|
||||||
"investigator",
|
|
||||||
`Use the investigator skill to confirm PLAN.md against real sources, append findings, commit. Then run: touch .stage-done-${task}-investigator`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-investigator`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("investigator", result);
|
|
||||||
|
|
||||||
let planRevisions = 0;
|
|
||||||
let implementAttempt = 0;
|
|
||||||
let verdict = null;
|
|
||||||
let resultText = "";
|
|
||||||
let justRevisedPlan = false;
|
|
||||||
|
|
||||||
while (true) {
|
|
||||||
implementAttempt++;
|
|
||||||
const feedbackHint = fs.existsSync(resultFile)
|
|
||||||
? justRevisedPlan
|
|
||||||
? `.task-result-${task} holds the judge's feedback against the OLD plan, which prompted a plan revision -- ` +
|
|
||||||
`PLAN.md has since changed. Read the current PLAN.md as the source of truth, not the old feedback verbatim.`
|
|
||||||
: `A previous judge review exists at .task-result-${task} -- read it and address every issue it raises.`
|
|
||||||
: "";
|
|
||||||
justRevisedPlan = false;
|
|
||||||
|
|
||||||
result = await runRole(
|
|
||||||
"implementer",
|
|
||||||
`Use the implementer skill to implement what the current PLAN.md specifies (commit as you go). ${feedbackHint} Then run: touch .stage-done-${task}-implementer-${implementAttempt}`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-implementer-${implementAttempt}`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("implementer", result, implementAttempt);
|
|
||||||
|
|
||||||
result = await runRole(
|
|
||||||
"judge",
|
|
||||||
`Use the judge skill to review the diff against ${baseBranch}...HEAD. Write your verdict to ` +
|
|
||||||
`.task-result-${task} as a single "VERDICT: PASS" or "VERDICT: FAIL" line plus one line of ` +
|
|
||||||
`rationale, then run: touch .stage-done-${task}-judge-${implementAttempt}`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-judge-${implementAttempt}`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("judge", result, implementAttempt);
|
|
||||||
|
|
||||||
resultText = fs.existsSync(resultFile) ? fs.readFileSync(resultFile, "utf8") : "";
|
|
||||||
verdict = parseVerdictLine(resultText, "VERDICT");
|
|
||||||
if (verdict === "PASS") break;
|
|
||||||
|
|
||||||
if (implementAttempt >= MAX_IMPLEMENT_ATTEMPTS) {
|
|
||||||
if (planRevisions >= MAX_PLAN_REVISIONS) break;
|
|
||||||
planRevisions++;
|
|
||||||
result = await runRole(
|
|
||||||
"planner-revise",
|
|
||||||
`Implementer failed judge review ${MAX_IMPLEMENT_ATTEMPTS} times in a row for task ${task}. Read PLAN.md, ` +
|
|
||||||
`the judge's feedback in .task-result-${task}, and the current diff against ${baseBranch}...HEAD. Decide ` +
|
|
||||||
`whether the plan's approach itself is wrong, not just the implementation -- if so, revise PLAN.md and ` +
|
|
||||||
`commit. If you change the approach, also use the investigator skill to confirm the new approach against ` +
|
|
||||||
`real sources before committing. If the plan is sound, note why in PLAN.md and leave it as-is. Then run: ` +
|
|
||||||
`touch .stage-done-${task}-planner-revise-${planRevisions}`,
|
|
||||||
path.join(cwd, `.stage-done-${task}-planner-revise-${planRevisions}`)
|
|
||||||
);
|
|
||||||
if (!result.ok) return abandon("planner-revise", result, planRevisions);
|
|
||||||
implementAttempt = 0;
|
|
||||||
justRevisedPlan = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
delete pipelineSession.activeTasks[task];
|
|
||||||
broadcast("event", pipelineSession);
|
|
||||||
|
|
||||||
if (verdict !== "PASS" && planRevisions >= MAX_PLAN_REVISIONS) {
|
|
||||||
return { task, status: "unresolved", judgeRationale: resultText, implementAttempts: implementAttempt, planRevisions };
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
task,
|
|
||||||
status: verdict === "PASS" ? "done" : "done-with-concerns",
|
|
||||||
judgeRationale: resultText,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
const PHASE_CONCURRENCY = 3;
|
|
||||||
|
|
||||||
// Runs one phase (a batch of tasks with no declared dependency on each
|
|
||||||
// other) with up to PHASE_CONCURRENCY tasks in flight at once. Each task
|
|
||||||
// gets its own git worktree off workBranch -- concurrent pi sessions writing
|
|
||||||
// into one shared working tree would corrupt the index; worktrees share the
|
|
||||||
// same object database but give each task an isolated checkout. After a
|
|
||||||
// task's session ends, its branch is merged back into workBranch and pushed,
|
|
||||||
// one merge at a time (git ref updates aren't safe to run concurrently even
|
|
||||||
// though the worktrees themselves are isolated).
|
|
||||||
async function runPhase(pipelineId, cwd, workBranch, phaseTasks, pipelineSession) {
|
|
||||||
// Each entry is either a plain task id, or { id, judgeOnly: true } when
|
|
||||||
// the task's implementation already exists (e.g. inherited from a base
|
|
||||||
// branch) and just needs a real judge pass rather than a full
|
|
||||||
// planner/investigator/implementer redo.
|
|
||||||
const entries = phaseTasks.map((t) => (typeof t === "string" ? { id: t, judgeOnly: false } : t));
|
|
||||||
|
|
||||||
const worktrees = {};
|
|
||||||
for (const entry of entries) {
|
|
||||||
const task = entry.id;
|
|
||||||
const wtDir = path.join(WORK_DIR, pipelineId, `wt-${task.replace(/[^a-zA-Z0-9]/g, "-")}`);
|
|
||||||
const taskBranch = `task/${task}`;
|
|
||||||
const add = await runGit(cwd, ["worktree", "add", "-b", taskBranch, wtDir, workBranch]);
|
|
||||||
if (add.code !== 0) {
|
|
||||||
pipelineSession.taskResults.push({ task, status: "worktree-crashed", error: add.out });
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
worktrees[task] = { wtDir, taskBranch };
|
|
||||||
}
|
|
||||||
|
|
||||||
const runnable = entries.filter((e) => worktrees[e.id]);
|
|
||||||
await runConcurrent(runnable, PHASE_CONCURRENCY, async (entry) => {
|
|
||||||
const { wtDir } = worktrees[entry.id];
|
|
||||||
const result = await runTaskInteractive(pipelineId, wtDir, workBranch, entry.id, pipelineSession, entry.judgeOnly);
|
|
||||||
pipelineSession.taskResults.push(result);
|
|
||||||
return result;
|
|
||||||
});
|
|
||||||
|
|
||||||
// Merge + push sequentially -- ref updates on the shared repo, one at a
|
|
||||||
// time, in the declared task order for this phase.
|
|
||||||
for (const entry of runnable) {
|
|
||||||
const task = entry.id;
|
|
||||||
const { wtDir, taskBranch } = worktrees[task];
|
|
||||||
const result = pipelineSession.taskResults.find((r) => r.task === task);
|
|
||||||
|
|
||||||
const merge = await runGit(cwd, ["merge", "--no-ff", taskBranch, "-m", `merge: ${task}`]);
|
|
||||||
if (merge.code !== 0) {
|
|
||||||
await runGit(cwd, ["merge", "--abort"]);
|
|
||||||
if (result) {
|
|
||||||
result.status = "merge-conflict";
|
|
||||||
result.mergeError = merge.out;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
const push = await runGit(cwd, ["push", "-u", "origin", workBranch]);
|
|
||||||
if (result) {
|
|
||||||
result.pushed = push.code === 0;
|
|
||||||
if (!result.pushed) result.pushError = push.out;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
await runGit(cwd, ["worktree", "remove", wtDir, "--force"]);
|
|
||||||
await runGit(cwd, ["branch", "-D", taskBranch]);
|
|
||||||
broadcast("event", pipelineSession);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// tasks: array of phases, each phase an array of task ids with no declared
|
|
||||||
// dependency on each other (e.g. [["T0.1","T0.2"], ["T1.1","T1.2","T1.3"]]) --
|
|
||||||
// caller's responsibility to supply real phase grouping (see tasks/INDEX.md;
|
|
||||||
// filename/numeric sort does NOT match execution order on boards like this).
|
|
||||||
// A flat array of ids is also accepted and treated as one single phase.
|
|
||||||
// Phases run strictly sequentially (a phase boundary is a real dependency
|
|
||||||
// gate); tasks within a phase run concurrently, each in its own worktree --
|
|
||||||
// see runPhase.
|
|
||||||
function runPipeline({ pipelineId, repo, baseBranch, tasks, branchName }) {
|
|
||||||
const cwd = path.join(WORK_DIR, pipelineId);
|
|
||||||
fs.mkdirSync(cwd, { recursive: true });
|
|
||||||
|
|
||||||
const phases = Array.isArray(tasks[0]) ? tasks : [tasks];
|
|
||||||
|
|
||||||
const pipelineSession = startSession("pipeline", {
|
|
||||||
id: pipelineId,
|
|
||||||
pipelineId,
|
|
||||||
stage: "pipeline",
|
|
||||||
taskResults: [],
|
|
||||||
activeTasks: {},
|
|
||||||
totalTasks: phases.flat().length,
|
|
||||||
});
|
|
||||||
|
|
||||||
(async () => {
|
|
||||||
const clone = await runStageWithResolver(
|
|
||||||
pipelineId,
|
|
||||||
cwd,
|
|
||||||
"planner",
|
|
||||||
`Run exactly this command, verbatim, no variation: git clone --branch ${baseBranch} ${repo} . -- the trailing dot is required, it clones directly into the current directory instead of creating a subdirectory. Do not cd anywhere first or after. Do nothing else.`,
|
|
||||||
"clone"
|
|
||||||
);
|
|
||||||
if (clone.code !== 0) return endSession(pipelineSession, "clone-crashed");
|
|
||||||
if (!fs.existsSync(path.join(cwd, ".git"))) {
|
|
||||||
// The model deciding to `cd` elsewhere before cloning (instead of
|
|
||||||
// cloning into the assigned cwd) is a real failure mode seen in
|
|
||||||
// practice, not a hypothetical -- exit code 0 doesn't mean the clone
|
|
||||||
// landed where every later stage's cwd assumes it did.
|
|
||||||
return endSession(pipelineSession, "clone-missing");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Dedicated branch, never main -- and pushed after every single task
|
|
||||||
// (not just at the end) so a pod restart mid-run loses at most the
|
|
||||||
// in-progress task's work, not everything since the start.
|
|
||||||
const workBranch = branchName || `agent-run/${pipelineId}`;
|
|
||||||
const branchResult = await runGit(cwd, ["checkout", "-b", workBranch]);
|
|
||||||
if (branchResult.code !== 0) {
|
|
||||||
pipelineSession.gitError = branchResult.out;
|
|
||||||
return endSession(pipelineSession, "branch-crashed");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Seen in practice: a task manually downloads a dependency tarball
|
|
||||||
// (crates.io registry access isn't guaranteed from every sandboxed
|
|
||||||
// checkout) and it lands at repo root, outside whatever .gitignore
|
|
||||||
// already covers -- then git add -A (ours or the model's own) commits
|
|
||||||
// it. Append broad build-artifact/archive patterns before any task
|
|
||||||
// runs, so it's excluded regardless of who stages files later.
|
|
||||||
const gitignoreAdditions = [
|
|
||||||
"",
|
|
||||||
"# agent-harness: build artifacts and vendored archives never belong in source control",
|
|
||||||
"*.tar.gz",
|
|
||||||
"*.tgz",
|
|
||||||
"*.crate",
|
|
||||||
"*.zip",
|
|
||||||
"*.bin",
|
|
||||||
"*.whl",
|
|
||||||
"vendor/",
|
|
||||||
"node_modules/",
|
|
||||||
"",
|
|
||||||
"# agent-harness: task completion sentinel files, harness bookkeeping only",
|
|
||||||
".task-result-*",
|
|
||||||
".stage-done-*",
|
|
||||||
].join("\n");
|
|
||||||
fs.appendFileSync(path.join(cwd, ".gitignore"), gitignoreAdditions + "\n");
|
|
||||||
await runGit(cwd, ["add", ".gitignore"]);
|
|
||||||
await runGit(cwd, ["commit", "-m", "chore: broaden .gitignore for agent-run artifacts"]);
|
|
||||||
|
|
||||||
for (const phaseTasks of phases) {
|
|
||||||
await runPhase(pipelineId, cwd, workBranch, phaseTasks, pipelineSession);
|
|
||||||
}
|
|
||||||
|
|
||||||
const crashed = pipelineSession.taskResults.filter((r) => r.status.endsWith("-crashed"));
|
|
||||||
endSession(pipelineSession, crashed.length > 0 ? "completed-with-crashes" : "completed");
|
|
||||||
})();
|
|
||||||
|
|
||||||
return pipelineSession;
|
|
||||||
}
|
|
||||||
|
|
||||||
const server = http.createServer((req, res) => {
|
|
||||||
const url = new URL(req.url, "http://localhost");
|
|
||||||
|
|
||||||
if (url.pathname === "/healthz") {
|
|
||||||
res.writeHead(200).end();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (url.pathname === "/sessions" && req.method === "GET") {
|
|
||||||
res.writeHead(200, { "Content-Type": "application/json" });
|
|
||||||
res.end(JSON.stringify([...sessions.values()]));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (url.pathname === "/run" && req.method === "POST") {
|
|
||||||
let body = "";
|
|
||||||
req.on("data", (chunk) => (body += chunk));
|
|
||||||
req.on("end", () => {
|
|
||||||
try {
|
|
||||||
const { agent, prompt, provider, model } = JSON.parse(body);
|
|
||||||
if (!agent || !prompt) throw new Error("agent and prompt are required");
|
|
||||||
const session = runAgent(agent, prompt, { provider, model });
|
|
||||||
res.writeHead(200, { "Content-Type": "application/json" });
|
|
||||||
res.end(JSON.stringify({ id: session.id }));
|
|
||||||
} catch (err) {
|
|
||||||
res.writeHead(400, { "Content-Type": "application/json" });
|
|
||||||
res.end(JSON.stringify({ error: err.message }));
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (url.pathname === "/pipeline" && req.method === "POST") {
|
|
||||||
let body = "";
|
|
||||||
req.on("data", (chunk) => (body += chunk));
|
|
||||||
req.on("end", () => {
|
|
||||||
try {
|
|
||||||
const { repo, baseBranch, tasks, branchName } = JSON.parse(body);
|
|
||||||
if (!repo || !baseBranch || !Array.isArray(tasks) || tasks.length === 0) {
|
|
||||||
throw new Error("repo, baseBranch, and a non-empty tasks array are required");
|
|
||||||
}
|
|
||||||
const pipelineId = crypto.randomUUID();
|
|
||||||
runPipeline({ pipelineId, repo, baseBranch, tasks, branchName });
|
|
||||||
res.writeHead(200, { "Content-Type": "application/json" });
|
|
||||||
res.end(JSON.stringify({ id: pipelineId }));
|
|
||||||
} catch (err) {
|
|
||||||
res.writeHead(400, { "Content-Type": "application/json" });
|
|
||||||
res.end(JSON.stringify({ error: err.message }));
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
res.writeHead(404).end();
|
|
||||||
});
|
|
||||||
|
|
||||||
const wss = new WebSocketServer({ server, path: "/console" });
|
|
||||||
wss.on("connection", (ws) => {
|
|
||||||
for (const session of sessions.values()) {
|
|
||||||
ws.send(JSON.stringify({ type: "snapshot", session }));
|
|
||||||
}
|
|
||||||
viewers.add(ws);
|
|
||||||
ws.on("close", () => viewers.delete(ws));
|
|
||||||
});
|
|
||||||
|
|
||||||
server.listen(PORT, () => console.log(`agent-hub listening on :${PORT}`));
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: hub-src
|
|
||||||
namespace: agent-pod
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: agent-hub
|
|
||||||
namespace: agent-pod
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: agent-pod
|
|
||||||
ports:
|
|
||||||
- port: 9090
|
|
||||||
targetPort: 9090
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
namespace: agent-pod
|
|
||||||
resources:
|
|
||||||
- deployment.yaml
|
|
||||||
- configmap.yaml
|
|
||||||
- hub-configmap.yaml
|
|
||||||
- coordinator-configmap.yaml
|
|
||||||
- pi-skills-configmap.yaml
|
|
||||||
- ssh-configmap.yaml
|
|
||||||
- hub-service.yaml
|
|
||||||
- console-ingress.yaml
|
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
implementer-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: implementer
|
|
||||||
description: Turns a confirmed PLAN.md into real code changes in the current checkout, committing incrementally. Use as the implementation stage of a spec-to-push pipeline, after planner and investigator have run.
|
|
||||||
allowed-tools: Read Grep Find Ls Write Edit Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
Execute the already-agreed plan; don't re-litigate it. `PLAN.md` (plus any `## Investigation` flags) is the source of truth for *what*; use judgment only for *how*, within the codebase's existing conventions.
|
|
||||||
|
|
||||||
- Read `PLAN.md` top to bottom. Treat flagged/unconfirmed steps conservatively (safer, more literal reading; note it in the commit). Work steps in order. Commit after each meaningful step (`git add -A && git commit -m "..."`), not one giant commit — the judge stage needs real diff history.
|
|
||||||
- Push only if the task explicitly asks for it.
|
|
||||||
|
|
||||||
Match existing style. Don't refactor or "improve" code the plan didn't ask you to touch.
|
|
||||||
|
|
||||||
**UI/frontend changes:** don't trust that the code compiles as proof it works. Start the app (or its dev server) and use `npx playwright` via Bash to actually load the page and look — screenshot the affected view before and after your change, and click through the golden path the plan describes. `npx playwright screenshot <url> out.png` for a quick visual check; for interaction (clicks, form fills, navigation), write a small throwaway script under a scratch path (e.g. `/tmp/`, never committed) using `playwright` the library, run it with `node`, then delete it. This doesn't apply to non-UI work (a Rust library, a CLI, a backend-only change) — use judgment.
|
|
||||||
|
|
||||||
**Hard rules:**
|
|
||||||
- Follow DRY and SOLID. Don't duplicate logic that already exists elsewhere in the codebase you're touching — reuse or extract instead. Keep each unit responsible for one thing.
|
|
||||||
- Never commit anything that doesn't belong in source control: build artifacts, downloaded/vendored dependencies, secrets, scratch/debug files. `.gitignore` already blocks common patterns; if you create something outside those patterns, delete it before committing rather than relying on `.gitignore` to catch it.
|
|
||||||
|
|
||||||
**Never vendor a dependency by downloading/extracting it into the repo.** Use the language's real package manager (`cargo add`, `npm install`, etc.) so the dependency is declared in the manifest and lockfile, not a tarball or extracted source tree sitting in the checkout. If the package manager can't reach its registry from here, say so in your commit message rather than working around it — a later commit sweep (`git add -A`) commits whatever's in the checkout, including anything downloaded for a workaround, even if you never intended to keep it.
|
|
||||||
|
|
||||||
info-collector-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: info-collector
|
|
||||||
description: Gathers and summarizes information on a topic from the web without judging or confirming any particular approach. Use standalone when you need raw research/context on a subject, not a verdict on a specific plan (that's the investigator skill).
|
|
||||||
allowed-tools: Read Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
**Persona:** You are a research assistant. Your job is to gather relevant information on a topic and summarize it neutrally — you are not asked to approve, reject, or recommend anything, just to collect and organize what's out there.
|
|
||||||
|
|
||||||
**Thinking mode:** Medium — breadth of coverage matters more than deep verification here (that's the investigator skill's job).
|
|
||||||
|
|
||||||
**Modes:**
|
|
||||||
|
|
||||||
- **Collect mode** (default) — search the web (`curl` against `https://api.search.brave.com/res/v1/web/search`, header `X-Subscription-Token: $BRAVE_API_KEY`, `--data-urlencode "q=<query>"`) with varied queries to cover the topic from multiple angles, then produce a structured summary: topic areas found, key facts, and links to sources for each. Do not editorialize about which approach is "right" — that's out of scope for this skill.
|
|
||||||
- If asked to write the summary to a file, write it and report the path; otherwise return it directly in your response.
|
|
||||||
investigator-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: investigator
|
|
||||||
description: Reads an existing PLAN.md and confirms its approach against real, current sources via web search, appending findings and flags. Use to sanity-check a plan before implementation, or standalone to verify a claimed approach is actually correct.
|
|
||||||
allowed-tools: Read Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
Check whether the plan's claims about the real world are actually true right now. Cite sources; don't assert without one.
|
|
||||||
|
|
||||||
- Read `PLAN.md` and the spec docs on disk. For each claim depending on external facts (a library's current API, a service's behavior), search the web (`curl` against `https://api.search.brave.com/res/v1/web/search`, header `X-Subscription-Token: $BRAVE_API_KEY`) to confirm or refute it. Append a `## Investigation` section to `PLAN.md`: each claim, its source(s), PASS/FLAG. Commit: `git add PLAN.md && git commit -m "investigate: confirm plan against sources"`.
|
|
||||||
- No `PLAN.md`? Just answer the question asked, citing sources.
|
|
||||||
|
|
||||||
Flag unconfirmed/contradicted claims rather than silently fixing them — that decision belongs to whoever reads the flag next.
|
|
||||||
|
|
||||||
**Never download anything into the repo checkout.** Need to inspect a dependency's real source/docs? Fetch into `/tmp/`, not the repo tree — a later `git add -A` sweep commits whatever's sitting in the checkout, staged or not.
|
|
||||||
judge-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: judge
|
|
||||||
description: LLM-as-judge. Reviews a git diff against PLAN.md and the original spec, and returns a PASS/FAIL verdict with rationale. Use as the final review/report stage of a spec-to-push pipeline (the implementer stage already pushed; this reports on what shipped), or standalone to review any diff against stated criteria.
|
|
||||||
allowed-tools: Read Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
Independent reviewer. Judge whether the implementation satisfies the plan and spec, on the evidence in front of you — not on how confident the commit messages sound. Don't rubber-stamp.
|
|
||||||
|
|
||||||
- Run `git diff <base-branch>...HEAD` to see exactly what changed. Compare against `PLAN.md`'s steps and the spec docs. Does every step have a corresponding change? Does the diff contradict any investigator flag? Anything obviously broken on inspection?
|
|
||||||
- **UI/frontend changes:** a diff that reads correctly can still render broken. Start the app and use `npx playwright` via Bash to actually look — screenshot the affected view, click through the golden path the plan/spec describes. FAIL on a visual defect the diff alone wouldn't show (broken layout, a control that doesn't do what its code claims, a state the plan promised that never renders). Doesn't apply to non-UI work — use judgment.
|
|
||||||
- FAIL on DRY/SOLID violations (duplicated logic that should reuse existing code, mixed-responsibility units) and on anything committed that doesn't belong in source control (build artifacts, vendored dependencies, secrets, scratch files) — name the specific file/lines in your rationale.
|
|
||||||
- No `PLAN.md`/base given? Review whatever diff/criteria are in the task directly.
|
|
||||||
|
|
||||||
You MUST end your final message with a literal verdict line, exactly one of:
|
|
||||||
|
|
||||||
```
|
|
||||||
VERDICT: PASS
|
|
||||||
```
|
|
||||||
```
|
|
||||||
VERDICT: FAIL
|
|
||||||
```
|
|
||||||
|
|
||||||
followed by your rationale. The pipeline driver parses this exact line mechanically to record the outcome — omitting it or rephrasing it breaks the pipeline.
|
|
||||||
|
|
||||||
planner-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: planner
|
|
||||||
description: Clones a target repo/branch, reads its markdown specs, and writes a verifiable step-by-step implementation plan (PLAN.md). Use as the first stage of a spec-to-PR pipeline, or standalone when asked to plan out a task before implementing it.
|
|
||||||
allowed-tools: Read Grep Find Ls Write Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
Plan a concrete, verifiable implementation. Don't implement — that's `implementer`'s job, after `investigator` confirms.
|
|
||||||
|
|
||||||
- If cwd is empty: `git clone` the given repo/branch first (only at pipeline start).
|
|
||||||
- Read every markdown spec file for the task. Decompose into a numbered list of concrete steps, each naming the files/areas it touches and how to verify it's done. State assumptions explicitly; if ambiguous, pick the literal reading and note the ambiguity.
|
|
||||||
- Write the plan to exactly `./PLAN.md` in the repo root — not `tasks/PLAN.md`, not `plans/<name>.md`, not any other name or location. Every later stage looks for the plan at that exact path. Commit: `git add PLAN.md && git commit -m "plan: <summary>"`.
|
|
||||||
|
|
||||||
Don't touch any other file.
|
|
||||||
resolver-SKILL.md: |
|
|
||||||
---
|
|
||||||
name: resolver
|
|
||||||
description: Diagnoses why a pipeline stage crashed (nonzero exit, not a semantic pass/fail) and decides whether it's safe to retry. Invoked by the pipeline driver when a planner/investigator/implementer/judge stage process fails to run to completion.
|
|
||||||
allowed-tools: Read Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
**Persona:** You are an incident triager, not a fixer. A pipeline stage stopped running — your job is to look at what's on disk and what the failed stage's own output said, figure out why, and decide whether re-running that stage is likely to succeed or would just fail the same way again.
|
|
||||||
|
|
||||||
**Thinking mode:** Medium — this is triage (root cause + retry/no-retry judgment), not deep design work.
|
|
||||||
|
|
||||||
**Modes:**
|
|
||||||
|
|
||||||
- **Triage mode** (default) — read the failed stage's name and its last stdout/stderr tail (given in the task). Check the working directory's current state (`git status`, `git log -1`) to see what, if anything, that stage managed to do before stopping. Distinguish transient causes (network blip, a flaky command, an interrupted git operation left in a bad-but-fixable state) from structural ones (the plan itself is broken, a required tool/credential is missing, the repo is in a state no retry will fix).
|
|
||||||
|
|
||||||
You MUST end your final message with a literal resolution line, exactly one of:
|
|
||||||
|
|
||||||
```
|
|
||||||
RESOLUTION: RETRY
|
|
||||||
```
|
|
||||||
```
|
|
||||||
RESOLUTION: ABORT
|
|
||||||
```
|
|
||||||
|
|
||||||
followed by your rationale. The pipeline driver parses this exact line mechanically and retries the failed stage **at most once** regardless of what you recommend a second time — don't assume unlimited retries. If the working directory is left in a broken state that a retry needs cleaned up first (e.g. a half-finished `git` operation), say so and do that cleanup yourself (via `bash`) before recommending `RETRY`.
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: pi-skills
|
|
||||||
namespace: agent-pod
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: agent-pod-ssh-config
|
|
||||||
namespace: agent-pod
|
|
||||||
data:
|
|
||||||
config: |
|
|
||||||
Host git.riotpiao.com
|
|
||||||
IdentityFile /root/.ssh/id_forgejo
|
|
||||||
Port 2222
|
|
||||||
User git
|
|
||||||
StrictHostKeyChecking accept-new
|
|
||||||
@@ -1,46 +0,0 @@
|
|||||||
# Edge route for the API gateway.
|
|
||||||
#
|
|
||||||
# Lives here rather than in the central k8s/bootstrap/ingress/ingress.yaml
|
|
||||||
# because that Application syncs in wave 1, before namespace `api` exists.
|
|
||||||
#
|
|
||||||
# nginx terminates TLS with the wildcard *.riotpiao.com cert (served as its
|
|
||||||
# default-ssl-certificate, so no per-rule `tls:` block is needed) and forwards
|
|
||||||
# plain HTTP to the gateway.
|
|
||||||
#
|
|
||||||
# Backend was kong-proxy:80 until Kong was retired on 2026-08-19; it is now the
|
|
||||||
# Go gateway's Service, api-gateway:8080, deployed from rock/homelab-frontend.
|
|
||||||
# Reverting the cutover is a change to these two lines and nothing else.
|
|
||||||
#
|
|
||||||
# Catch-all `/` on purpose: everything under this host belongs to the gateway.
|
|
||||||
# Listing per-API paths here would duplicate the gateway's routing table inside
|
|
||||||
# nginx, and the two copies would drift.
|
|
||||||
#
|
|
||||||
# In-cluster callers should prefer http://api-gateway.api.svc.cluster.local:8080
|
|
||||||
# directly. Resolving api.riotpiao.com sends them out to nginx and back in,
|
|
||||||
# which is a pointless hairpin unless they need TLS or the public hostname.
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: Ingress
|
|
||||||
metadata:
|
|
||||||
name: api
|
|
||||||
namespace: api
|
|
||||||
annotations:
|
|
||||||
# An API gateway carries streaming responses (SSE, gRPC-web, LLM token
|
|
||||||
# streams). nginx's 60s default read timeout and its response buffering
|
|
||||||
# would truncate or stall those.
|
|
||||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-body-size: "0"
|
|
||||||
spec:
|
|
||||||
ingressClassName: nginx
|
|
||||||
rules:
|
|
||||||
- host: api.riotpiao.com
|
|
||||||
http:
|
|
||||||
paths:
|
|
||||||
- path: /
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: api-gateway
|
|
||||||
port:
|
|
||||||
number: 8080
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
# Explicit allowlist. Anything added to this directory and not listed here is
|
|
||||||
# silently dropped — no error, no drift shown.
|
|
||||||
#
|
|
||||||
# Down to a single Ingress since Kong was retired (2026-08-19). The Kong Helm
|
|
||||||
# values, the KongClusterPlugin for Prometheus, the six KongPlugin CRs behind
|
|
||||||
# the path-per-model LLM surface, the KongConsumer and the key-auth plan all
|
|
||||||
# went with it.
|
|
||||||
resources:
|
|
||||||
- ingress.yaml
|
|
||||||
# No top-level `namespace:` transformer on purpose: ingress.yaml sets its own
|
|
||||||
# namespace, and the transformer rewrites metadata.namespace on every resource
|
|
||||||
# it builds, which is a trap for anything cross-namespace added later.
|
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
# No namespace: RuntimeClass is cluster-scoped.
|
||||||
|
resources:
|
||||||
|
- namespace.yaml
|
||||||
|
- runtimeclass.yaml
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# Namespace for GPU node-level plumbing (device plugin, and later DCGM).
|
||||||
|
#
|
||||||
|
# PodSecurity must be `privileged` here. The cluster default from the Talos
|
||||||
|
# controlplane config is `enforce: baseline` with exemptions only for
|
||||||
|
# kube-system, and a device plugin cannot satisfy baseline: it has to mount the
|
||||||
|
# kubelet device-plugin socket and the CDI/driver directories as hostPath
|
||||||
|
# volumes, which baseline forbids outright:
|
||||||
|
#
|
||||||
|
# Error creating: pods "nvidia-device-plugin-xxxxx" is forbidden:
|
||||||
|
# violates PodSecurity "baseline:latest": hostPath volumes
|
||||||
|
# (volumes "kubelet-device-plugins-dir", "mps-root", "mps-shm", "cdi-root")
|
||||||
|
#
|
||||||
|
# This is inherent to how device plugins work, not a workaround. Scope is
|
||||||
|
# limited to this namespace; the engine namespace (llm-serving) stays on the
|
||||||
|
# cluster default.
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Namespace
|
||||||
|
metadata:
|
||||||
|
name: gpu-system
|
||||||
|
labels:
|
||||||
|
pod-security.kubernetes.io/enforce: privileged
|
||||||
|
pod-security.kubernetes.io/audit: privileged
|
||||||
|
pod-security.kubernetes.io/warn: privileged
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# Cluster-scoped prerequisite for every GPU workload on worker-1.
|
||||||
|
#
|
||||||
|
# The Talos nvidia-container-toolkit extension already registers the containerd
|
||||||
|
# runtime handler (/etc/cri/conf.d/10-nvidia-container-runtime.part ->
|
||||||
|
# plugins."io.containerd.cri.v1.runtime".containerd.runtimes.nvidia), but the
|
||||||
|
# Kubernetes RuntimeClass object is separate and is NOT created by the
|
||||||
|
# extension. Without it every pod carrying runtimeClassName: nvidia is rejected
|
||||||
|
# at admission with:
|
||||||
|
# pods "..." is forbidden: pod rejected: RuntimeClass "nvidia" not found
|
||||||
|
#
|
||||||
|
# Deliberately NOT setting nvidia as containerd's default_runtime_name (the
|
||||||
|
# 20-customization.part patch in the Talos guide): that would route every pod on
|
||||||
|
# the node through the NVIDIA runtime. Opting in per-pod is narrower.
|
||||||
|
apiVersion: node.k8s.io/v1
|
||||||
|
kind: RuntimeClass
|
||||||
|
metadata:
|
||||||
|
name: nvidia
|
||||||
|
handler: nvidia
|
||||||
@@ -30,19 +30,10 @@ replicaCount: 1
|
|||||||
env:
|
env:
|
||||||
AUTH_PROVIDERS: "oidc,credentials"
|
AUTH_PROVIDERS: "oidc,credentials"
|
||||||
AUTH_OIDC_ISSUER: "https://authentik.riotpiao.com/application/o/homarr/"
|
AUTH_OIDC_ISSUER: "https://authentik.riotpiao.com/application/o/homarr/"
|
||||||
# AUTH_OIDC_URI (authorize endpoint) is REQUIRED in addition to ISSUER — homarr
|
|
||||||
# hides the "Sign in with Authentik" button entirely when it's absent (per the
|
|
||||||
# authentik Homarr integration + homarr SSO docs). This was the missing var.
|
|
||||||
AUTH_OIDC_URI: "https://authentik.riotpiao.com/application/o/authorize/"
|
|
||||||
AUTH_OIDC_CLIENT_NAME: "Authentik"
|
AUTH_OIDC_CLIENT_NAME: "Authentik"
|
||||||
AUTH_OIDC_GROUPS_ATTRIBUTE: "groups"
|
AUTH_OIDC_GROUPS_ATTRIBUTE: "groups"
|
||||||
AUTH_OIDC_SCOPE_OVERWRITE: "openid email profile groups"
|
AUTH_OIDC_SCOPE_OVERWRITE: "openid email profile groups"
|
||||||
AUTH_OIDC_AUTO_LOGIN: "false"
|
AUTH_OIDC_AUTO_LOGIN: "false"
|
||||||
# Link the OIDC identity to an existing homarr account with the same email.
|
|
||||||
OAUTH_ALLOW_DANGEROUS_EMAIL_ACCOUNT_LINKING: "true"
|
|
||||||
# The analytics cron blocked the (single-threaded) Next.js event loop for ~16s
|
|
||||||
# per run ("callback took longer than expected"), compounding CPU pressure.
|
|
||||||
DISABLE_ANALYTICS: "true"
|
|
||||||
BASE_URL: "https://homarr.riotpiao.com"
|
BASE_URL: "https://homarr.riotpiao.com"
|
||||||
NEXTAUTH_URL: "https://homarr.riotpiao.com"
|
NEXTAUTH_URL: "https://homarr.riotpiao.com"
|
||||||
|
|
||||||
@@ -67,12 +58,8 @@ tolerations:
|
|||||||
|
|
||||||
resources:
|
resources:
|
||||||
requests:
|
requests:
|
||||||
cpu: 250m
|
cpu: 100m
|
||||||
memory: 384Mi
|
memory: 256Mi
|
||||||
limits:
|
limits:
|
||||||
# Next.js 16 + bundled redis + the icon-updater (28k icons) saturated the old
|
cpu: 500m
|
||||||
# 500m limit; CPU throttling made Next.js abort with exit 134 (SIGABRT) and
|
memory: 512Mi
|
||||||
# self-restart in a loop, so nginx saw no upstream and returned 502. Give it
|
|
||||||
# real CPU headroom.
|
|
||||||
cpu: "2"
|
|
||||||
memory: 1Gi
|
|
||||||
|
|||||||
@@ -1,14 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: immich-config
|
|
||||||
data:
|
|
||||||
DB_HOSTNAME: "immich-db-rw"
|
|
||||||
DB_DATABASE_NAME: "immich"
|
|
||||||
# Only pgvector is installed (see db.yaml) - no vectorchord extension image
|
|
||||||
# exists for pg18 in CNPG's catalog yet. Explicit instead of relying on
|
|
||||||
# auto-detect's vectorchord-first preference order.
|
|
||||||
DB_VECTOR_EXTENSION: "pgvector"
|
|
||||||
REDIS_HOSTNAME: "immich-redis"
|
|
||||||
IMMICH_MACHINE_LEARNING_URL: "http://immich-machine-learning:3003"
|
|
||||||
TZ: "America/Los_Angeles"
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
# Dedicated CNPG Postgres for Immich. Same recipe as paperless-db/authentik-db
|
|
||||||
# (2 instances, default longhorn storage class) except the operand is
|
|
||||||
# PostgreSQL 18, not 16.2 - the official CNPG pgvector extension image
|
|
||||||
# (ghcr.io/cloudnative-pg/pgvector) is only published for pg18, no pg16 tags
|
|
||||||
# exist in that registry. Immich itself supports pg18 fine (immich-app's own
|
|
||||||
# postgres image already ships 18-vectorchord builds).
|
|
||||||
#
|
|
||||||
# pgvector loaded via CNPG's ImageVolume extension mechanism (CNPG 1.27+,
|
|
||||||
# k8s ImageVolume feature - both present here: operator is 1.30.0, cluster is
|
|
||||||
# v1.36.1). No shared_preload_libraries needed - pgvector doesn't require
|
|
||||||
# preload, just CREATE EXTENSION, which immich-server issues itself at
|
|
||||||
# startup. Distro/pg-major must match between the operand image and the
|
|
||||||
# extension image (both "18"+"trixie" here) - CNPG's own compatibility rule.
|
|
||||||
apiVersion: postgresql.cnpg.io/v1
|
|
||||||
kind: Cluster
|
|
||||||
metadata:
|
|
||||||
name: immich-db
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
|
|
||||||
spec:
|
|
||||||
instances: 2
|
|
||||||
imageName: ghcr.io/cloudnative-pg/postgresql:18-minimal-trixie
|
|
||||||
postgresql:
|
|
||||||
extensions:
|
|
||||||
- name: pgvector
|
|
||||||
image:
|
|
||||||
reference: ghcr.io/cloudnative-pg/pgvector:0.8.1-18-trixie
|
|
||||||
bootstrap:
|
|
||||||
initdb:
|
|
||||||
database: immich
|
|
||||||
owner: app
|
|
||||||
encoding: UTF8
|
|
||||||
localeCollate: C
|
|
||||||
localeCType: C
|
|
||||||
# CREATE EXTENSION vector requires superuser (pgvector's control file
|
|
||||||
# isn't marked trusted) and the "app" owner role isn't one
|
|
||||||
# (enableSuperuserAccess: false, repo convention) - postInitApplicationSQL
|
|
||||||
# runs as superuser during initdb, before the app ever connects. Only
|
|
||||||
# fires on a fresh bootstrap; the live cluster already had this run
|
|
||||||
# manually once (kubectl exec ... psql -U postgres -c 'CREATE EXTENSION').
|
|
||||||
postInitApplicationSQL:
|
|
||||||
- "CREATE EXTENSION IF NOT EXISTS vector;"
|
|
||||||
- "CREATE EXTENSION IF NOT EXISTS cube;"
|
|
||||||
- "CREATE EXTENSION IF NOT EXISTS earthdistance;"
|
|
||||||
enableSuperuserAccess: false
|
|
||||||
resources:
|
|
||||||
requests: { memory: "512Mi", cpu: "250m" }
|
|
||||||
limits: { memory: "2Gi", cpu: "1" }
|
|
||||||
storage:
|
|
||||||
size: 20Gi
|
|
||||||
storageClass: longhorn
|
|
||||||
affinity:
|
|
||||||
podAntiAffinityType: preferred
|
|
||||||
topologyKey: kubernetes.io/hostname
|
|
||||||
tolerations:
|
|
||||||
- key: node-role.kubernetes.io/control-plane
|
|
||||||
operator: Exists
|
|
||||||
effect: NoSchedule
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
# immich-server: pinned to talos-cp-3, same reasoning as paperless
|
|
||||||
# (deployment.yaml comment there) - immich-media is a ReadWriteOnce Longhorn
|
|
||||||
# volume with a single replica physically on that node's disk (shared with
|
|
||||||
# paperless-media on the same 4TB HDD). Recreate strategy for the same
|
|
||||||
# reason: two pods can't both attach an RWO volume.
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: immich-server
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: immich-server
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: immich-server
|
|
||||||
spec:
|
|
||||||
serviceAccountName: immich
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: talos-cp-3
|
|
||||||
containers:
|
|
||||||
- name: immich-server
|
|
||||||
image: ghcr.io/immich-app/immich-server:release
|
|
||||||
ports:
|
|
||||||
- containerPort: 2283
|
|
||||||
envFrom:
|
|
||||||
- configMapRef:
|
|
||||||
name: immich-config
|
|
||||||
env:
|
|
||||||
- name: DB_USERNAME
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: immich-db-app
|
|
||||||
key: username
|
|
||||||
- name: DB_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: immich-db-app
|
|
||||||
key: password
|
|
||||||
# Composed by k8s/infra/iam's provisioning script (system-config
|
|
||||||
# JSON, oauth section) - see immich-oidc Secret.
|
|
||||||
- name: IMMICH_CONFIG_FILE
|
|
||||||
value: /config/immich.json
|
|
||||||
resources:
|
|
||||||
requests: { cpu: "500m", memory: "1Gi" }
|
|
||||||
limits: { cpu: "2", memory: "4Gi" }
|
|
||||||
volumeMounts:
|
|
||||||
- name: media
|
|
||||||
mountPath: /usr/src/app/upload
|
|
||||||
- name: oidc-config
|
|
||||||
mountPath: /config
|
|
||||||
readOnly: true
|
|
||||||
volumes:
|
|
||||||
- name: media
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: immich-media
|
|
||||||
- name: oidc-config
|
|
||||||
secret:
|
|
||||||
secretName: immich-oidc
|
|
||||||
items:
|
|
||||||
- key: config.json
|
|
||||||
path: immich.json
|
|
||||||
---
|
|
||||||
# CPU-only for now - the cluster's one GPU node (worker-1) is already
|
|
||||||
# dedicated to llm-serving predictors. Not node-pinned: its cache PVC is on
|
|
||||||
# the default 3-replica pool, not the single-disk cp-3 HDD.
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: immich-machine-learning
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: immich-machine-learning
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: immich-machine-learning
|
|
||||||
spec:
|
|
||||||
serviceAccountName: immich
|
|
||||||
containers:
|
|
||||||
- name: immich-machine-learning
|
|
||||||
image: ghcr.io/immich-app/immich-machine-learning:release
|
|
||||||
ports:
|
|
||||||
- containerPort: 3003
|
|
||||||
resources:
|
|
||||||
requests: { cpu: "500m", memory: "1Gi" }
|
|
||||||
limits: { cpu: "2", memory: "4Gi" }
|
|
||||||
volumeMounts:
|
|
||||||
- name: ml-cache
|
|
||||||
mountPath: /cache
|
|
||||||
volumes:
|
|
||||||
- name: ml-cache
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: immich-ml-cache
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
# Direct nginx ingress, same reasoning as paperless: large uploads (photos/
|
|
||||||
# videos) and long-lived operations (video transcode, big batch uploads) need
|
|
||||||
# proxy-body-size/timeouts raised past nginx's defaults.
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: Ingress
|
|
||||||
metadata:
|
|
||||||
name: immich
|
|
||||||
annotations:
|
|
||||||
nginx.ingress.kubernetes.io/proxy-body-size: "0"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
|
||||||
spec:
|
|
||||||
ingressClassName: nginx
|
|
||||||
rules:
|
|
||||||
- host: img.riotpiao.com
|
|
||||||
http:
|
|
||||||
paths:
|
|
||||||
- path: /
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: immich-server
|
|
||||||
port:
|
|
||||||
number: 2283
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
namespace: immich
|
|
||||||
resources:
|
|
||||||
- db.yaml
|
|
||||||
- pvc.yaml
|
|
||||||
- configmap.yaml
|
|
||||||
- redis.yaml
|
|
||||||
- deployment.yaml
|
|
||||||
- service.yaml
|
|
||||||
- ingress.yaml
|
|
||||||
- rbac.yaml
|
|
||||||
# immich-oidc Secret written by the PostSync provisioning Job in
|
|
||||||
# k8s/infra/iam (same as paperless-oidc) - not duplicated here.
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
# Two volumes:
|
|
||||||
#
|
|
||||||
# - media: original photos/videos + generated thumbnails/encoded videos.
|
|
||||||
# Shares the cp-3 USB HDD with paperless-media, same StorageClass/disk tag,
|
|
||||||
# single replica (single disk, no redundancy possible - same tradeoff
|
|
||||||
# paperless already accepts). Sized 1400Gi, not 2000Gi: the disk's real
|
|
||||||
# usable capacity (~3724GiB, formatting overhead) minus paperless-media's
|
|
||||||
# 2000Gi and ~231GiB of other apps' default-class replicas that Longhorn
|
|
||||||
# placed here anyway (disk tags only pull matching volumes in, they don't
|
|
||||||
# exclude non-matching ones when the untagged pool elsewhere is full) only
|
|
||||||
# leaves ~1493Gi of real scheduling headroom right now.
|
|
||||||
# - ml-cache: downloaded ML model weights for immich-machine-learning
|
|
||||||
# (face detection / CLIP embeddings). Small, disposable (re-downloads on
|
|
||||||
# loss), but persisted so a pod restart doesn't re-pull multi-GB models -
|
|
||||||
# default 3-replica pool, not node-pinned.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: immich-media
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: longhorn-paperless-media
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 1400Gi
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: immich-ml-cache
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: longhorn
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 5Gi
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
# Scoped operator access for immich-admins: restart/config-edit rights on
|
|
||||||
# just this service's own resources, nothing CNPG-managed (immich-db-*) or
|
|
||||||
# provisioning-managed (immich-oidc). Same pattern as
|
|
||||||
# k8s/apps/paperless/rbac.yaml. Inert until kube-apiserver's OIDC wiring
|
|
||||||
# lands (--oidc-groups-claim=groups, --oidc-groups-prefix=oidc:).
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ServiceAccount
|
|
||||||
metadata:
|
|
||||||
name: immich
|
|
||||||
---
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: Role
|
|
||||||
metadata:
|
|
||||||
name: immich-operator
|
|
||||||
rules:
|
|
||||||
- apiGroups: ["apps"]
|
|
||||||
resources: ["deployments"]
|
|
||||||
resourceNames: ["immich-server", "immich-machine-learning"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["configmaps"]
|
|
||||||
resourceNames: ["immich-config"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["secrets"]
|
|
||||||
resourceNames: ["immich-oidc"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
---
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: RoleBinding
|
|
||||||
metadata:
|
|
||||||
name: immich-admins-binding
|
|
||||||
subjects:
|
|
||||||
- kind: Group
|
|
||||||
name: "oidc:immich-admins"
|
|
||||||
apiGroup: rbac.authorization.k8s.io
|
|
||||||
- kind: ServiceAccount
|
|
||||||
name: immich
|
|
||||||
namespace: immich
|
|
||||||
roleRef:
|
|
||||||
kind: Role
|
|
||||||
name: immich-operator
|
|
||||||
apiGroup: rbac.authorization.k8s.io
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
# Job queue broker for immich-server. No PVC: queue state is disposable - a
|
|
||||||
# lost queue on restart just re-triggers the affected background jobs
|
|
||||||
# (thumbnail generation, ML jobs, etc.), no photo data loss since originals
|
|
||||||
# live on immich-media.
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: immich-redis
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: immich-redis
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: immich-redis
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: redis
|
|
||||||
image: redis:7-alpine
|
|
||||||
ports:
|
|
||||||
- containerPort: 6379
|
|
||||||
resources:
|
|
||||||
requests: { cpu: "50m", memory: "64Mi" }
|
|
||||||
limits: { cpu: "250m", memory: "256Mi" }
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: immich-redis
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: immich-redis
|
|
||||||
ports:
|
|
||||||
- port: 6379
|
|
||||||
targetPort: 6379
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: immich-server
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: immich-server
|
|
||||||
ports:
|
|
||||||
- port: 2283
|
|
||||||
targetPort: 2283
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: immich-machine-learning
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: immich-machine-learning
|
|
||||||
ports:
|
|
||||||
- port: 3003
|
|
||||||
targetPort: 3003
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
apiVersion: serving.kserve.io/v1beta1
|
|
||||||
kind: InferenceService
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
serving.kserve.io/deploymentMode: RawDeployment
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: llm-embeddings
|
|
||||||
app.kubernetes.io/part-of: llm-serving
|
|
||||||
name: embeddings
|
|
||||||
namespace: llm-serving
|
|
||||||
spec:
|
|
||||||
predictor:
|
|
||||||
containers:
|
|
||||||
- args:
|
|
||||||
- --model-id=nomic-ai/nomic-embed-text-v2-moe
|
|
||||||
- --port=8080
|
|
||||||
- --hostname=0.0.0.0
|
|
||||||
- --auto-truncate
|
|
||||||
env:
|
|
||||||
- name: HUGGINGFACE_HUB_CACHE
|
|
||||||
value: /mnt/models
|
|
||||||
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2@sha256:4d632b76bd14cb57044a1ffb0ad48ab0ba4939e705a9a615ccc740658575c26e
|
|
||||||
name: kserve-container
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
protocol: TCP
|
|
||||||
readinessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 10
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: '16'
|
|
||||||
memory: 8Gi
|
|
||||||
requests:
|
|
||||||
cpu: '8'
|
|
||||||
memory: 4Gi
|
|
||||||
startupProbe:
|
|
||||||
failureThreshold: 60
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 10
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /mnt/models
|
|
||||||
name: models
|
|
||||||
maxReplicas: 1
|
|
||||||
minReplicas: 1
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: worker-1
|
|
||||||
volumes:
|
|
||||||
- name: models
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: llm-models
|
|
||||||
|
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Embeddings — Nomic Embed Text v2 (MoE), on CPU via HuggingFace TEI.
|
||||||
|
#
|
||||||
|
# CPU, not GPU, deliberately. All 4 V100s are claimed by the generation models,
|
||||||
|
# and the device plugin hands out WHOLE GPUs — a 5th GPU-requesting pod is
|
||||||
|
# unschedulable no matter how much VRAM is free. Sharing would need global
|
||||||
|
# time-slicing, which on a single node cannot be scoped to one card and would let
|
||||||
|
# the scheduler co-locate two ~20GB models and OOM both.
|
||||||
|
#
|
||||||
|
# worker-1 has 96 cores with ~250m requested, and this is a 475M-param encoder
|
||||||
|
# (305M active). Retrieval runs once per agent-loop iteration, not per token, so
|
||||||
|
# CPU latency here is immaterial. This is also what Plan 1 originally specified.
|
||||||
|
#
|
||||||
|
# TEI (not vLLM) because it is purpose-built for encoders and explicitly lists
|
||||||
|
# nomic-embed-text-v2-moe as supported.
|
||||||
|
#
|
||||||
|
# NOTE: Nomic v2 requires task prefixes on the CLIENT side —
|
||||||
|
# documents: "search_document: <text>"
|
||||||
|
# queries: "search_query: <text>"
|
||||||
|
# Embedding without the prefix silently degrades retrieval quality.
|
||||||
|
apiVersion: serving.kserve.io/v1beta1
|
||||||
|
kind: InferenceService
|
||||||
|
metadata:
|
||||||
|
name: embeddings
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-embeddings
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
spec:
|
||||||
|
predictor:
|
||||||
|
minReplicas: 1
|
||||||
|
maxReplicas: 1
|
||||||
|
# Pinned to worker-1 only so it can share the RWO models PVC with the GPU
|
||||||
|
# pods (RWO = single node, any number of pods on it).
|
||||||
|
nodeSelector:
|
||||||
|
kubernetes.io/hostname: worker-1
|
||||||
|
containers:
|
||||||
|
- name: kserve-container
|
||||||
|
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2@sha256:4d632b76bd14cb57044a1ffb0ad48ab0ba4939e705a9a615ccc740658575c26e
|
||||||
|
args:
|
||||||
|
- --model-id=nomic-ai/nomic-embed-text-v2-moe
|
||||||
|
- --port=8080
|
||||||
|
- --hostname=0.0.0.0
|
||||||
|
# Truncate rather than 413 on over-long input.
|
||||||
|
- --auto-truncate
|
||||||
|
env:
|
||||||
|
- name: HUGGINGFACE_HUB_CACHE
|
||||||
|
value: /mnt/models
|
||||||
|
ports:
|
||||||
|
- containerPort: 8080
|
||||||
|
protocol: TCP
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "8"
|
||||||
|
memory: 4Gi
|
||||||
|
limits:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 8Gi
|
||||||
|
volumeMounts:
|
||||||
|
- name: models
|
||||||
|
mountPath: /mnt/models
|
||||||
|
startupProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
failureThreshold: 60
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
volumes:
|
||||||
|
- name: models
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: llm-models
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
# Action engine — Ornith-1.0-35B on Ollama.
|
||||||
|
#
|
||||||
|
# Why not vLLM like the other two: Ornith is
|
||||||
|
# Qwen3_5MoeForConditionalGeneration (Qwen3.5 MoE, 256 experts / 8 active,
|
||||||
|
# hybrid attention — 30 linear_attention + 10 full_attention layers). vLLM's
|
||||||
|
# Qwen3.5 support landed 2026-07-29, AFTER vLLM dropped Volta (sm_70) at
|
||||||
|
# v0.11.1. No vLLM build has both, so Ornith cannot run on vLLM on a V100.
|
||||||
|
#
|
||||||
|
# Ollama ships `ornith:35b` in its library and runs a llama-server runner
|
||||||
|
# underneath, which keeps Volta support. q4 is ~21GB — fits one 32GB V100 with
|
||||||
|
# room for KV.
|
||||||
|
#
|
||||||
|
# OLLAMA_KEEP_ALIVE=-1 is load-bearing: the harness calls this every loop
|
||||||
|
# iteration, and Ollama's default is to evict an idle model after 5m, which
|
||||||
|
# would add a ~21GB reload to a random future request.
|
||||||
|
apiVersion: serving.kserve.io/v1beta1
|
||||||
|
kind: InferenceService
|
||||||
|
metadata:
|
||||||
|
name: ornith
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-ornith
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
spec:
|
||||||
|
predictor:
|
||||||
|
minReplicas: 1
|
||||||
|
# Recreate, not the default RollingUpdate: GPUs are allocated exactly 4/4,
|
||||||
|
# so a surge pod has no card to claim and sits Pending while the old pod is
|
||||||
|
# never torn down — a deadlock. Recreate tears down first, accepting a brief
|
||||||
|
# gap during updates.
|
||||||
|
deploymentStrategy:
|
||||||
|
type: Recreate
|
||||||
|
maxReplicas: 1
|
||||||
|
nodeSelector:
|
||||||
|
kubernetes.io/hostname: worker-1
|
||||||
|
runtimeClassName: nvidia
|
||||||
|
containers:
|
||||||
|
- name: kserve-container
|
||||||
|
image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
|
||||||
|
# `ollama serve` does not pull models, and `ollama pull` needs a running
|
||||||
|
# server — so background the server, wait for it, pull, then hand the
|
||||||
|
# foreground back to serve.
|
||||||
|
command:
|
||||||
|
- /bin/sh
|
||||||
|
- -c
|
||||||
|
- |
|
||||||
|
set -e
|
||||||
|
ollama serve &
|
||||||
|
SERVE_PID=$!
|
||||||
|
until ollama list >/dev/null 2>&1; do sleep 2; done
|
||||||
|
ollama pull ornith:35b
|
||||||
|
ollama pull qwen2.5:3b-instruct
|
||||||
|
wait $SERVE_PID
|
||||||
|
env:
|
||||||
|
# Match the port the other two engines use.
|
||||||
|
- name: OLLAMA_HOST
|
||||||
|
value: "0.0.0.0:8080"
|
||||||
|
- name: OLLAMA_MODELS
|
||||||
|
value: /mnt/models/ollama
|
||||||
|
# Ollama defaults to a 4096 context, far too small for an agentic
|
||||||
|
# coding model. Ornith's hybrid attention means only 10 of its 40
|
||||||
|
# layers hold a conventional KV cache, so 32K is affordable in the
|
||||||
|
# ~11GiB left after its 21GB of weights.
|
||||||
|
- name: OLLAMA_CONTEXT_LENGTH
|
||||||
|
value: "32768"
|
||||||
|
# Never evict — this model is on the harness's hot path.
|
||||||
|
- name: OLLAMA_KEEP_ALIVE
|
||||||
|
value: "-1"
|
||||||
|
# Serial agent loop; no benefit from parallel slots.
|
||||||
|
- name: OLLAMA_NUM_PARALLEL
|
||||||
|
value: "1"
|
||||||
|
# 2, so ornith and the small utility model stay co-resident on GPU2
|
||||||
|
# instead of evicting one another on every alternating request.
|
||||||
|
- name: OLLAMA_MAX_LOADED_MODELS
|
||||||
|
value: "2"
|
||||||
|
ports:
|
||||||
|
- containerPort: 8080
|
||||||
|
protocol: TCP
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "8"
|
||||||
|
memory: 8Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
limits:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 16Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
volumeMounts:
|
||||||
|
- name: models
|
||||||
|
mountPath: /mnt/models
|
||||||
|
# Probes must confirm the MODEL is present, not just that the server
|
||||||
|
# answers. Ollama's `GET /` returns 200 ("Ollama is running") the moment
|
||||||
|
# `ollama serve` binds — which is before the ~21GB pull finishes. An
|
||||||
|
# httpGet probe would therefore mark this pod Ready with no model
|
||||||
|
# loaded, and KServe would route traffic to it.
|
||||||
|
startupProbe:
|
||||||
|
exec:
|
||||||
|
command: ["/bin/sh", "-c", "ollama list 2>/dev/null | grep -q ornith && ollama list 2>/dev/null | grep -q qwen2.5"]
|
||||||
|
periodSeconds: 15
|
||||||
|
failureThreshold: 120
|
||||||
|
readinessProbe:
|
||||||
|
exec:
|
||||||
|
command: ["/bin/sh", "-c", "ollama list 2>/dev/null | grep -q ornith && ollama list 2>/dev/null | grep -q qwen2.5"]
|
||||||
|
periodSeconds: 10
|
||||||
|
volumes:
|
||||||
|
- name: models
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: llm-models
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
# Reasoning engine — DeepSeek-R1-Distill-Qwen-32B, GPTQ INT4, vLLM.
|
||||||
|
#
|
||||||
|
# TP=1 with 2 data-parallel replicas (GPU0 + GPU1) rather than one TP=2 engine:
|
||||||
|
# worker-1 has NO NVLink, so tensor-parallel's per-token all-reduce would cross
|
||||||
|
# PCIe on every decode step. Two independent replicas need zero inter-GPU
|
||||||
|
# communication and KServe load-balances them behind one Service.
|
||||||
|
#
|
||||||
|
# vLLM is pinned to v0.11.0 — the LAST release that compiles sm_70 (Volta)
|
||||||
|
# kernels. v0.11.1 dropped 7.0 from CUDA_SUPPORTED_ARCHS. Do not bump this
|
||||||
|
# without re-checking CMakeLists.txt, or every pod dies with "no kernel image".
|
||||||
|
apiVersion: serving.kserve.io/v1beta1
|
||||||
|
kind: InferenceService
|
||||||
|
metadata:
|
||||||
|
name: reasoning
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-reasoning
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
spec:
|
||||||
|
predictor:
|
||||||
|
minReplicas: 2
|
||||||
|
# Recreate, not the default RollingUpdate: GPUs are allocated exactly 4/4,
|
||||||
|
# so a surge pod has no card to claim and sits Pending while the old pod is
|
||||||
|
# never torn down — a deadlock. Recreate tears down first, accepting a brief
|
||||||
|
# gap during updates.
|
||||||
|
deploymentStrategy:
|
||||||
|
type: Recreate
|
||||||
|
maxReplicas: 2
|
||||||
|
nodeSelector:
|
||||||
|
kubernetes.io/hostname: worker-1
|
||||||
|
runtimeClassName: nvidia
|
||||||
|
containers:
|
||||||
|
- name: kserve-container
|
||||||
|
image: vllm/vllm-openai:v0.11.0@sha256:014a95f21c9edf6abe0aea6b07353f96baa4ec291c427bb1176dc7c93a85845c
|
||||||
|
args:
|
||||||
|
# GPTQ, NOT AWQ. vLLM hard-refuses AWQ below compute capability 75:
|
||||||
|
# "The quantization method awq is not supported for the current GPU.
|
||||||
|
# Minimum capability: 75. Current capability: 70."
|
||||||
|
# V100 is sm_70. GPTQ's min capability is 60, so it runs. (gptq_marlin
|
||||||
|
# needs 80 and fp8 needs 80 — both also out.) Same 19.3GB footprint.
|
||||||
|
# desc_act=False in this build: no activation reordering, faster.
|
||||||
|
- --model=unsloth/DeepSeek-R1-Distill-Qwen-32B-bnb-4bit
|
||||||
|
- --served-model-name=reasoning
|
||||||
|
# bitsandbytes nf4. GPTQ passed vLLM's min_capability=60 check but was
|
||||||
|
# numerically WRONG on sm_70 (garbage logits) — proven by the fp16
|
||||||
|
# control run producing correct text with the identical backend. bnb
|
||||||
|
# declares min_capability=70, but treat that as unverified until the
|
||||||
|
# output itself is checked.
|
||||||
|
# NOTE the repo sets bnb_4bit_compute_dtype=bfloat16, which Volta does
|
||||||
|
# not have; --dtype=float16 must override it.
|
||||||
|
- --quantization=bitsandbytes
|
||||||
|
# Volta has no bf16 — must be explicit, the repo's weights are bf16.
|
||||||
|
- --dtype=float16
|
||||||
|
# No FP8 KV on Volta; stays fp16.
|
||||||
|
- --kv-cache-dtype=auto
|
||||||
|
- --tensor-parallel-size=1
|
||||||
|
- --max-model-len=16384
|
||||||
|
# VRAM budget on a 32GiB V100: 0.92 => ~29.4GiB, minus ~18GiB of GPTQ
|
||||||
|
# weights leaves ~11GiB for KV + activations. One full 32K sequence
|
||||||
|
# costs 32768 x 256KB = 8GiB of KV, so 8 concurrent full-length
|
||||||
|
# sequences is not physically possible here — 4 is honest, and a
|
||||||
|
# serial single-user harness never needs more.
|
||||||
|
- --gpu-memory-utilization=0.90
|
||||||
|
- --max-num-seqs=4
|
||||||
|
# Smooths Volta's slow prefill (no FlashAttention2 on sm_70).
|
||||||
|
- --enable-chunked-prefill
|
||||||
|
- --enable-prefix-caching
|
||||||
|
# Splits <think>…</think> into its own channel.
|
||||||
|
- --reasoning-parser=deepseek_r1
|
||||||
|
- --host=0.0.0.0
|
||||||
|
- --port=8080
|
||||||
|
env:
|
||||||
|
# FlashAttention2 requires sm_80; Volta must fall back to xformers.
|
||||||
|
# flashinfer's check_cuda_arch() has an upstream bug that crashes on
|
||||||
|
# ANY sm_7x GPU: `elif major == 7 and minor.isdigit()` calls .isdigit()
|
||||||
|
# on an int, so instead of reporting "unsupported" it raises
|
||||||
|
# AttributeError: 'int' object has no attribute 'isdigit'
|
||||||
|
# and engine init dies. Default is None (auto-detect), which walks
|
||||||
|
# straight into that path. 0 disables the flashinfer sampler outright.
|
||||||
|
# Only affects the generate runner — the verifier (pooling) never hits
|
||||||
|
# the sampler, which is why it started fine and this did not.
|
||||||
|
- name: VLLM_USE_FLASHINFER_SAMPLER
|
||||||
|
value: "0"
|
||||||
|
# TRITON_ATTN, not XFORMERS. On sm_70 every xformers kernel is
|
||||||
|
# rejected for V1's paged-attention bias type:
|
||||||
|
# fa2F / triton_splitKF -> require sm_80
|
||||||
|
# cutlassF -> supports sm_70 but not
|
||||||
|
# PagedBlockDiagonalCausalWithOffsetPaddedKeysMask
|
||||||
|
# -> NotImplementedError kills EngineCore on the FIRST request, which
|
||||||
|
# takes the whole pod down (vLLM treats engine death as fatal).
|
||||||
|
# V0, whose hand-written paged kernels did support sm_70, was REMOVED
|
||||||
|
# in v0.11.0, so VLLM_USE_V1=0 has nothing to fall back to.
|
||||||
|
# Triton JIT-compiles for the local arch, so it is the last option.
|
||||||
|
- name: VLLM_ATTENTION_BACKEND
|
||||||
|
value: TRITON_ATTN
|
||||||
|
- name: HF_HOME
|
||||||
|
value: /mnt/models
|
||||||
|
ports:
|
||||||
|
- containerPort: 8080
|
||||||
|
protocol: TCP
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "8"
|
||||||
|
memory: 8Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
limits:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 16Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
volumeMounts:
|
||||||
|
- name: models
|
||||||
|
mountPath: /mnt/models
|
||||||
|
- name: shm
|
||||||
|
mountPath: /dev/shm
|
||||||
|
startupProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
# Cold start pulls ~18Gi of weights over Longhorn, then loads to VRAM.
|
||||||
|
periodSeconds: 15
|
||||||
|
failureThreshold: 80
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
volumes:
|
||||||
|
- name: models
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: llm-models
|
||||||
|
- name: shm
|
||||||
|
emptyDir:
|
||||||
|
medium: Memory
|
||||||
|
sizeLimit: 2Gi
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Reranker — BAAI/bge-reranker-base, on CPU via HuggingFace TEI.
|
||||||
|
#
|
||||||
|
# Second stage of retrieval: the embedding model fetches a coarse top-k by
|
||||||
|
# vector similarity, this cross-encoder re-scores those candidates against the
|
||||||
|
# query directly. That is what fixes the "semantic dilution" problem in Plan 1 —
|
||||||
|
# a single embedding vector cannot represent a large chunk faithfully, so
|
||||||
|
# ranking by cosine alone surfaces near-misses.
|
||||||
|
#
|
||||||
|
# CPU for the same reason as the embedding service: all 4 GPUs are claimed and
|
||||||
|
# the device plugin allocates whole cards. A 568M cross-encoder scoring ~20-50
|
||||||
|
# candidates per query is well within CPU budget.
|
||||||
|
#
|
||||||
|
# Arch is XLMRobertaForSequenceClassification, which TEI serves as /rerank.
|
||||||
|
# 278M params — smaller than v2-m3 (568M) and English/Chinese rather than
|
||||||
|
# multilingual, which suits code+docs retrieval and is faster on CPU.
|
||||||
|
apiVersion: serving.kserve.io/v1beta1
|
||||||
|
kind: InferenceService
|
||||||
|
metadata:
|
||||||
|
name: reranker
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-reranker
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
spec:
|
||||||
|
predictor:
|
||||||
|
minReplicas: 1
|
||||||
|
maxReplicas: 1
|
||||||
|
# Same worker-1 pin as the embedding service, to share the RWO models PVC.
|
||||||
|
nodeSelector:
|
||||||
|
kubernetes.io/hostname: worker-1
|
||||||
|
containers:
|
||||||
|
- name: kserve-container
|
||||||
|
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2@sha256:4d632b76bd14cb57044a1ffb0ad48ab0ba4939e705a9a615ccc740658575c26e
|
||||||
|
args:
|
||||||
|
# bge-reranker-base, NOT v2-m3. TEI's CPU image starts the ONNX
|
||||||
|
# Runtime backend and v2-m3 ships no ONNX files, so it dies with
|
||||||
|
# "Model ONNX files not found in the repository". This build does.
|
||||||
|
- --model-id=BAAI/bge-reranker-base
|
||||||
|
- --port=8080
|
||||||
|
- --hostname=0.0.0.0
|
||||||
|
- --auto-truncate
|
||||||
|
env:
|
||||||
|
- name: HUGGINGFACE_HUB_CACHE
|
||||||
|
value: /mnt/models
|
||||||
|
ports:
|
||||||
|
- containerPort: 8080
|
||||||
|
protocol: TCP
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "8"
|
||||||
|
memory: 4Gi
|
||||||
|
limits:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 8Gi
|
||||||
|
volumeMounts:
|
||||||
|
- name: models
|
||||||
|
mountPath: /mnt/models
|
||||||
|
startupProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
failureThreshold: 60
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
volumes:
|
||||||
|
- name: models
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: llm-models
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Verifier — Qwen2.5-Math-PRM-7B, step-level process reward model, vLLM.
|
||||||
|
#
|
||||||
|
# Model choice was constrained by vLLM v0.11.0's registry: its arch
|
||||||
|
# (Qwen2ForProcessRewardModel) is natively registered, whereas the smaller
|
||||||
|
# community PRMs are Qwen2ForTokenClassification / Qwen2ForPrmModel, neither of
|
||||||
|
# which v0.11.0 can load (ForTokenClassification is absent from
|
||||||
|
# _SUFFIX_TO_DEFAULTS, so it won't even auto-convert).
|
||||||
|
#
|
||||||
|
# --runner pooling, NOT --task reward: --task is [DEPRECATED] in v0.11.0.
|
||||||
|
# Scoring goes to /pooling, not /v1/completions — this is a reward model, it
|
||||||
|
# returns scores, not tokens.
|
||||||
|
#
|
||||||
|
# Gets a whole dedicated GPU despite only needing ~15Gi: it sits on the
|
||||||
|
# harness's critical path (every reasoning->action->verify iteration waits on
|
||||||
|
# it), so isolation from the generation engines' decode loops matters more than
|
||||||
|
# the idle VRAM.
|
||||||
|
apiVersion: serving.kserve.io/v1beta1
|
||||||
|
kind: InferenceService
|
||||||
|
metadata:
|
||||||
|
name: verifier
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-verifier
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
spec:
|
||||||
|
predictor:
|
||||||
|
minReplicas: 1
|
||||||
|
# Recreate, not the default RollingUpdate: GPUs are allocated exactly 4/4,
|
||||||
|
# so a surge pod has no card to claim and sits Pending while the old pod is
|
||||||
|
# never torn down — a deadlock. Recreate tears down first, accepting a brief
|
||||||
|
# gap during updates.
|
||||||
|
deploymentStrategy:
|
||||||
|
type: Recreate
|
||||||
|
maxReplicas: 1
|
||||||
|
nodeSelector:
|
||||||
|
kubernetes.io/hostname: worker-1
|
||||||
|
runtimeClassName: nvidia
|
||||||
|
containers:
|
||||||
|
- name: kserve-container
|
||||||
|
image: vllm/vllm-openai:v0.11.0@sha256:014a95f21c9edf6abe0aea6b07353f96baa4ec291c427bb1176dc7c93a85845c
|
||||||
|
args:
|
||||||
|
- --model=Qwen/Qwen2.5-Math-PRM-7B
|
||||||
|
- --served-model-name=verifier
|
||||||
|
# Pooling runner => reward scoring. Weights are bf16; Volta needs fp16.
|
||||||
|
- --runner=pooling
|
||||||
|
- --dtype=float16
|
||||||
|
- --tensor-parallel-size=1
|
||||||
|
- --max-model-len=4096
|
||||||
|
- --max-num-seqs=8
|
||||||
|
- --host=0.0.0.0
|
||||||
|
- --port=8080
|
||||||
|
env:
|
||||||
|
# flashinfer's check_cuda_arch() has an upstream bug that crashes on
|
||||||
|
# ANY sm_7x GPU: `elif major == 7 and minor.isdigit()` calls .isdigit()
|
||||||
|
# on an int, so instead of reporting "unsupported" it raises
|
||||||
|
# AttributeError: 'int' object has no attribute 'isdigit'
|
||||||
|
# and engine init dies. Default is None (auto-detect), which walks
|
||||||
|
# straight into that path. 0 disables the flashinfer sampler outright.
|
||||||
|
# Only affects the generate runner — the verifier (pooling) never hits
|
||||||
|
# the sampler, which is why it started fine and this did not.
|
||||||
|
- name: VLLM_USE_FLASHINFER_SAMPLER
|
||||||
|
value: "0"
|
||||||
|
- name: VLLM_ATTENTION_BACKEND
|
||||||
|
value: XFORMERS
|
||||||
|
- name: HF_HOME
|
||||||
|
value: /mnt/models
|
||||||
|
ports:
|
||||||
|
- containerPort: 8080
|
||||||
|
protocol: TCP
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "4"
|
||||||
|
memory: 8Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
limits:
|
||||||
|
cpu: "16"
|
||||||
|
memory: 16Gi
|
||||||
|
nvidia.com/gpu: "1"
|
||||||
|
volumeMounts:
|
||||||
|
- name: models
|
||||||
|
mountPath: /mnt/models
|
||||||
|
- name: shm
|
||||||
|
mountPath: /dev/shm
|
||||||
|
startupProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 15
|
||||||
|
failureThreshold: 60
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8080
|
||||||
|
periodSeconds: 10
|
||||||
|
volumes:
|
||||||
|
- name: models
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: llm-models
|
||||||
|
- name: shm
|
||||||
|
emptyDir:
|
||||||
|
medium: Memory
|
||||||
|
sizeLimit: 1Gi
|
||||||
@@ -1,18 +1,13 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
kind: Kustomization
|
kind: Kustomization
|
||||||
# Explicit allowlist, matching k8s/apps/api. Anything added to this directory
|
namespace: llm-serving
|
||||||
# and not listed here is silently dropped — no error, no drift shown.
|
|
||||||
#
|
|
||||||
# These five were adopted from live state on 2026-08-15; they had been applied
|
|
||||||
# by hand and carried no ArgoCD ownership. Each was exported and verified with
|
|
||||||
# `kubectl diff -f <file>` returning empty before the Application below was
|
|
||||||
# created, so the first sync was a no-op rather than a redeploy. Re-verify that
|
|
||||||
# way after any edit here: a GPU predictor restart is a weights reload measured
|
|
||||||
# in tens of seconds, not a rolling update.
|
|
||||||
resources:
|
resources:
|
||||||
- embeddings.yaml
|
- namespace.yaml
|
||||||
- ornith.yaml
|
- storageclass.yaml
|
||||||
- reasoning.yaml
|
- pvc-models.yaml
|
||||||
- reranker.yaml
|
- inferenceservice-reasoning.yaml
|
||||||
# No namespace transformer: every file sets its own, and the transformer would
|
- inferenceservice-ornith.yaml
|
||||||
# rewrite metadata.namespace on anything cross-namespace added later.
|
- inferenceservice-verifier.yaml
|
||||||
|
- inferenceservice-embeddings.yaml
|
||||||
|
- inferenceservice-reranker.yaml
|
||||||
|
- networkpolicy.yaml
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Namespace
|
||||||
|
metadata:
|
||||||
|
name: llm-serving
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# Default-deny ingress for the serving pods.
|
||||||
|
#
|
||||||
|
# This is a real compensating control, not hygiene: vLLM is pinned to v0.11.0
|
||||||
|
# (forced — last release with Volta kernels), which sits below the patch line on
|
||||||
|
# several advisories that will never be backported to that branch, incl.
|
||||||
|
# CVE-2026-54234 (remote DoS) and GHSA-7m6h-x95x-82q5 (cross-user data leak).
|
||||||
|
# Those are all remote/unauthenticated attack surface, so keeping the engines
|
||||||
|
# reachable only from opted-in in-cluster clients is what keeps exposure low.
|
||||||
|
#
|
||||||
|
# Consumers opt in with label `llm-client: "true"`. Do NOT expose these via
|
||||||
|
# Ingress.
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: llm-serving-default-deny
|
||||||
|
spec:
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
policyTypes:
|
||||||
|
- Ingress
|
||||||
|
ingress:
|
||||||
|
- from:
|
||||||
|
# Any pod, any namespace, that explicitly opts in as an LLM client.
|
||||||
|
- namespaceSelector: {}
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
llm-client: "true"
|
||||||
|
# Sibling engines (harness may chain calls between them).
|
||||||
|
- podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/part-of: llm-serving
|
||||||
|
ports:
|
||||||
|
- protocol: TCP
|
||||||
|
port: 8080
|
||||||
|
- from:
|
||||||
|
# Prometheus scraping /metrics.
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: monitoring
|
||||||
|
ports:
|
||||||
|
- protocol: TCP
|
||||||
|
port: 8080
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
apiVersion: serving.kserve.io/v1beta1
|
|
||||||
kind: InferenceService
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
serving.kserve.io/deploymentMode: RawDeployment
|
|
||||||
# The konghq.com/{connect,read,write}-timeout annotations that used to live
|
|
||||||
# here went with Kong (retired 2026-08-19). They existed because Kong read
|
|
||||||
# its upstream timeouts off the Kubernetes Service, and its 60s default cut
|
|
||||||
# off the first request after any pod restart — a restart flushes VRAM and
|
|
||||||
# reloading ornith:35b takes longer than that. OLLAMA_KEEP_ALIVE=-1 hid the
|
|
||||||
# problem in steady state.
|
|
||||||
#
|
|
||||||
# The equivalent budget now belongs to the Go gateway's per-route timeout
|
|
||||||
# config in rock/homelab-frontend, not to an annotation on this object.
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: llm-ornith
|
|
||||||
app.kubernetes.io/part-of: llm-serving
|
|
||||||
name: ornith
|
|
||||||
namespace: llm-serving
|
|
||||||
spec:
|
|
||||||
predictor:
|
|
||||||
containers:
|
|
||||||
- command:
|
|
||||||
- /bin/sh
|
|
||||||
- -c
|
|
||||||
- 'set -e
|
|
||||||
|
|
||||||
ollama serve &
|
|
||||||
|
|
||||||
SERVE_PID=$!
|
|
||||||
|
|
||||||
until ollama list >/dev/null 2>&1; do sleep 2; done
|
|
||||||
|
|
||||||
ollama pull ornith:35b
|
|
||||||
|
|
||||||
ollama pull qwen2.5:3b-instruct
|
|
||||||
|
|
||||||
ollama run ornith:35b "ok" >/dev/null 2>&1 || true
|
|
||||||
|
|
||||||
ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
|
|
||||||
|
|
||||||
wait $SERVE_PID
|
|
||||||
|
|
||||||
'
|
|
||||||
env:
|
|
||||||
- name: OLLAMA_HOST
|
|
||||||
value: 0.0.0.0:8080
|
|
||||||
- name: OLLAMA_MODELS
|
|
||||||
value: /mnt/models/ollama
|
|
||||||
- name: OLLAMA_CONTEXT_LENGTH
|
|
||||||
value: '32768'
|
|
||||||
- name: OLLAMA_KEEP_ALIVE
|
|
||||||
value: '-1'
|
|
||||||
- name: OLLAMA_NUM_PARALLEL
|
|
||||||
value: '1'
|
|
||||||
- name: OLLAMA_MAX_LOADED_MODELS
|
|
||||||
value: '2'
|
|
||||||
image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
|
|
||||||
name: kserve-container
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
protocol: TCP
|
|
||||||
readinessProbe:
|
|
||||||
exec:
|
|
||||||
command:
|
|
||||||
- /bin/sh
|
|
||||||
- -c
|
|
||||||
- ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
|
|
||||||
grep -q qwen2.5
|
|
||||||
periodSeconds: 10
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: '16'
|
|
||||||
memory: 16Gi
|
|
||||||
nvidia.com/gpu: '1'
|
|
||||||
requests:
|
|
||||||
cpu: '8'
|
|
||||||
memory: 8Gi
|
|
||||||
nvidia.com/gpu: '1'
|
|
||||||
startupProbe:
|
|
||||||
exec:
|
|
||||||
command:
|
|
||||||
- /bin/sh
|
|
||||||
- -c
|
|
||||||
- ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
|
|
||||||
grep -q qwen2.5
|
|
||||||
failureThreshold: 120
|
|
||||||
periodSeconds: 15
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /mnt/models
|
|
||||||
name: models
|
|
||||||
deploymentStrategy:
|
|
||||||
type: Recreate
|
|
||||||
# 2 replicas -- each its own GPU, each loading both ornith:35b and
|
|
||||||
# qwen2.5:3b-instruct -- so 2 concurrent implementer-style calls each
|
|
||||||
# get an independent instance instead of contending on one, at the
|
|
||||||
# cost of judge/qwen traffic still sharing whichever replica an
|
|
||||||
# implementer call also lands on.
|
|
||||||
maxReplicas: 2
|
|
||||||
minReplicas: 2
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: worker-1
|
|
||||||
runtimeClassName: nvidia
|
|
||||||
volumes:
|
|
||||||
- name: models
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: llm-models
|
|
||||||
|
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Shared HuggingFace cache for all three engines.
|
||||||
|
#
|
||||||
|
# ReadWriteOnce is correct here despite six pods mounting it: RWO means "one
|
||||||
|
# NODE", and every pod in this app is pinned to worker-1 via nodeSelector, so
|
||||||
|
# they share the volume legally. If a pod is ever allowed onto another node,
|
||||||
|
# this must become RWX first.
|
||||||
|
#
|
||||||
|
# StorageClass is longhorn-llm-local (1 replica, strict-local, disk tag `llm`)
|
||||||
|
# — NOT the default 3-replica class, which could not place this volume at all:
|
||||||
|
# every control-plane disk was already at its over-provisioning ceiling.
|
||||||
|
#
|
||||||
|
# Sizing (measured, not estimated):
|
||||||
|
# reasoning GPTQ INT4 19.3 GB
|
||||||
|
# ornith:35b q4 (ollama) 21.0 GB
|
||||||
|
# verifier Qwen2.5-Math-PRM-7B fp16 15.3 GB
|
||||||
|
# nomic-embed-text-v2-moe (CPU) 1.9 GB
|
||||||
|
# bge-reranker-base (CPU) 1.1 GB
|
||||||
|
# ------------------------------------------
|
||||||
|
# total ~58.6 GB (+ HF temp during pulls)
|
||||||
|
# The two reasoning replicas share ONE on-disk copy; they differ only in which
|
||||||
|
# GPU they load it onto.
|
||||||
|
apiVersion: v1
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
metadata:
|
||||||
|
name: llm-models
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
storageClassName: longhorn-llm-local
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 120Gi
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
apiVersion: serving.kserve.io/v1beta1
|
|
||||||
kind: InferenceService
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
serving.kserve.io/deploymentMode: RawDeployment
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: llm-reasoning
|
|
||||||
app.kubernetes.io/part-of: llm-serving
|
|
||||||
name: reasoning
|
|
||||||
namespace: llm-serving
|
|
||||||
spec:
|
|
||||||
predictor:
|
|
||||||
containers:
|
|
||||||
- args:
|
|
||||||
# bnb-4bit retired: no int4 tensor cores on sm70/V100, dequant-then-
|
|
||||||
# matmul is two slow kernel launches instead of one fused int4 GEMM,
|
|
||||||
# decode crawled at 2.5-10 tok/s regardless of TP/PP. Switched to
|
|
||||||
# JunHowie/Qwen3-32B-GPTQ-Int4 -- same dense Qwen3-32B weights, same
|
|
||||||
# hermes/qwen3 parser stack (no narration-bug risk, same as before),
|
|
||||||
# only the quant format changes. Plain (non-Marlin) GPTQ kernel is
|
|
||||||
# confirmed Volta-compatible; Marlin needs sm80+ and vLLM would try
|
|
||||||
# to auto-upgrade to it, so --quantization is pinned explicitly to
|
|
||||||
# `gptq` to force the plain kernel. Verified checkpoint size: 19.34GB
|
|
||||||
# (summed from the real safetensors index, not bits-per-param math).
|
|
||||||
# max-model-len=131072 is Qwen3-32B's real ceiling (config.json YaRN:
|
|
||||||
# factor=4.0, original_max_position_embeddings=32768) -- 200k was
|
|
||||||
# asked for but exceeds this architecturally regardless of VRAM.
|
|
||||||
# KV cache math: 256KB/token total (64 layers, 8 KV heads, 128
|
|
||||||
# head_dim, fp16), PP=2 splits both weights and KV load ~evenly, so
|
|
||||||
# each GPU carries ~9.67GB weights + ~128KB/token KV. At
|
|
||||||
# gpu-memory-utilization=0.90 (28.8GB/GPU usable), that leaves
|
|
||||||
# ~19.1GB/GPU for KV cache -> ~156k tokens/GPU capacity, comfortably
|
|
||||||
# above the 131072 target with room to spare -- the old
|
|
||||||
# OffloadingConnector CPU-DRAM spillover (tuned for the previous
|
|
||||||
# model's much smaller 16384 context) is no longer needed and is
|
|
||||||
# dropped. Staying on PP=2 and vLLM 0.11.0 (no version bump needed,
|
|
||||||
# this checkpoint only requires vllm>=0.9.2) -- plain GPTQ has no
|
|
||||||
# TP>1 restriction unlike bnb, so tensor-parallel-size=2 is worth
|
|
||||||
# trying later, but not risking a parallelism-strategy change in the
|
|
||||||
# same rollout as the quant+context-length change.
|
|
||||||
# This GPTQ requant's own config.json ships max_position_embeddings=
|
|
||||||
# 40960 and rope_scaling=None -- confirmed directly (curl'd the raw
|
|
||||||
# config.json), the base Qwen3-32B repo's YaRN block did NOT carry
|
|
||||||
# over during quantization. Re-applying it explicitly here restores
|
|
||||||
# the same math the base model documents (32768 * 4.0 = 131072);
|
|
||||||
# without this, --max-model-len=131072 fails ModelConfig validation
|
|
||||||
# against the checkpoint's own (unscaled) 40960 ceiling.
|
|
||||||
- --model=JunHowie/Qwen3-32B-GPTQ-Int4
|
|
||||||
- --served-model-name=reasoning
|
|
||||||
- --quantization=gptq
|
|
||||||
- --dtype=float16
|
|
||||||
- --kv-cache-dtype=auto
|
|
||||||
- --rope-scaling={"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}
|
|
||||||
- --tensor-parallel-size=1
|
|
||||||
- --pipeline-parallel-size=2
|
|
||||||
- --max-model-len=131072
|
|
||||||
- --gpu-memory-utilization=0.90
|
|
||||||
- --max-num-seqs=4
|
|
||||||
- --enable-chunked-prefill
|
|
||||||
- --enable-prefix-caching
|
|
||||||
# qwen3 is vLLM's dedicated reasoning parser for this family's <think>
|
|
||||||
# blocks.
|
|
||||||
- --reasoning-parser=qwen3
|
|
||||||
# hermes is the documented tool-call parser for general (non-Coder)
|
|
||||||
# Qwen3 models -- native chat template support, not narrated text.
|
|
||||||
- --enable-auto-tool-choice
|
|
||||||
- --tool-call-parser=hermes
|
|
||||||
- --host=0.0.0.0
|
|
||||||
- --port=8080
|
|
||||||
env:
|
|
||||||
- name: VLLM_USE_FLASHINFER_SAMPLER
|
|
||||||
value: '0'
|
|
||||||
- name: VLLM_ATTENTION_BACKEND
|
|
||||||
value: TRITON_ATTN
|
|
||||||
- name: HF_HOME
|
|
||||||
value: /mnt/models
|
|
||||||
image: vllm/vllm-openai:v0.11.0@sha256:014a95f21c9edf6abe0aea6b07353f96baa4ec291c427bb1176dc7c93a85845c
|
|
||||||
name: kserve-container
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
protocol: TCP
|
|
||||||
readinessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 10
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: '16'
|
|
||||||
memory: 36Gi
|
|
||||||
nvidia.com/gpu: '2'
|
|
||||||
requests:
|
|
||||||
cpu: '8'
|
|
||||||
memory: 12Gi
|
|
||||||
nvidia.com/gpu: '2'
|
|
||||||
startupProbe:
|
|
||||||
failureThreshold: 80
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 15
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /mnt/models
|
|
||||||
name: models
|
|
||||||
- mountPath: /dev/shm
|
|
||||||
name: shm
|
|
||||||
deploymentStrategy:
|
|
||||||
type: Recreate
|
|
||||||
maxReplicas: 1
|
|
||||||
minReplicas: 1
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: worker-1
|
|
||||||
runtimeClassName: nvidia
|
|
||||||
volumes:
|
|
||||||
- name: models
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: llm-models
|
|
||||||
- emptyDir:
|
|
||||||
medium: Memory
|
|
||||||
sizeLimit: 2Gi
|
|
||||||
name: shm
|
|
||||||
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
apiVersion: serving.kserve.io/v1beta1
|
|
||||||
kind: InferenceService
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
serving.kserve.io/deploymentMode: RawDeployment
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: llm-reranker
|
|
||||||
app.kubernetes.io/part-of: llm-serving
|
|
||||||
name: reranker
|
|
||||||
namespace: llm-serving
|
|
||||||
spec:
|
|
||||||
predictor:
|
|
||||||
containers:
|
|
||||||
- args:
|
|
||||||
- --model-id=BAAI/bge-reranker-base
|
|
||||||
- --port=8080
|
|
||||||
- --hostname=0.0.0.0
|
|
||||||
- --auto-truncate
|
|
||||||
env:
|
|
||||||
- name: HUGGINGFACE_HUB_CACHE
|
|
||||||
value: /mnt/models
|
|
||||||
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2@sha256:4d632b76bd14cb57044a1ffb0ad48ab0ba4939e705a9a615ccc740658575c26e
|
|
||||||
name: kserve-container
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
protocol: TCP
|
|
||||||
readinessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 10
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: '16'
|
|
||||||
memory: 8Gi
|
|
||||||
requests:
|
|
||||||
cpu: '8'
|
|
||||||
memory: 4Gi
|
|
||||||
startupProbe:
|
|
||||||
failureThreshold: 60
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 8080
|
|
||||||
periodSeconds: 10
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /mnt/models
|
|
||||||
name: models
|
|
||||||
maxReplicas: 1
|
|
||||||
minReplicas: 1
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: worker-1
|
|
||||||
volumes:
|
|
||||||
- name: models
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: llm-models
|
|
||||||
|
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# Dedicated StorageClass for model weights on worker-1's local NVMe.
|
||||||
|
#
|
||||||
|
# Why not the default `longhorn` class (3 replicas, network-attached):
|
||||||
|
#
|
||||||
|
# 1. numberOfReplicas: 1 — model weights are re-downloadable from HuggingFace.
|
||||||
|
# Replicating them 3x buys nothing; losing a replica costs a re-pull, not
|
||||||
|
# data. The repo's "never delete a PVC without replicas/backups" rule exists
|
||||||
|
# for irreplaceable data, which this is not.
|
||||||
|
#
|
||||||
|
# 2. dataLocality: strict-local — keeps the single replica on the SAME node as
|
||||||
|
# the pod. All engines are pinned to worker-1, so weights are read from its
|
||||||
|
# local 751GB NVMe instead of over the network from a control-plane node.
|
||||||
|
# Removes ~60GB of network reads on every cold start.
|
||||||
|
#
|
||||||
|
# 3. diskSelector: llm — restricts this class to disks tagged `llm`, i.e. only
|
||||||
|
# worker-1's disk. Equally important, worker-1's disk carries that tag so
|
||||||
|
# UNTAGGED volumes (any ordinary cluster PVC) will not land on it. Before
|
||||||
|
# tagging, worker-1 had been silently hosting a replica of cicd/runner-dind,
|
||||||
|
# consuming GPU-node storage for general cluster workloads.
|
||||||
|
#
|
||||||
|
# The default 3-replica class also physically could not place this volume: all
|
||||||
|
# three control-plane disks were already at their over-provisioning ceiling
|
||||||
|
# (storage-over-provisioning-percentage=100, 30% reserved), so a 120Gi x3
|
||||||
|
# request failed with ReplicaSchedulingFailure on every node.
|
||||||
|
apiVersion: storage.k8s.io/v1
|
||||||
|
kind: StorageClass
|
||||||
|
metadata:
|
||||||
|
name: longhorn-llm-local
|
||||||
|
provisioner: driver.longhorn.io
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
volumeBindingMode: Immediate
|
||||||
|
parameters:
|
||||||
|
numberOfReplicas: "1"
|
||||||
|
dataLocality: "strict-local"
|
||||||
|
diskSelector: "llm"
|
||||||
|
staleReplicaTimeout: "30"
|
||||||
|
fsType: "ext4"
|
||||||
@@ -13,7 +13,6 @@ spec:
|
|||||||
labels:
|
labels:
|
||||||
app: management-service
|
app: management-service
|
||||||
spec:
|
spec:
|
||||||
serviceAccountName: kmsvc
|
|
||||||
topologySpreadConstraints:
|
topologySpreadConstraints:
|
||||||
- maxSkew: 1
|
- maxSkew: 1
|
||||||
topologyKey: kubernetes.io/hostname
|
topologyKey: kubernetes.io/hostname
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ namespace: sqs
|
|||||||
replicaCount: 3
|
replicaCount: 3
|
||||||
|
|
||||||
image:
|
image:
|
||||||
repository: forgejo.riotpiao.com/rock/kmsvc-manage
|
repository: ghcr.io/riotpiaole/kmsvc-management-service
|
||||||
tag: latest
|
tag: latest
|
||||||
pullPolicy: Always
|
pullPolicy: Always
|
||||||
|
|
||||||
|
|||||||
@@ -10,13 +10,13 @@ metadata:
|
|||||||
name: queue-operator
|
name: queue-operator
|
||||||
rules:
|
rules:
|
||||||
- apiGroups: ["kmsvc.io"]
|
- apiGroups: ["kmsvc.io"]
|
||||||
resources: ["queues"]
|
resources: ["queues", "temporalworkers"]
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
verbs: ["get", "list", "watch", "update", "patch"]
|
||||||
- apiGroups: ["kmsvc.io"]
|
- apiGroups: ["kmsvc.io"]
|
||||||
resources: ["queues/status"]
|
resources: ["queues/status", "temporalworkers/status"]
|
||||||
verbs: ["get", "update", "patch"]
|
verbs: ["get", "update", "patch"]
|
||||||
- apiGroups: ["kmsvc.io"]
|
- apiGroups: ["kmsvc.io"]
|
||||||
resources: ["queues/finalizers"]
|
resources: ["queues/finalizers", "temporalworkers/finalizers"]
|
||||||
verbs: ["update"]
|
verbs: ["update"]
|
||||||
- apiGroups: ["coordination.k8s.io"]
|
- apiGroups: ["coordination.k8s.io"]
|
||||||
resources: ["leases"]
|
resources: ["leases"]
|
||||||
@@ -27,6 +27,9 @@ rules:
|
|||||||
- apiGroups: [""]
|
- apiGroups: [""]
|
||||||
resources: ["pods", "nodes"]
|
resources: ["pods", "nodes"]
|
||||||
verbs: ["get"]
|
verbs: ["get"]
|
||||||
|
- apiGroups: ["apps"]
|
||||||
|
resources: ["deployments"]
|
||||||
|
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
||||||
---
|
---
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
kind: ClusterRoleBinding
|
kind: ClusterRoleBinding
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
apiVersion: apiextensions.k8s.io/v1
|
||||||
|
kind: CustomResourceDefinition
|
||||||
|
metadata:
|
||||||
|
name: temporalworkers.kmsvc.io
|
||||||
|
spec:
|
||||||
|
group: kmsvc.io
|
||||||
|
names:
|
||||||
|
kind: TemporalWorker
|
||||||
|
plural: temporalworkers
|
||||||
|
singular: temporalworker
|
||||||
|
scope: Namespaced
|
||||||
|
versions:
|
||||||
|
- name: v1
|
||||||
|
served: true
|
||||||
|
storage: true
|
||||||
|
schema:
|
||||||
|
openAPIV3Schema:
|
||||||
|
type: object
|
||||||
|
required:
|
||||||
|
- spec
|
||||||
|
properties:
|
||||||
|
apiVersion:
|
||||||
|
type: string
|
||||||
|
kind:
|
||||||
|
type: string
|
||||||
|
metadata:
|
||||||
|
type: object
|
||||||
|
spec:
|
||||||
|
type: object
|
||||||
|
description: Temporal worker specification
|
||||||
|
properties:
|
||||||
|
namespace:
|
||||||
|
type: string
|
||||||
|
description: Temporal namespace
|
||||||
|
taskQueue:
|
||||||
|
type: string
|
||||||
|
description: Task queue name
|
||||||
|
workflowTypes:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
description: List of workflow types to execute
|
||||||
|
activityTypes:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
description: List of activity types to execute
|
||||||
|
concurrency:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
description: Worker concurrency level
|
||||||
|
status:
|
||||||
|
type: object
|
||||||
|
description: Temporal worker status
|
||||||
|
properties:
|
||||||
|
ready:
|
||||||
|
type: boolean
|
||||||
|
lastHeartbeat:
|
||||||
|
type: string
|
||||||
|
format: date-time
|
||||||
|
error:
|
||||||
|
type: string
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
namespace: sqs
|
namespace: sqs
|
||||||
|
|
||||||
image:
|
image:
|
||||||
repository: forgejo.riotpiao.com/rock/kmsvc-manage
|
repository: ghcr.io/riotpiaole/kmsvc-management-service
|
||||||
tag: latest
|
tag: latest
|
||||||
pullPolicy: Always
|
pullPolicy: Always
|
||||||
|
|
||||||
|
|||||||
@@ -1,95 +0,0 @@
|
|||||||
# Overrides paperless-ngx's own paperless/adapter.py at the same import path
|
|
||||||
# (mounted via subPath in deployment.yaml) - settings.py hardcodes
|
|
||||||
# SOCIALACCOUNT_ADAPTER = "paperless.adapter.CustomSocialAccountAdapter", so
|
|
||||||
# no Django setting needs to change, just the file content underneath it.
|
|
||||||
#
|
|
||||||
# Stock CustomSocialAccountAdapter.populate_user() is a stub ("kept in case
|
|
||||||
# global default permissions are implemented in the future" - they aren't),
|
|
||||||
# so every OIDC signup lands with zero permissions and 403s on every API
|
|
||||||
# endpoint. This adds the actual mapping: Authentik's "permissions" claim
|
|
||||||
# (via the permissions scope, requested in PAPERLESS_SOCIALACCOUNT_PROVIDERS,
|
|
||||||
# computed server-side from group membership by authentik-provision.py) ->
|
|
||||||
# "paperless:write" or "*" (homelab-admins) grants is_staff+is_superuser,
|
|
||||||
# same convention already used for MinIO's policy claim and Grafana's
|
|
||||||
# role_attribute_path. Checking the permission string rather than a literal
|
|
||||||
# group name decouples "what grants access" from which group happens to
|
|
||||||
# hold it - same pattern applies to every other service's Role/RoleBinding
|
|
||||||
# in k8s/infra/rbac/.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: paperless-adapter
|
|
||||||
data:
|
|
||||||
adapter.py: |
|
|
||||||
from urllib.parse import quote
|
|
||||||
|
|
||||||
from allauth.account.adapter import DefaultAccountAdapter
|
|
||||||
from allauth.core import context
|
|
||||||
from allauth.socialaccount.adapter import DefaultSocialAccountAdapter
|
|
||||||
from django.conf import settings
|
|
||||||
from django.forms import ValidationError
|
|
||||||
from django.urls import reverse
|
|
||||||
|
|
||||||
REQUIRED_PERMISSIONS = {"paperless:write", "*"}
|
|
||||||
|
|
||||||
|
|
||||||
class CustomAccountAdapter(DefaultAccountAdapter):
|
|
||||||
def is_open_for_signup(self, request):
|
|
||||||
allow_signups = super().is_open_for_signup(request)
|
|
||||||
return getattr(settings, "ACCOUNT_ALLOW_SIGNUPS", allow_signups)
|
|
||||||
|
|
||||||
def pre_authenticate(self, request, **credentials):
|
|
||||||
if settings.DISABLE_REGULAR_LOGIN:
|
|
||||||
raise ValidationError("Regular login is disabled")
|
|
||||||
return super().pre_authenticate(request, **credentials)
|
|
||||||
|
|
||||||
def is_safe_url(self, url):
|
|
||||||
from django.utils.http import url_has_allowed_host_and_scheme
|
|
||||||
|
|
||||||
allowed_hosts = {context.request.get_host()} | set(settings.ALLOWED_HOSTS)
|
|
||||||
if "*" in allowed_hosts:
|
|
||||||
allowed_hosts.remove("*")
|
|
||||||
allowed_hosts.add(context.request.get_host())
|
|
||||||
return url_has_allowed_host_and_scheme(url, allowed_hosts=allowed_hosts)
|
|
||||||
return url_has_allowed_host_and_scheme(url, allowed_hosts=allowed_hosts)
|
|
||||||
|
|
||||||
def get_reset_password_from_key_url(self, key):
|
|
||||||
if settings.PAPERLESS_URL is None:
|
|
||||||
return super().get_reset_password_from_key_url(key)
|
|
||||||
path = reverse(
|
|
||||||
"account_reset_password_from_key",
|
|
||||||
kwargs={"uidb36": "UID", "key": "KEY"},
|
|
||||||
)
|
|
||||||
path = path.replace("UID-KEY", quote(key))
|
|
||||||
return settings.PAPERLESS_URL + path
|
|
||||||
|
|
||||||
|
|
||||||
class CustomSocialAccountAdapter(DefaultSocialAccountAdapter):
|
|
||||||
def is_open_for_signup(self, request, sociallogin):
|
|
||||||
allow_signups = super().is_open_for_signup(request, sociallogin)
|
|
||||||
return getattr(settings, "SOCIALACCOUNT_ALLOW_SIGNUPS", allow_signups)
|
|
||||||
|
|
||||||
def get_connect_redirect_url(self, request, socialaccount):
|
|
||||||
return reverse("base")
|
|
||||||
|
|
||||||
def populate_user(self, request, sociallogin, data):
|
|
||||||
user = super().populate_user(request, sociallogin, data)
|
|
||||||
perms = set(sociallogin.account.extra_data.get("permissions") or [])
|
|
||||||
if perms & REQUIRED_PERMISSIONS:
|
|
||||||
user.is_staff = True
|
|
||||||
user.is_superuser = True
|
|
||||||
return user
|
|
||||||
|
|
||||||
def save_user(self, request, sociallogin, form=None):
|
|
||||||
# populate_user() sets the flags on the in-memory user, but
|
|
||||||
# allauth's default save_user() re-derives is_staff from
|
|
||||||
# ACCOUNT_DEFAULT_HTTP_PROTOCOL-independent defaults and can
|
|
||||||
# overwrite them on save - re-apply after super().save_user()
|
|
||||||
# persists the row, matching the permissions check above exactly.
|
|
||||||
user = super().save_user(request, sociallogin, form)
|
|
||||||
perms = set(sociallogin.account.extra_data.get("permissions") or [])
|
|
||||||
if perms & REQUIRED_PERMISSIONS and not (user.is_staff and user.is_superuser):
|
|
||||||
user.is_staff = True
|
|
||||||
user.is_superuser = True
|
|
||||||
user.save(update_fields=["is_staff", "is_superuser"])
|
|
||||||
return user
|
|
||||||
@@ -1,94 +0,0 @@
|
|||||||
# Nightly: pg_dump the paperless DB + mirror the media PVC into the scoped
|
|
||||||
# `paperless` MinIO bucket (see minio-provision-paperless-job.yaml). This is a
|
|
||||||
# BACKUP target, not live storage - paperless-ngx has no native S3 backend, it
|
|
||||||
# only ever reads/writes the local media PVC directly.
|
|
||||||
#
|
|
||||||
# Pinned to talos-cp-3, same as deployment.yaml: media is a ReadWriteOnce
|
|
||||||
# Longhorn volume with a single replica physically on that node's disk -
|
|
||||||
# mounting it read-only here from a different node would conflict with the
|
|
||||||
# live webserver's attachment.
|
|
||||||
apiVersion: batch/v1
|
|
||||||
kind: CronJob
|
|
||||||
metadata:
|
|
||||||
name: paperless-backup
|
|
||||||
spec:
|
|
||||||
schedule: "0 3 * * *" # 03:00 daily, low-traffic window
|
|
||||||
jobTemplate:
|
|
||||||
spec:
|
|
||||||
backoffLimit: 2
|
|
||||||
template:
|
|
||||||
spec:
|
|
||||||
restartPolicy: Never
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: talos-cp-3
|
|
||||||
initContainers:
|
|
||||||
- name: pg-dump
|
|
||||||
image: postgres:16-alpine
|
|
||||||
env:
|
|
||||||
- name: PGHOST
|
|
||||||
value: paperless-db-rw
|
|
||||||
- name: PGDATABASE
|
|
||||||
value: paperless
|
|
||||||
- name: PGUSER
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-db-app
|
|
||||||
key: username
|
|
||||||
- name: PGPASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-db-app
|
|
||||||
key: password
|
|
||||||
command:
|
|
||||||
- sh
|
|
||||||
- -c
|
|
||||||
- pg_dump --format=custom --file=/backup/paperless-db.dump
|
|
||||||
volumeMounts:
|
|
||||||
- name: backup
|
|
||||||
mountPath: /backup
|
|
||||||
containers:
|
|
||||||
- name: mc-mirror
|
|
||||||
image: minio/mc:latest
|
|
||||||
env:
|
|
||||||
- name: ACCESS_KEY
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-minio-creds
|
|
||||||
key: ACCESS_KEY
|
|
||||||
- name: SECRET_KEY
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-minio-creds
|
|
||||||
key: SECRET_KEY
|
|
||||||
- name: BUCKET
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-minio-creds
|
|
||||||
key: BUCKET
|
|
||||||
- name: ENDPOINT
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-minio-creds
|
|
||||||
key: ENDPOINT
|
|
||||||
command:
|
|
||||||
- /bin/sh
|
|
||||||
- -c
|
|
||||||
- |
|
|
||||||
set -e
|
|
||||||
mc alias set b "$ENDPOINT" "$ACCESS_KEY" "$SECRET_KEY"
|
|
||||||
mc cp /backup/paperless-db.dump "b/$BUCKET/db/paperless-db-$(date +%Y%m%d).dump"
|
|
||||||
mc mirror --overwrite /media "b/$BUCKET/media"
|
|
||||||
echo "Backup done."
|
|
||||||
volumeMounts:
|
|
||||||
- name: backup
|
|
||||||
mountPath: /backup
|
|
||||||
- name: media
|
|
||||||
mountPath: /media
|
|
||||||
readOnly: true
|
|
||||||
volumes:
|
|
||||||
- name: backup
|
|
||||||
emptyDir: {}
|
|
||||||
- name: media
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: paperless-media
|
|
||||||
readOnly: true
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: paperless-config
|
|
||||||
data:
|
|
||||||
PAPERLESS_URL: "https://paperless.riotpiao.com"
|
|
||||||
PAPERLESS_TIME_ZONE: "America/Los_Angeles"
|
|
||||||
PAPERLESS_OCR_LANGUAGE: "eng"
|
|
||||||
PAPERLESS_DBHOST: "paperless-db-rw"
|
|
||||||
PAPERLESS_DBNAME: "paperless"
|
|
||||||
PAPERLESS_REDIS: "redis://paperless-redis:6379"
|
|
||||||
# django-allauth generic OIDC provider. The client_id/secret/server_url
|
|
||||||
# bundle itself lives in the paperless-oidc Secret
|
|
||||||
# (SOCIALACCOUNT_PROVIDERS_JSON key, composed by authentik-provision.py) -
|
|
||||||
# env vars can't be split across a ConfigMap + Secret for the same key, so
|
|
||||||
# this whole value is sourced from the Secret in deployment.yaml instead.
|
|
||||||
PAPERLESS_APPS: "allauth.socialaccount.providers.openid_connect"
|
|
||||||
# Authentik already verifies identity via OIDC - a second email-confirmation
|
|
||||||
# step has no SMTP configured to send it anyway, and paperless-ngx doesn't
|
|
||||||
# wire up allauth's confirm-email view, so signup 500s with NoReverseMatch
|
|
||||||
# on 'account_confirm_email' without this.
|
|
||||||
PAPERLESS_ACCOUNT_EMAIL_VERIFICATION: "none"
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
# Single container runs webserver + consumer + scheduler (paperless-ngx's
|
|
||||||
# stock entrypoint does this internally) - no need to split into separate
|
|
||||||
# Deployments. replicas: 1 only: paperless-media is ReadWriteOnce, and the
|
|
||||||
# consumer polling the media dir doesn't benefit from horizontal scaling here.
|
|
||||||
#
|
|
||||||
# Pinned to talos-cp-3: paperless-media's disk physically lives there. Longhorn
|
|
||||||
# RWO volumes can only be attached from one node at a time, and the nightly
|
|
||||||
# backup-cronjob.yaml also mounts this same PVC (read-only) to mirror it into
|
|
||||||
# MinIO - pinning both to the same node avoids a cross-node attach conflict,
|
|
||||||
# and keeps the 3.5Ti read/write path off the network entirely.
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
strategy:
|
|
||||||
type: Recreate # ReadWriteOnce media PVC - avoid two pods fighting over it
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: paperless
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: paperless
|
|
||||||
spec:
|
|
||||||
# Kubernetes injects legacy Docker-links env vars for every Service in
|
|
||||||
# this namespace (<SVC>_SERVICE_HOST, <SVC>_PORT, ...). The Service here
|
|
||||||
# is named "paperless", so that becomes PAPERLESS_PORT=tcp://<ip>:8000 -
|
|
||||||
# paperless-ngx's own entrypoint reads PAPERLESS_PORT for gunicorn's
|
|
||||||
# bind address, collides, and gunicorn crash-loops on "not a valid port
|
|
||||||
# number". Disable the injection instead of renaming the Service.
|
|
||||||
enableServiceLinks: false
|
|
||||||
nodeSelector:
|
|
||||||
kubernetes.io/hostname: talos-cp-3
|
|
||||||
containers:
|
|
||||||
- name: paperless
|
|
||||||
image: ghcr.io/paperless-ngx/paperless-ngx:2.20.15
|
|
||||||
ports:
|
|
||||||
- containerPort: 8000
|
|
||||||
envFrom:
|
|
||||||
- configMapRef:
|
|
||||||
name: paperless-config
|
|
||||||
env:
|
|
||||||
- name: PAPERLESS_DBUSER
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-db-app
|
|
||||||
key: username
|
|
||||||
- name: PAPERLESS_DBPASS
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-db-app
|
|
||||||
key: password
|
|
||||||
- name: PAPERLESS_SECRET_KEY
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_SECRET_KEY
|
|
||||||
- name: PAPERLESS_ADMIN_USER
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_ADMIN_USER
|
|
||||||
- name: PAPERLESS_ADMIN_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_ADMIN_PASSWORD
|
|
||||||
- name: PAPERLESS_SOCIALACCOUNT_PROVIDERS
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-oidc
|
|
||||||
key: SOCIALACCOUNT_PROVIDERS_JSON
|
|
||||||
resources:
|
|
||||||
requests: { cpu: "500m", memory: "1Gi" }
|
|
||||||
limits: { cpu: "2", memory: "4Gi" }
|
|
||||||
volumeMounts:
|
|
||||||
- name: media
|
|
||||||
mountPath: /usr/src/paperless/media
|
|
||||||
- name: data
|
|
||||||
mountPath: /usr/src/paperless/data
|
|
||||||
- name: consume
|
|
||||||
mountPath: /usr/src/paperless/consume
|
|
||||||
# Overrides paperless-ngx's own adapter.py in place - settings.py
|
|
||||||
# hardcodes the import path, so no Django setting changes, just
|
|
||||||
# the file content underneath it (see adapter-configmap.yaml).
|
|
||||||
- name: adapter
|
|
||||||
mountPath: /usr/src/paperless/src/paperless/adapter.py
|
|
||||||
subPath: adapter.py
|
|
||||||
readOnly: true
|
|
||||||
volumes:
|
|
||||||
- name: media
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: paperless-media
|
|
||||||
- name: data
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: paperless-data
|
|
||||||
- name: consume
|
|
||||||
emptyDir: {}
|
|
||||||
- name: adapter
|
|
||||||
configMap:
|
|
||||||
name: paperless-adapter
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
# Direct nginx ingress to the paperless Service - not routed via the Go
|
|
||||||
# api-gateway (api.riotpiao.com), which has no WebSocket upgrade support and
|
|
||||||
# paperless-ngx keeps a long-lived /ws/ connection open for live task status.
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: Ingress
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
annotations:
|
|
||||||
nginx.ingress.kubernetes.io/proxy-body-size: "0" # large scanned PDF uploads
|
|
||||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
|
||||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
|
||||||
spec:
|
|
||||||
ingressClassName: nginx
|
|
||||||
rules:
|
|
||||||
- host: paperless.riotpiao.com
|
|
||||||
http:
|
|
||||||
paths:
|
|
||||||
- path: /
|
|
||||||
pathType: Prefix
|
|
||||||
backend:
|
|
||||||
service:
|
|
||||||
name: paperless
|
|
||||||
port:
|
|
||||||
number: 8000
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
namespace: paperless
|
|
||||||
resources:
|
|
||||||
- pvc.yaml
|
|
||||||
- configmap.yaml
|
|
||||||
- redis.yaml
|
|
||||||
- deployment.yaml
|
|
||||||
- service.yaml
|
|
||||||
- ingress.yaml
|
|
||||||
- backup-cronjob.yaml
|
|
||||||
- adapter-configmap.yaml
|
|
||||||
- rbac.yaml
|
|
||||||
# postgres: paperless-db CNPG Cluster, deployed by k8s/infra/databases (wave 2,
|
|
||||||
# before this app at wave 8) - not duplicated here. Same for the paperless-oidc
|
|
||||||
# and paperless-minio-creds Secrets, written by PostSync provisioning Jobs in
|
|
||||||
# k8s/infra/iam and k8s/infra/minio respectively.
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
# Two volumes, deliberately separate storage classes:
|
|
||||||
#
|
|
||||||
# - media: the actual documents (originals + OCR'd archive PDFs + thumbnails).
|
|
||||||
# Lives on the cp-3 USB HDD, single replica (see
|
|
||||||
# k8s/infra/longhorn/longhorn-paperless-storageclass.yaml). Shares the disk
|
|
||||||
# with Immich's immich-media PVC (k8s/apps/immich/pvc.yaml, 2000Gi) - photo
|
|
||||||
# libraries grow much faster than scanned documents, so paperless gets the
|
|
||||||
# smaller 500Gi share.
|
|
||||||
# - data: the SQLite classification model + search index. Small (low GB),
|
|
||||||
# frequently rewritten, and disposable (rebuilds from the DB + media on
|
|
||||||
# next consume) - stays on the default 3-replica pool instead of the
|
|
||||||
# single-disk HDD.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: paperless-media
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: longhorn-paperless-media
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 500Gi
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: paperless-data
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: longhorn
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 5Gi
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
# Scoped operator access for paperless-admins: restart/config-edit rights on
|
|
||||||
# just this service's own resources, nothing CNPG-managed (paperless-db-*)
|
|
||||||
# or provisioning-managed (paperless-oidc, paperless-minio-creds). Inert
|
|
||||||
# until kube-apiserver's OIDC wiring lands (--oidc-groups-claim=groups,
|
|
||||||
# --oidc-groups-prefix=oidc:) - subject name below assumes that prefix.
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: Role
|
|
||||||
metadata:
|
|
||||||
name: paperless-operator
|
|
||||||
rules:
|
|
||||||
- apiGroups: ["apps"]
|
|
||||||
resources: ["deployments"]
|
|
||||||
resourceNames: ["paperless"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["configmaps"]
|
|
||||||
resourceNames: ["paperless-config"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["secrets"]
|
|
||||||
resourceNames: ["paperless-secrets"]
|
|
||||||
verbs: ["get", "list", "watch", "update", "patch"]
|
|
||||||
---
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: RoleBinding
|
|
||||||
metadata:
|
|
||||||
name: paperless-admins-binding
|
|
||||||
subjects:
|
|
||||||
- kind: Group
|
|
||||||
name: "oidc:paperless-admins"
|
|
||||||
apiGroup: rbac.authorization.k8s.io
|
|
||||||
roleRef:
|
|
||||||
kind: Role
|
|
||||||
name: paperless-operator
|
|
||||||
apiGroup: rbac.authorization.k8s.io
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
# Task queue broker + websocket channel layer for paperless-ngx. No PVC:
|
|
||||||
# queued/scheduled task state is disposable - a lost queue on restart just
|
|
||||||
# means re-triggering consumption, not data loss (documents themselves live
|
|
||||||
# on paperless-media).
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: paperless-redis
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: paperless-redis
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: paperless-redis
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: redis
|
|
||||||
image: redis:7-alpine
|
|
||||||
ports:
|
|
||||||
- containerPort: 6379
|
|
||||||
resources:
|
|
||||||
requests: { cpu: "50m", memory: "64Mi" }
|
|
||||||
limits: { cpu: "250m", memory: "256Mi" }
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: paperless-redis
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: paperless-redis
|
|
||||||
ports:
|
|
||||||
- port: 6379
|
|
||||||
targetPort: 6379
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: paperless
|
|
||||||
ports:
|
|
||||||
- port: 8000
|
|
||||||
targetPort: 8000
|
|
||||||
@@ -1,115 +0,0 @@
|
|||||||
# macOS VM (Docker-OSX) hosting the BlueBubbles server.
|
|
||||||
#
|
|
||||||
# ── Why a VM and not a container ────────────────────────────────────────────
|
|
||||||
# Containers share the host kernel. macOS binaries are Mach-O and need XNU plus
|
|
||||||
# Cocoa/IOKit, which a Linux kernel cannot provide, so no macOS container exists
|
|
||||||
# or can exist. Docker-OSX is QEMU running a macOS guest, packaged in a
|
|
||||||
# container — a VM in a box, not a macOS container.
|
|
||||||
#
|
|
||||||
# ── Why this works on worker-2 ──────────────────────────────────────────────
|
|
||||||
# Verified on the existing hardware: amd64, `vmx` (Intel VT-x) present, and
|
|
||||||
# /dev/kvm exists on Talos nodes (KVM is compiled into Talos' kernel, not a
|
|
||||||
# module). Bare metal, so no nested virtualisation needed.
|
|
||||||
#
|
|
||||||
# ── Read this before relying on it ──────────────────────────────────────────
|
|
||||||
# 1. Setup is INTERACTIVE. First boot runs the macOS installer: connect over
|
|
||||||
# VNC (:5999), erase the disk in Disk Utility, install, create a user, sign
|
|
||||||
# into iMessage, THEN install BlueBubbles inside the guest. This manifest
|
|
||||||
# only provides the machine; it does not provision macOS.
|
|
||||||
# 2. iMessage activation on non-Apple hardware is a coin flip. BlueBubbles'
|
|
||||||
# own guidance: "test sending an iMessage to yourself. If it does not
|
|
||||||
# succeed, it's likely best to restart from the beginning."
|
|
||||||
# 3. Apple's macOS licence permits virtualisation only on Apple hardware. This
|
|
||||||
# is a Hackintosh. Use a throwaway Apple ID, not a primary one.
|
|
||||||
# 4. BlueBubbles labels this path "not for beginners", "no guarantees or
|
|
||||||
# warranty".
|
|
||||||
#
|
|
||||||
# Private API (reactions, typing indicators, edit/unsend) needs SIP disabled
|
|
||||||
# inside the guest and is NOT required for plain send/receive. Skip it.
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: macos-bluebubbles
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: macos-bluebubbles
|
|
||||||
app.kubernetes.io/part-of: sms
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
# Recreate: the qcow2 disk is RWO and a second pod must never attach it
|
|
||||||
# concurrently — two QEMU processes on one image corrupts it.
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: macos-bluebubbles
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: macos-bluebubbles
|
|
||||||
app.kubernetes.io/part-of: sms
|
|
||||||
spec:
|
|
||||||
# Dedicated node. The taint keeps everything else off worker-2; this
|
|
||||||
# toleration is what lets the VM on. Both halves are required.
|
|
||||||
nodeSelector:
|
|
||||||
workload: imessage
|
|
||||||
tolerations:
|
|
||||||
- key: workload
|
|
||||||
operator: Equal
|
|
||||||
value: imessage
|
|
||||||
effect: NoSchedule
|
|
||||||
containers:
|
|
||||||
- name: macos
|
|
||||||
image: sickcodes/docker-osx:latest@sha256:3a3c82c79bc4e73531f819ccdfa4053b3084efd7c1f645678dbf8b4b3a24369c
|
|
||||||
# QEMU needs /dev/kvm; Talos enforces `baseline` cluster-wide, so this
|
|
||||||
# only schedules because the sms namespace is labelled privileged.
|
|
||||||
securityContext:
|
|
||||||
privileged: true
|
|
||||||
env:
|
|
||||||
# Generates a unique serial / board-serial / UUID / MAC and persists
|
|
||||||
# them to bootdisk.qcow2. This synthetic identity is what iMessage
|
|
||||||
# activates against — it must stay stable across restarts, which is
|
|
||||||
# why the PVC matters.
|
|
||||||
- name: GENERATE_UNIQUE
|
|
||||||
value: "true"
|
|
||||||
# Identity is only plausible if it matches a real product line.
|
|
||||||
- name: DEVICE_MODEL
|
|
||||||
value: "iMacPro1,1"
|
|
||||||
- name: RAM
|
|
||||||
value: "12"
|
|
||||||
- name: CORES
|
|
||||||
value: "6"
|
|
||||||
- name: EXTRA
|
|
||||||
# Expose the BlueBubbles server port from the guest to the pod.
|
|
||||||
# Guest :1234 (BlueBubbles default) -> pod :1234.
|
|
||||||
value: "-device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp::1234-:1234"
|
|
||||||
ports:
|
|
||||||
- name: vnc
|
|
||||||
containerPort: 5999
|
|
||||||
protocol: TCP
|
|
||||||
- name: bluebubbles
|
|
||||||
containerPort: 1234
|
|
||||||
protocol: TCP
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: "6"
|
|
||||||
memory: 14Gi
|
|
||||||
limits:
|
|
||||||
cpu: "12"
|
|
||||||
memory: 20Gi
|
|
||||||
volumeMounts:
|
|
||||||
- name: macos-disk
|
|
||||||
mountPath: /home/arch/OSX-KVM/disk
|
|
||||||
- name: kvm
|
|
||||||
mountPath: /dev/kvm
|
|
||||||
# No readiness probe on purpose. The guest takes many minutes to boot,
|
|
||||||
# and until macOS + BlueBubbles are installed BY HAND there is nothing
|
|
||||||
# listening on 1234. A probe here would crash-loop the pod through the
|
|
||||||
# entire interactive install.
|
|
||||||
volumes:
|
|
||||||
- name: macos-disk
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: macos-disk
|
|
||||||
- name: kvm
|
|
||||||
hostPath:
|
|
||||||
path: /dev/kvm
|
|
||||||
type: CharDevice
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
namespace: sms
|
|
||||||
resources:
|
|
||||||
- namespace.yaml
|
|
||||||
- storageclass.yaml
|
|
||||||
- pvc-macos.yaml
|
|
||||||
- deployment-macos.yaml
|
|
||||||
- service.yaml
|
|
||||||
- networkpolicy.yaml
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
# iMessage delivery for the cluster.
|
|
||||||
#
|
|
||||||
# BlueBubbles' server is a macOS Electron app paired with an Objective-C helper
|
|
||||||
# that hooks Messages.app private APIs — it cannot be containerised on Linux,
|
|
||||||
# because containers share the host kernel and macOS needs XNU + Cocoa. The only
|
|
||||||
# way to run it on Talos is a full macOS VM under QEMU/KVM (Docker-OSX), which
|
|
||||||
# needs a privileged pod with /dev/kvm.
|
|
||||||
#
|
|
||||||
# Hence privileged PodSecurity: the cluster default from the Talos controlplane
|
|
||||||
# is `enforce: baseline`, which forbids privileged containers and host devices.
|
|
||||||
# Scope is limited to this namespace.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: sms
|
|
||||||
labels:
|
|
||||||
pod-security.kubernetes.io/enforce: privileged
|
|
||||||
pod-security.kubernetes.io/audit: privileged
|
|
||||||
pod-security.kubernetes.io/warn: privileged
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
# Default-deny. This namespace runs a privileged QEMU VM signed into an Apple
|
|
||||||
# ID and exposes an unauthenticated VNC console; nothing should reach it except
|
|
||||||
# opted-in clients.
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: NetworkPolicy
|
|
||||||
metadata:
|
|
||||||
name: sms-default-deny
|
|
||||||
spec:
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/part-of: sms
|
|
||||||
policyTypes:
|
|
||||||
- Ingress
|
|
||||||
ingress:
|
|
||||||
- from:
|
|
||||||
- namespaceSelector: {}
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
sms-client: "true"
|
|
||||||
ports:
|
|
||||||
- protocol: TCP
|
|
||||||
port: 1234
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
# Persistent macOS disk image + generated hardware identity (bootdisk.qcow2).
|
|
||||||
#
|
|
||||||
# This volume is NOT disposable: it holds the VM's serial number, board serial,
|
|
||||||
# UUID and MAC, which together form the identity iMessage was activated against.
|
|
||||||
# Losing it means re-running activation, which is the least reliable step of the
|
|
||||||
# whole setup.
|
|
||||||
#
|
|
||||||
# Docker-OSX documents 128GB minimum for the guest image; 200Gi leaves room for
|
|
||||||
# the installer, the base system, and qcow2 growth.
|
|
||||||
#
|
|
||||||
# ⚠️ Single replica (see storageclass.yaml — capacity and IO both rule out 3).
|
|
||||||
# Losing worker-2's disk therefore means losing the activated identity and
|
|
||||||
# redoing iMessage activation. Once the guest is installed and activated, take
|
|
||||||
# a Longhorn snapshot/backup of this volume; that is the only redundancy here.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: macos-disk
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: longhorn-imessage-local
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 200Gi
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
# VNC is how you drive the interactive macOS install. Deliberately ClusterIP —
|
|
||||||
# it is an unauthenticated console onto a machine holding a live Apple ID
|
|
||||||
# session. Reach it with `kubectl port-forward`, never an Ingress.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: macos-vnc
|
|
||||||
spec:
|
|
||||||
type: ClusterIP
|
|
||||||
selector:
|
|
||||||
app.kubernetes.io/name: macos-bluebubbles
|
|
||||||
ports:
|
|
||||||
- name: vnc
|
|
||||||
port: 5999
|
|
||||||
targetPort: vnc
|
|
||||||
---
|
|
||||||
# The BlueBubbles REST API, once installed inside the guest. This is the stable
|
|
||||||
# name cluster services use, so callers never depend on the pod IP or on whether
|
|
||||||
# the backend is this VM or a real Mac mini later.
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: bluebubbles
|
|
||||||
spec:
|
|
||||||
type: ClusterIP
|
|
||||||
selector:
|
|
||||||
app.kubernetes.io/name: macos-bluebubbles
|
|
||||||
ports:
|
|
||||||
- name: http
|
|
||||||
port: 1234
|
|
||||||
targetPort: bluebubbles
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
# Dedicated StorageClass for the macOS VM disk.
|
|
||||||
#
|
|
||||||
# The default `longhorn` class does not work here, for two independent reasons:
|
|
||||||
#
|
|
||||||
# 1. Replica count. Default is 3, and Longhorn schedules against
|
|
||||||
# storageMaximum - storageScheduled with over-provisioning at 100%. Free
|
|
||||||
# space is cp-1 146Gi / cp-2 8Gi / cp-3 146Gi / worker-1 292Gi, so a 200Gi
|
|
||||||
# volume has only one node that can hold even a single replica — a 3-replica
|
|
||||||
# volume fails outright with ReplicaSchedulingFailure.
|
|
||||||
# 2. Binding mode. `Immediate` provisions the volume the moment the PVC is
|
|
||||||
# created, before any pod is scheduled. Combined with strict-local that
|
|
||||||
# pins the data to an arbitrary node, not the one the VM runs on.
|
|
||||||
#
|
|
||||||
# So: one replica, kept local to the VM, bound only once the pod has a node.
|
|
||||||
apiVersion: storage.k8s.io/v1
|
|
||||||
kind: StorageClass
|
|
||||||
metadata:
|
|
||||||
name: longhorn-imessage-local
|
|
||||||
provisioner: driver.longhorn.io
|
|
||||||
allowVolumeExpansion: true
|
|
||||||
reclaimPolicy: Delete
|
|
||||||
# The pod is pinned to worker-2 by nodeSelector; wait for it to be scheduled so
|
|
||||||
# the replica is placed on that node and not somewhere else.
|
|
||||||
volumeBindingMode: WaitForFirstConsumer
|
|
||||||
parameters:
|
|
||||||
# A qcow2 backing a live VM is latency-sensitive and rewritten constantly.
|
|
||||||
# Serving it over the network from another node's disk would be the single
|
|
||||||
# worst thing for guest responsiveness, so force it local.
|
|
||||||
numberOfReplicas: "1"
|
|
||||||
dataLocality: "strict-local"
|
|
||||||
staleReplicaTimeout: "30"
|
|
||||||
fsType: "ext4"
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
apiVersion: kmsvc.io/v1
|
|
||||||
kind: TemporalWorker
|
|
||||||
metadata:
|
|
||||||
name: worker-production
|
|
||||||
namespace: temporal
|
|
||||||
spec:
|
|
||||||
namespace: production
|
|
||||||
taskQueue: worker-production
|
|
||||||
concurrency: 10
|
|
||||||
workflowTypes:
|
|
||||||
- HelloWorldWorkflow
|
|
||||||
- GreeterWorkflow
|
|
||||||
- ProcessOrderWorkflow
|
|
||||||
activityTypes:
|
|
||||||
- GreetActivity
|
|
||||||
- ValidateOrderActivity
|
|
||||||
- ProcessPaymentActivity
|
|
||||||
- NotifyCustomerActivity
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
# Wave -1 — AppProject definitions (must sync before any Application that references them).
|
|
||||||
# Syncs k8s/argocd/projects/ which was previously applied by hand.
|
|
||||||
# Enabled by Stage 1 (A2).
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: argocd-projects
|
|
||||||
namespace: argocd
|
|
||||||
finalizers:
|
|
||||||
- resources-finalizer.argocd.argoproj.io
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "-1"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
revisionHistoryLimit: 3
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/argocd/projects
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Wave 0 — Nginx Ingress Controller
|
||||||
|
# Foundational infrastructure required for all ingress resources and ArgoCD UI access.
|
||||||
|
# Must be wave 0 to ensure LoadBalancer IP is available before other apps deploy.
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: ingress-nginx
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "0"
|
||||||
|
finalizers:
|
||||||
|
- resources-finalizer.argocd.argoproj.io
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
revisionHistoryLimit: 3
|
||||||
|
sources:
|
||||||
|
- repoURL: https://kubernetes.github.io/ingress-nginx
|
||||||
|
chart: ingress-nginx
|
||||||
|
targetRevision: "4.15.1"
|
||||||
|
helm:
|
||||||
|
valueFiles:
|
||||||
|
- $values/k8s/bootstrap/ingress/nginx-values.yaml
|
||||||
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
ref: values
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: ingress-nginx
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
- ServerSideApply=true
|
||||||
|
retry:
|
||||||
|
limit: 3
|
||||||
|
backoff:
|
||||||
|
duration: 10s
|
||||||
|
factor: 2
|
||||||
|
maxDuration: 3m
|
||||||
@@ -17,7 +17,7 @@ spec:
|
|||||||
syncOptions:
|
syncOptions:
|
||||||
- CreateNamespace=true
|
- CreateNamespace=true
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
# ksops decrypts every *.enc.yaml here at kustomize-build time (repo-server
|
# ksops decrypts every *.enc.yaml here at kustomize-build time (repo-server
|
||||||
# runs `kustomize build --enable-alpha-plugins --enable-exec`). Replaces the
|
# runs `kustomize build --enable-alpha-plugins --enable-exec`). Replaces the
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/bootstrap/cert-manager/cert-manager-values.yaml
|
- $values/k8s/bootstrap/cert-manager/cert-manager-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -93,7 +93,7 @@ spec:
|
|||||||
project: homelab
|
project: homelab
|
||||||
revisionHistoryLimit: 3
|
revisionHistoryLimit: 3
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
# A real kustomization.yaml (resources: the 3 issuer/CA files) renders these
|
# A real kustomization.yaml (resources: the 3 issuer/CA files) renders these
|
||||||
# deterministically. The previous directory.include with bare filenames
|
# deterministically. The previous directory.include with bare filenames
|
||||||
@@ -127,7 +127,7 @@ spec:
|
|||||||
project: homelab
|
project: homelab
|
||||||
revisionHistoryLimit: 3
|
revisionHistoryLimit: 3
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/bootstrap/ingress
|
path: k8s/bootstrap/ingress
|
||||||
destination:
|
destination:
|
||||||
@@ -138,75 +138,3 @@ spec:
|
|||||||
selfHeal: true
|
selfHeal: true
|
||||||
syncOptions:
|
syncOptions:
|
||||||
- CreateNamespace=true
|
- CreateNamespace=true
|
||||||
---
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: cluster-maintenance
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "0"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/cluster-maintenance
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: kube-system
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
---
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: kyverno
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "0"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://kyverno.github.io/kyverno/
|
|
||||||
chart: kyverno
|
|
||||||
targetRevision: "1.14.0"
|
|
||||||
helm:
|
|
||||||
valueFiles:
|
|
||||||
- $values/k8s/bootstrap/kyverno/kyverno-values.yaml
|
|
||||||
sources:
|
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
ref: values
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: kyverno
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
---
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: kyverno-policies
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "0"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/bootstrap/kyverno
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: kyverno
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
|
|||||||
@@ -1,33 +0,0 @@
|
|||||||
# ArgoCD Image Updater - auto-updates Application images from registry
|
|
||||||
# Watches forgejo.riotpiao.com for new image tags and updates Applications
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: argocd-image-updater
|
|
||||||
namespace: argocd
|
|
||||||
finalizers:
|
|
||||||
- resources-finalizer.argocd.argoproj.io
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "1"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
revisionHistoryLimit: 3
|
|
||||||
sources:
|
|
||||||
- repoURL: https://argoproj.github.io/argo-helm
|
|
||||||
chart: argocd-image-updater
|
|
||||||
targetRevision: "0.11.2"
|
|
||||||
helm:
|
|
||||||
valueFiles:
|
|
||||||
- $values/k8s/infra/argocd-image-updater/values.yaml
|
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
ref: values
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: argocd
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=false
|
|
||||||
@@ -1,8 +1,26 @@
|
|||||||
# Wave 0 — networking substrate is Talos-owned (terraform inlineManifests), not
|
# Wave 0 — networking policies layered on the Cilium CNI + CoreDNS that the
|
||||||
# ArgoCD:
|
# cluster bootstrap already installed (substrate). These are raw manifests only.
|
||||||
# - CoreDNS Corefile + hostname rewrites -> terraform/files/coredns/Corefile
|
apiVersion: argoproj.io/v1alpha1
|
||||||
# - Cilium LB-IPAM pool + L2 announcement -> terraform/files/cilium/*.yaml
|
kind: Application
|
||||||
# Both were previously ArgoCD apps here whose empty `resources: []`
|
metadata:
|
||||||
# kustomizations never actually applied them (live objects came from manual
|
name: cilium-policy
|
||||||
# kubectl). Managing them from ArgoCD too would let two reconcilers fight. This
|
namespace: argocd
|
||||||
# file intentionally defines no Applications now.
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "0"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
path: k8s/bootstrap/cilium
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: kube-system
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
# coredns-config Application removed: CoreDNS (incl. homelab hostname rewrites)
|
||||||
|
# is owned by Talos via an inlineManifest (terraform/files/coredns/Corefile).
|
||||||
|
# Managing the coredns ConfigMap from ArgoCD too would let the two reconcilers
|
||||||
|
# fight and revert the rewrites.
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/minio/minio-operator-values.yaml
|
- $values/k8s/infra/minio/minio-operator-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -41,7 +41,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/minio
|
path: k8s/infra/minio
|
||||||
destination:
|
destination:
|
||||||
@@ -66,20 +66,12 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/longhorn
|
path: k8s/infra/longhorn
|
||||||
destination:
|
destination:
|
||||||
server: https://kubernetes.default.svc
|
server: https://kubernetes.default.svc
|
||||||
namespace: longhorn-system
|
namespace: longhorn-system
|
||||||
# Longhorn writes disk state back into its own Node CRs — the disk key it
|
|
||||||
# generates, storageReserved, diskType, evictionRequested. Git declares only
|
|
||||||
# allowScheduling; without this the controller's writes read as drift forever.
|
|
||||||
ignoreDifferences:
|
|
||||||
- group: longhorn.io
|
|
||||||
kind: Node
|
|
||||||
jsonPointers:
|
|
||||||
- /spec/disks
|
|
||||||
syncPolicy:
|
syncPolicy:
|
||||||
automated:
|
automated:
|
||||||
prune: true
|
prune: true
|
||||||
@@ -102,7 +94,7 @@ spec:
|
|||||||
skipCrds: true
|
skipCrds: true
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/monitoring/prometheus-values.yaml
|
- $values/k8s/infra/monitoring/prometheus-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -152,7 +144,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/monitoring/crds
|
path: k8s/infra/monitoring/crds
|
||||||
destination:
|
destination:
|
||||||
@@ -183,7 +175,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/monitoring
|
path: k8s/infra/monitoring
|
||||||
destination:
|
destination:
|
||||||
@@ -213,7 +205,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/monitoring/blackbox-exporter-values.yaml
|
- $values/k8s/infra/monitoring/blackbox-exporter-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -223,29 +215,3 @@ spec:
|
|||||||
automated:
|
automated:
|
||||||
prune: true
|
prune: true
|
||||||
selfHeal: true
|
selfHeal: true
|
||||||
---
|
|
||||||
# Distributed tracing: Tempo + OpenTelemetry Collector.
|
|
||||||
# Receives traces from instrumented services, stores in local volume (72h retention).
|
|
||||||
# Grafana datasource auto-configured, service graph + latency dashboards included.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: tracing
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "1"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/tracing
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: tracing
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/logging/loki-values.yaml
|
- $values/k8s/infra/logging/loki-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -53,7 +53,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/logging/grafana-values.yaml
|
- $values/k8s/infra/logging/grafana-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -87,7 +87,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/logging/promtail-values.yaml
|
- $values/k8s/infra/logging/promtail-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/iam/vault-values.yaml
|
- $values/k8s/infra/iam/vault-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -46,7 +46,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/infra/iam/authentik-values.yaml
|
- $values/k8s/infra/iam/authentik-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -68,7 +68,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/iam
|
path: k8s/infra/iam
|
||||||
destination:
|
destination:
|
||||||
@@ -79,83 +79,18 @@ spec:
|
|||||||
prune: true
|
prune: true
|
||||||
selfHeal: true
|
selfHeal: true
|
||||||
---
|
---
|
||||||
# Forgejo itself. Was a bootstrap Helm release (phase 3) until it was brought
|
# Forgejo runner (local chart). Forgejo itself is Phase 0 (bootstrap).
|
||||||
# under Argo, because values changes there were inert — a proxy-body-size fix
|
|
||||||
# sat committed while the live Ingress kept nginx's 1m default and rejected
|
|
||||||
# every OCI push with 413.
|
|
||||||
#
|
|
||||||
# Wave 3: after databases (wave 2) — Forgejo needs CNPG and Redis up first.
|
|
||||||
#
|
|
||||||
# Retiring the Helm release: Argo adopts the existing objects on first sync.
|
|
||||||
# Delete the release secrets afterwards so helm stops claiming ownership:
|
|
||||||
# kubectl -n cicd delete secret -l owner=helm,name=forgejo
|
|
||||||
#
|
|
||||||
# automated sync is deliberately absent. This chart owns the Forgejo PVC and
|
|
||||||
# the git forge itself; the first sync is manual so its diff can be read before
|
|
||||||
# anything is applied. Turn on automated+selfHeal once that diff is clean.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
apiVersion: argoproj.io/v1alpha1
|
||||||
kind: Application
|
kind: Application
|
||||||
metadata:
|
metadata:
|
||||||
name: forgejo
|
name: forgejo-runner
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "3"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
sources:
|
|
||||||
- repoURL: https://dl.gitea.com/charts/
|
|
||||||
chart: gitea
|
|
||||||
targetRevision: 12.7.0
|
|
||||||
helm:
|
|
||||||
valueFiles:
|
|
||||||
- $values/k8s/bootstrap/phase3-forgejo/forgejo-values.yaml
|
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
ref: values
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: cicd
|
|
||||||
# Reloader injects a STAKATER_* env var carrying a hash of the config Secret,
|
|
||||||
# so the pod rolls when that Secret changes. The chart does not render it, so
|
|
||||||
# Argo would strip it on every sync — and with selfHeal on, Argo and Reloader
|
|
||||||
# would fight over the field and Recreate the forge each round.
|
|
||||||
ignoreDifferences:
|
|
||||||
- group: apps
|
|
||||||
kind: Deployment
|
|
||||||
name: forgejo-gitea
|
|
||||||
jqPathExpressions:
|
|
||||||
- '.spec.template.spec.containers[].env[] | select(.name | startswith("STAKATER_"))'
|
|
||||||
syncPolicy:
|
|
||||||
syncOptions:
|
|
||||||
# Adopt the objects the bootstrap Helm release already created rather
|
|
||||||
# than failing on "already exists".
|
|
||||||
- ServerSideApply=true
|
|
||||||
---
|
|
||||||
# Forgejo runners (local chart, one instance per language), replacing the
|
|
||||||
# single generic "docker"-labeled runner. Each instance is a full standalone
|
|
||||||
# Deployment with its own dind sidecar, own PVCs (registration + layer
|
|
||||||
# cache) and own registered label -- there is no shared generic runner
|
|
||||||
# anymore, so each instance also builds and pushes images for the repos it
|
|
||||||
# serves (the chart's ConfigMap/NetworkPolicy fixes for that -- valid_volumes,
|
|
||||||
# network: host, egress to ingress-nginx -- apply identically to all three).
|
|
||||||
#
|
|
||||||
# `values.yaml` is the chart's default and doubles as the golang instance's
|
|
||||||
# config; node and rust layer a small values-<lang>.yaml override on top for
|
|
||||||
# just runner.name/runner.labels. All three share one runner-token Secret
|
|
||||||
# (Forgejo registration tokens are reusable across multiple runners, unlike
|
|
||||||
# GitHub's one-time tokens) -- if that assumption is ever wrong, registration
|
|
||||||
# will fail loudly in the register initContainer's logs, not silently.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: forgejo-runner-golang
|
|
||||||
namespace: argocd
|
namespace: argocd
|
||||||
annotations:
|
annotations:
|
||||||
argocd.argoproj.io/sync-wave: "3"
|
argocd.argoproj.io/sync-wave: "3"
|
||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/forgejo-runner
|
path: k8s/infra/forgejo-runner
|
||||||
destination:
|
destination:
|
||||||
@@ -165,51 +100,3 @@ spec:
|
|||||||
automated:
|
automated:
|
||||||
prune: true
|
prune: true
|
||||||
selfHeal: true
|
selfHeal: true
|
||||||
---
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: forgejo-runner-node
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "3"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/forgejo-runner
|
|
||||||
helm:
|
|
||||||
valueFiles:
|
|
||||||
- values-node.yaml
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: cicd
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
---
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: forgejo-runner-rust
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "3"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/forgejo-runner
|
|
||||||
helm:
|
|
||||||
valueFiles:
|
|
||||||
- values-rust.yaml
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: cicd
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/infra/databases
|
path: k8s/infra/databases
|
||||||
destination:
|
destination:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Wave 5 — Kafka (Strimzi operator + cluster CR), Redis infrastructure.
|
# Wave 5 — Kafka (Strimzi operator + cluster CR), Redis, and the SQS-like
|
||||||
# Strimzi/Redis are public Helm charts; kafka-cluster is a local chart.
|
# queue services. Strimzi/Redis are public Helm charts; kafka-cluster/queue-crd/
|
||||||
# queue-crd and management-service are managed by kmsvc-root (kmsvc-manage.git).
|
# management-service are local charts (rendered from their own Chart.yaml).
|
||||||
apiVersion: argoproj.io/v1alpha1
|
apiVersion: argoproj.io/v1alpha1
|
||||||
kind: Application
|
kind: Application
|
||||||
metadata:
|
metadata:
|
||||||
@@ -68,7 +68,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/apps/messaging/kafka-cluster
|
path: k8s/apps/messaging/kafka-cluster
|
||||||
destination:
|
destination:
|
||||||
@@ -78,5 +78,45 @@ spec:
|
|||||||
automated:
|
automated:
|
||||||
prune: true
|
prune: true
|
||||||
selfHeal: true
|
selfHeal: true
|
||||||
# queue-crd and management-service moved to kmsvc-manage.git repo
|
---
|
||||||
# Managed by kmsvc-root Application
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: queue-crd
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "6"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
path: k8s/apps/messaging/queue-crd
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: sqs
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: management-service
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "7"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
path: k8s/apps/messaging/management-service
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: sqs
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
|||||||
@@ -1,68 +0,0 @@
|
|||||||
# Wave 7 — api-gw, the cluster's API gateway (namespace `api`).
|
|
||||||
#
|
|
||||||
# Replaces Kong OSS 3.4.1, removed 2026-08-19. Kong existed to route
|
|
||||||
# `api.riotpiao.com`, but Kong OSS cannot dispatch on a request body, so the
|
|
||||||
# LLM surface had to be expressed as one path per model
|
|
||||||
# (`/v1/reasoning/chat/completions`, `/v1/ornith/...`, `/v1/qwen/...`) with a
|
|
||||||
# `request-transformer` plugin forcing the body's `model` field on each. The Go
|
|
||||||
# gateway reads the body and picks the upstream, so a single canonical
|
|
||||||
# `POST /v1/chat/completions` covers every model. See
|
|
||||||
# docs/adr/ADR-0001-retire-kong-for-go-gateway.md in the frontend repo.
|
|
||||||
#
|
|
||||||
# UPDATED 2026-08-22: Tracks main branch of homelab-frontend (auto-syncs on each push).
|
|
||||||
# Image built on every main commit with tag <commit-sha>.
|
|
||||||
# ArgoCD auto-pulls the latest image (live reconciliation ~3min).
|
|
||||||
#
|
|
||||||
# Two sources:
|
|
||||||
# 1. rock/homelab-frontend on the in-cluster Forgejo (prod branch) — the gateway's own
|
|
||||||
# kustomization (Deployment, Service, ConfigMap, RBAC, NetworkPolicy). It
|
|
||||||
# sets `namespace: api` itself, so no transformer is needed here. The
|
|
||||||
# Forgejo host must stay listed in the `homelab` AppProject sourceRepos or
|
|
||||||
# this Application is rejected with "is not permitted in project".
|
|
||||||
# 2. k8s/apps/api in this repo — the nginx edge Ingress for
|
|
||||||
# api.riotpiao.com, inherited from the retired `kong` Application. It
|
|
||||||
# cannot move to k8s/bootstrap/ingress/ingress.yaml because that syncs in
|
|
||||||
# wave 1, before namespace `api` exists.
|
|
||||||
#
|
|
||||||
# No resources-finalizer: deleting this Application leaves the workload running
|
|
||||||
# rather than cascading the delete.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: api-gw
|
|
||||||
namespace: argocd
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: api-gateway
|
|
||||||
app.kubernetes.io/component: gateway
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "7"
|
|
||||||
# ArgoCD Image Updater - auto-update on new image push
|
|
||||||
argocd-image-updater.argoproj.io/image-list: gw=forgejo.riotpiao.com/rock/api-gateway
|
|
||||||
argocd-image-updater.argoproj.io/gw.update-strategy: newest-build
|
|
||||||
argocd-image-updater.argoproj.io/gw.allow-tags: regexp:^[0-9a-f]{7}$
|
|
||||||
argocd-image-updater.argoproj.io/write-back-method: argocd
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
revisionHistoryLimit: 3
|
|
||||||
sources:
|
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab-frontend.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s
|
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/api
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: api
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
retry:
|
|
||||||
limit: 5
|
|
||||||
backoff:
|
|
||||||
duration: 5s
|
|
||||||
factor: 2
|
|
||||||
maxDuration: 3m
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
# Wave 6 — the model servers behind api.riotpiao.com (namespace `llm-serving`).
|
|
||||||
#
|
|
||||||
# Syncs before wave 7 (api-gw), so the predictor Services exist before the
|
|
||||||
# gateway that routes to them. KServe itself is part of the substrate; this Application
|
|
||||||
# owns only the InferenceServices.
|
|
||||||
#
|
|
||||||
# Adopted from live state on 2026-08-15. These five had been `kubectl apply`-ed
|
|
||||||
# by hand — no ArgoCD ownership, present in no repo — so every change to them
|
|
||||||
# was drift by definition. Each manifest was exported from the cluster and
|
|
||||||
# verified with `kubectl diff` returning empty before this file existed; the
|
|
||||||
# first sync therefore adopted them without restarting anything.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: llm-serving
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "6"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
revisionHistoryLimit: 3
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/llm-serving
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: llm-serving
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
# `prune: false` here, unlike every other Application in this repo, and it
|
|
||||||
# is not an oversight.
|
|
||||||
#
|
|
||||||
# ArgoCD tracks ownership with the `argocd.argoproj.io/instance` label
|
|
||||||
# (argocd-cm `application.instanceLabelKey`). KServe copies an
|
|
||||||
# InferenceService's labels onto the Deployment and Service it generates —
|
|
||||||
# visible today as `app.kubernetes.io/name` and `part-of` on
|
|
||||||
# `ornith-predictor`. So once ArgoCD labels an InferenceService, KServe
|
|
||||||
# propagates that tracking label to children that are not in git, ArgoCD
|
|
||||||
# reads them as extraneous, prunes them, and KServe recreates them. That
|
|
||||||
# loop churns GPU pods.
|
|
||||||
#
|
|
||||||
# Deleting an InferenceService therefore means deleting the file AND
|
|
||||||
# removing the object, rather than relying on prune.
|
|
||||||
prune: false
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
# KServe CRDs are large; server-side apply avoids the
|
|
||||||
# "metadata.annotations: Too long" failure client-side apply hits, and is
|
|
||||||
# the correct mode for adopting objects an operator also writes to.
|
|
||||||
- ServerSideApply=true
|
|
||||||
retry:
|
|
||||||
limit: 3
|
|
||||||
backoff:
|
|
||||||
duration: 10s
|
|
||||||
factor: 2
|
|
||||||
maxDuration: 3m
|
|
||||||
@@ -19,12 +19,9 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/apps/temporal/temporal-values.yaml
|
- $values/k8s/apps/temporal/temporal-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/temporal
|
|
||||||
destination:
|
destination:
|
||||||
server: https://kubernetes.default.svc
|
server: https://kubernetes.default.svc
|
||||||
namespace: temporal
|
namespace: temporal
|
||||||
@@ -51,7 +48,7 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/apps/portainer/portainer-values.yaml
|
- $values/k8s/apps/portainer/portainer-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
destination:
|
destination:
|
||||||
@@ -74,7 +71,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/apps/cloudflared
|
path: k8s/apps/cloudflared
|
||||||
destination:
|
destination:
|
||||||
@@ -87,121 +84,6 @@ spec:
|
|||||||
syncOptions:
|
syncOptions:
|
||||||
- CreateNamespace=true
|
- CreateNamespace=true
|
||||||
---
|
---
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: agent-pod
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "8"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/agent-pod
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: agent-pod
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
---
|
|
||||||
# iMessage/SMS delivery. Raw manifests: a privileged macOS VM (Docker-OSX)
|
|
||||||
# running the BlueBubbles server, plus its dedicated local StorageClass.
|
|
||||||
#
|
|
||||||
# Pinned to worker-2 via nodeSelector `workload: imessage` + a matching
|
|
||||||
# toleration for that node's taint. Until worker-2 is provisioned this app
|
|
||||||
# syncs everything except the pod, which stays Pending — that is expected.
|
|
||||||
#
|
|
||||||
# No CreateNamespace: namespace.yaml carries `pod-security: privileged`, which
|
|
||||||
# the VM needs (/dev/kvm, privileged), and an ArgoCD-created namespace would
|
|
||||||
# not have those labels.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: sms
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "8"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/sms
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: sms
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
---
|
|
||||||
# Document management. Raw manifests (no Helm): postgres is the dedicated
|
|
||||||
# paperless-db CNPG cluster in k8s/infra/databases (wave 2), redis is
|
|
||||||
# in-cluster only (no PVC), media lives on the cp-3 USB HDD (see
|
|
||||||
# k8s/infra/longhorn/longhorn-paperless-storageclass.yaml). OIDC via
|
|
||||||
# Authentik provisioned by k8s/infra/iam's PostSync job; MinIO backup bucket
|
|
||||||
# creds provisioned by k8s/infra/minio's PostSync job.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "8"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/paperless
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: paperless
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
---
|
|
||||||
# Photo/video backup. Self-contained (unlike paperless, its CNPG Postgres
|
|
||||||
# lives here too, not in k8s/infra/databases) - CreateNamespace=true creates
|
|
||||||
# the namespace before any manifest in this Application applies, including
|
|
||||||
# the Cluster CR, so no separate wave-2 pre-creation step is needed. Postgres
|
|
||||||
# is pg18 (not this repo's usual 16.2) because CNPG's official pgvector
|
|
||||||
# extension image only publishes pg18 builds - see k8s/apps/immich/db.yaml.
|
|
||||||
# media PVC shares the cp-3 HDD 2TB/2TB with paperless-media. OIDC via
|
|
||||||
# Authentik provisioned by k8s/infra/iam's PostSync job (immich entry in
|
|
||||||
# SERVICES + immich_role scope mapping for admin-via-claim).
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: immich
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "8"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/apps/immich
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: immich
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
---
|
|
||||||
# Consolidated: homarr + homarr-patches → homarr
|
# Consolidated: homarr + homarr-patches → homarr
|
||||||
# Helm chart + values + PostSync hook patch (fix-probes-job.yaml)
|
# Helm chart + values + PostSync hook patch (fix-probes-job.yaml)
|
||||||
apiVersion: argoproj.io/v1alpha1
|
apiVersion: argoproj.io/v1alpha1
|
||||||
@@ -220,10 +102,10 @@ spec:
|
|||||||
helm:
|
helm:
|
||||||
valueFiles:
|
valueFiles:
|
||||||
- $values/k8s/apps/homarr/homarr-values.yaml
|
- $values/k8s/apps/homarr/homarr-values.yaml
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
ref: values
|
ref: values
|
||||||
- repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
- repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/apps/homarr # PostSync hook: fix-probes-job.yaml
|
path: k8s/apps/homarr # PostSync hook: fix-probes-job.yaml
|
||||||
destination:
|
destination:
|
||||||
@@ -235,59 +117,3 @@ spec:
|
|||||||
selfHeal: true
|
selfHeal: true
|
||||||
syncOptions:
|
syncOptions:
|
||||||
- CreateNamespace=true
|
- CreateNamespace=true
|
||||||
---
|
|
||||||
# Portfolio site at riotpiao.com - static Next.js site from rock/riotpiao.com repo.
|
|
||||||
# Points directly to infra/portfolio/base (bypassing repo's own argocd-apps.yaml
|
|
||||||
# which has wrong URLs). Image built by Forgejo Actions on rock/portfolio repo.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: portfolio
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "8"
|
|
||||||
# ArgoCD Image Updater - auto-update on new image push
|
|
||||||
argocd-image-updater.argoproj.io/image-list: app=forgejo.riotpiao.com/rock/portfolio
|
|
||||||
argocd-image-updater.argoproj.io/app.update-strategy: newest-build
|
|
||||||
argocd-image-updater.argoproj.io/app.allow-tags: regexp:^[0-9a-f]{7}$
|
|
||||||
argocd-image-updater.argoproj.io/write-back-method: argocd
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/riotpiao.com.git
|
|
||||||
targetRevision: main
|
|
||||||
path: infra/portfolio/base
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: portfolio
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
---
|
|
||||||
# Wave 9 - per-service scoped RBAC (Role/RoleBinding), deliberately last so
|
|
||||||
# every target namespace above already exists. Inert until kube-apiserver
|
|
||||||
# gets --oidc-groups-claim=groups wired up (separate, not-yet-applied
|
|
||||||
# terraform/talosctl change) - these grant nothing until then.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: rbac
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "9"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/rbac
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: default
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
|
|||||||
@@ -1,28 +0,0 @@
|
|||||||
# kmsvc-manage bootstrap — manages itself and its supporting services
|
|
||||||
# (Strimzi/Kafka, Redis, queue-operator, message-plane server) from
|
|
||||||
# the kmsvc-manage repo's own k8s/argocd/ structure on the main branch.
|
|
||||||
# Image built on every main commit, auto-deployed to sqs namespace.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: kmsvc-root
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "6"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/kmsvc-manage.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/argocd/apps
|
|
||||||
directory:
|
|
||||||
recurse: false
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: sqs
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
# Wave 9-11 — GPU serving stack on worker-1 (4x Tesla V100 32GB).
|
||||||
|
#
|
||||||
|
# Ordering matters: device plugin must expose nvidia.com/gpu and the KServe CRDs
|
||||||
|
# must exist before any InferenceService is applied, hence three waves.
|
||||||
|
#
|
||||||
|
# NOTE the chart versions below are v-PREFIXED (v0.15.2, not 0.15.2) — that is
|
||||||
|
# how the KServe OCI tags are published; the unprefixed form 404s.
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: gpu-runtimeclass
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
# Wave 8: must precede the device plugin, whose DaemonSet sets
|
||||||
|
# runtimeClassName: nvidia and is rejected at admission if the
|
||||||
|
# RuntimeClass does not exist yet.
|
||||||
|
argocd.argoproj.io/sync-wave: "8"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
path: k8s/apps/gpu-runtimeclass
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
# Cluster-scoped resource; namespace is only the app's default context.
|
||||||
|
namespace: gpu-system
|
||||||
|
syncPolicy:
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: nvidia-device-plugin
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "9"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: https://nvidia.github.io/k8s-device-plugin
|
||||||
|
chart: nvidia-device-plugin
|
||||||
|
targetRevision: "0.19.3"
|
||||||
|
helm:
|
||||||
|
values: |
|
||||||
|
# Driver + container toolkit are supplied by Talos system extensions
|
||||||
|
# baked into the installer image (nonfree-kmod-nvidia-lts /
|
||||||
|
# nvidia-container-toolkit-lts). This chart ONLY advertises the GPUs to
|
||||||
|
# the kubelet — it does not and must not install drivers.
|
||||||
|
runtimeClassName: nvidia
|
||||||
|
nodeSelector:
|
||||||
|
nvidia.com/gpu: "true"
|
||||||
|
# Drop the chart's default nodeAffinity. It requires one of three
|
||||||
|
# Node-Feature-Discovery labels (feature.node.kubernetes.io/pci-10de.present,
|
||||||
|
# .../cpu-model.vendor_id=NVIDIA, or nvidia.com/gpu.present). NFD is not
|
||||||
|
# installed and Talos sets nvidia.com/gpu (no ".present" suffix), so the
|
||||||
|
# affinity matches zero nodes and the DaemonSet silently reports
|
||||||
|
# desiredNumberScheduled=0 with no events. nodeSelector is the constraint.
|
||||||
|
affinity: null
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: gpu-system
|
||||||
|
syncPolicy:
|
||||||
|
# Manual sync for first bring-up: watch device-plugin -> KServe -> models
|
||||||
|
# come up in order, and avoid auto-deploying while worker-1 is cordoned.
|
||||||
|
# Switch to `automated: {prune: true, selfHeal: true}` once proven.
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: kserve-crd
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "9"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: oci://ghcr.io/kserve/charts
|
||||||
|
chart: kserve-crd
|
||||||
|
targetRevision: v0.15.2
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: kserve
|
||||||
|
syncPolicy:
|
||||||
|
# Manual sync for first bring-up: watch device-plugin -> KServe -> models
|
||||||
|
# come up in order, and avoid auto-deploying while worker-1 is cordoned.
|
||||||
|
# Switch to `automated: {prune: true, selfHeal: true}` once proven.
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
# InferenceService CRD exceeds the annotation size limit for
|
||||||
|
# client-side apply.
|
||||||
|
- ServerSideApply=true
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: kserve
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "10"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: oci://ghcr.io/kserve/charts
|
||||||
|
chart: kserve
|
||||||
|
targetRevision: v0.15.2
|
||||||
|
helm:
|
||||||
|
values: |
|
||||||
|
kserve:
|
||||||
|
controller:
|
||||||
|
# RawDeployment => plain Deployments/Services, no Knative, no Istio.
|
||||||
|
# v0.18 renames this mode "Standard"; do not bump without checking.
|
||||||
|
deploymentMode: RawDeployment
|
||||||
|
gateway:
|
||||||
|
ingressGateway:
|
||||||
|
# Route through the existing ingress-nginx, not Gateway API.
|
||||||
|
# NOTE the nesting: it is gateway.ingressGateway.enableGatewayApi,
|
||||||
|
# not gateway.enableGatewayApi — Helm silently ignores the wrong
|
||||||
|
# key rather than erroring.
|
||||||
|
enableGatewayApi: false
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: kserve
|
||||||
|
syncPolicy:
|
||||||
|
# Manual sync for first bring-up: watch device-plugin -> KServe -> models
|
||||||
|
# come up in order, and avoid auto-deploying while worker-1 is cordoned.
|
||||||
|
# Switch to `automated: {prune: true, selfHeal: true}` once proven.
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
- ServerSideApply=true
|
||||||
|
---
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: llm-serving
|
||||||
|
namespace: argocd
|
||||||
|
annotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "11"
|
||||||
|
spec:
|
||||||
|
project: homelab
|
||||||
|
source:
|
||||||
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
targetRevision: main
|
||||||
|
path: k8s/apps/llm-serving
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: llm-serving
|
||||||
|
syncPolicy:
|
||||||
|
# Manual sync for first bring-up: watch device-plugin -> KServe -> models
|
||||||
|
# come up in order, and avoid auto-deploying while worker-1 is cordoned.
|
||||||
|
# Switch to `automated: {prune: true, selfHeal: true}` once proven.
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
# Poimen project collection — manages poimen-memory, poimen-workflows, and poiman
|
|
||||||
# Each repo tracks its own main branch (no prod branch). Poiman is the primary
|
|
||||||
# orchestrator with k8s/argocd/ containing the AppProject and deployment structure.
|
|
||||||
#
|
|
||||||
# CI: All three repos trigger on main branch pushes (no image builds yet).
|
|
||||||
# Future: Add build workflows for poiman once container runtime needs are clear.
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: poimen-root
|
|
||||||
namespace: argocd
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: poimen
|
|
||||||
app.kubernetes.io/component: orchestrator
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "7"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/poimen.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/argocd
|
|
||||||
directory:
|
|
||||||
recurse: false
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: poimen
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
retry:
|
|
||||||
limit: 5
|
|
||||||
backoff:
|
|
||||||
duration: 5s
|
|
||||||
factor: 2
|
|
||||||
maxDuration: 3m
|
|
||||||
@@ -11,19 +11,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
description: Homelab GitOps — single-repo, in-cluster destinations only
|
description: Homelab GitOps — single-repo, in-cluster destinations only
|
||||||
sourceRepos:
|
sourceRepos:
|
||||||
- https://github.com/Riotpiaole/riotpiao.homelab.com.git
|
- git@github.com:Riotpiaole/riotpiao.homelab.com.git
|
||||||
# Poimen services (GitHub)
|
|
||||||
- https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
- https://github.com/Riotpiaole/Poimen-workflows.git
|
|
||||||
- https://github.com/Riotpiaole/poimen*.git
|
|
||||||
# In-cluster Forgejo repos — explicit allowlist (no wildcard)
|
|
||||||
- https://forgejo.riotpiao.com/rock/homelab.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/homelab-frontend.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/kmsvc-manage.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/poimen.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/poimen-memory.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/poimen-workflows.git
|
|
||||||
- https://forgejo.riotpiao.com/rock/riotpiao.com.git
|
|
||||||
# Public Helm chart repos referenced by k8s/argocd/apps/* and bootstrap/*
|
# Public Helm chart repos referenced by k8s/argocd/apps/* and bootstrap/*
|
||||||
- https://cloudnative-pg.github.io/charts
|
- https://cloudnative-pg.github.io/charts
|
||||||
- https://dl.gitea.com/charts/
|
- https://dl.gitea.com/charts/
|
||||||
@@ -42,8 +30,6 @@ spec:
|
|||||||
- https://charts.jetstack.io
|
- https://charts.jetstack.io
|
||||||
- https://kubernetes.github.io/ingress-nginx
|
- https://kubernetes.github.io/ingress-nginx
|
||||||
- https://stakater.github.io/stakater-charts
|
- https://stakater.github.io/stakater-charts
|
||||||
# ArgoCD ecosystem charts
|
|
||||||
- https://argoproj.github.io/argo-helm
|
|
||||||
destinations:
|
destinations:
|
||||||
- server: https://kubernetes.default.svc
|
- server: https://kubernetes.default.svc
|
||||||
namespace: "*"
|
namespace: "*"
|
||||||
|
|||||||
@@ -1,10 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
metadata:
|
|
||||||
name: argocd-projects
|
|
||||||
|
|
||||||
# AppProject definitions for ArgoCD. Synced by wave -1 Application
|
|
||||||
# (k8s/argocd/apps/-1-projects.yaml) so they exist before any Application
|
|
||||||
# references them. Enabled by Stage 1 (A2).
|
|
||||||
resources:
|
|
||||||
- homelab-project.yaml
|
|
||||||
@@ -12,7 +12,7 @@ metadata:
|
|||||||
spec:
|
spec:
|
||||||
project: homelab
|
project: homelab
|
||||||
source:
|
source:
|
||||||
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
|
repoURL: [email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
targetRevision: main
|
targetRevision: main
|
||||||
path: k8s/argocd/apps
|
path: k8s/argocd/apps
|
||||||
directory:
|
directory:
|
||||||
@@ -26,4 +26,3 @@ spec:
|
|||||||
selfHeal: true
|
selfHeal: true
|
||||||
syncOptions:
|
syncOptions:
|
||||||
- CreateNamespace=true
|
- CreateNamespace=true
|
||||||
- ServerSideApply=true
|
|
||||||
|
|||||||
@@ -1,23 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:C3U=,iv:J6yvL9HYwzrR4AidMrxmTQZAA1AqtAO/nn9AQnS40JY=,tag:WkqaErH6Xfrpd68+4QrfrQ==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:BJsIWY38,iv:/8AGCpKvtaKoi+iuQNkJbKCSo/jSKi6WRs0f1tJj6d0=,tag:6/Rja8mrZu25kSqg0IzjYQ==,type:str]
|
|
||||||
metadata:
|
|
||||||
name: ENC[AES256_GCM,data:lTfngGBtsNA+,iv:shwcjVXeWhFRE+IMYlW6ffPyY4JqVzw5yySqcfyU+4I=,tag:qMmCemQcBjLwubaTnnKqTg==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:/VJI5GUu+jSX,iv:XCgFTLvytlYl8K09JyhGSsmVaTVCyztY5Xvw4TSkfEg=,tag:3eWHcCiedFHLvcKC8WVnUg==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:aibi62c0,iv:MJQRYJ27pTgtaRUKoJI2nb1qKZP47c4Ma+PvjIrCiE0=,tag:jB6mYfAPSnWnZUnY+rC+zQ==,type:str]
|
|
||||||
stringData:
|
|
||||||
models.json: ENC[AES256_GCM,data:SZNtmDYpM+ivOATbvUcOGylj5i7RIu6sps3tp63jQcPwrEjM9bNVCcIEdfC8owq3JU2yT3mUMdC5c/ViNuCMhIilLSGf35Da/uTnOTe4e3URNQ2r1LXTBYr5R4ESt6uZrI7llzgc68C7j+k1JEldbPeoyAef0sHHSIGdbQHaVzd/+j6RSnmDcpwNw5fopyeBkZivdnKgY1vW0bz/IEgMUHRpsIeZkENSJvp63pUwsmv7NZ0qaFWk2rgzZiZszIBFYP8K/AcDAaTid8T7F6Ro9A3ClRdHW9K8Zary2TSXMrZJ+Mo9Rcqzcrw5LBiCRCn+lSmILovZfohPHUN6atKbB5YAewo7XBVlJJa5Wl+V7sq3FHZH7+rbLSFQEPovxt8z8SW2k1W/ifz+0HYWLRrtJLADZ3iV6eDKCrkSnDiCvi9pAczwD4jrbQYDdnVb9Djgp+8qhUyjEZT3C7ONs/ZaTAawX434KqsUN4O5LSEKgNlITnvHgmoxMd6eR1Xc48Gb3CBmI9ChZNpXvtgzZX0t9EuKCR0HBoBg9IPz6vADdbEVRGNLVVT1ucDZU7Sp+vMXnaM9ZIw3jDgWnmX+L4sBBpe2H2mC71muYVpF0swHt7+H0o7Nc7vxNsNIqnqWJjTQt/w4joENvbKB4dBs5lNRCeXuWuwyh6/f+MZbz4ZfqtPfvoEsWqiXglGs0eQQttDlXS36R3lsF1brFKbHdO4j7rS7YEeZdKgJIALqp3I+w/v9hAyLh+bu+R6tqXz3QoKWQkDl10Jjip0I/GIkkw9W8ZDBgsy9YcVf9J0ZOa+wm0GZXhBYWyfrE55FiMkCL1IYP2GABb5HdAbMyccIB1dkx9/TAGWeYxsMPmjCvBghf3LawZNa3nDNMnfH7+MxQFfvlHlGJBWbgO7M4V/rOLQu0RhoMIplb1ZyucHSVMWDNt71kteKR7Fme3VdFES+nIoDi8usfUYSQraW99XMB9IsIkPMCz9nFAUoNrXhd/ADkyRmaXe+gb8UohY++7zBy6YllTICHzZFVvucU1YniZLR9i5lVBv16Gcpe0PxnLuCGFwQi+RdgIPglyXTEFh5Woo7ahrr0H/9knrG7p2cqpfJs777ZSWexQL7ncN40kk73fZyAQZFbYw6vhkRGdTHb5Dq2hrLC4FSPev33lWN9v1V4uMo/wa0CGXkxbjTcHeb2NkzfXWFM0+OlqSYTev8fnbZoBGvh/N/UcmdadAG20wPoD4O7kRaZMqZe3vf5s5AsirZtKGMB+FlpBw8f3BQ8GGV9X9bBcFgq42hhcXzGrelqdCwPKFqPpz5ItEasDAdU48dHBFnAn3OWYiKrDn4/uc0XURcMKg5JR0cwJEaAWd6ZPvPy58qYQwWoFA+MVqk1/fql2GjUxW10hTPNMePLq70FYEl5pRqcWAzS2v/9LSM53E/UD/tlyjm61llqPhirDt7QhYmnEawJeQxFzzrgizAdx4jp4SGCpa3envp1oPySJdEyTJkedf/zYxXHLKZ/sc9uuhM/Bl103SeXeuAQrXrFE/kJazdUBgEstOPiDiLSWq9FjMxgMIr9SMcChYCFm7U4irsjWCdboVWKb6iMZH96AFnm2AwP+QvXUSN6dNoRAGY9E7yeiebuAnvCMGQIZ4rxYJ+xsQvRDpIwZZhs1oDAKsvMaozp5o8zZXwU+UCoRfiHVQ+LmAKhDWW0PfLhMVRSsBA9tHDnigwwRNth+A+117IBwnOgpMf/vS2xOTPszAyso24+4np4etTRMWBZLai/sScnkWflGlEwTD2lb38IeCVfTH4Vp1wPfViI3bRGvpj8je6aBg5buRaUO/M2NPXdK9eroM3OQewEtv1QTtuGy7ihsXukeoll8GvOor+1m8qhLeK4/pMH2kEMvE7fFjiG7cVxUYczN1RPl3353BIh65BdSqi8ZbxezHbPuPnoST3teoBdiwVoHcrz80by76n5kIiLKrunuzDVugq3H7NgE8xpnIpttCADiYHL2jtx7Tg7ft3rSMEjep6OSThUWyKyJUUCgyoENmrR7zyWhk50Z1YCkFW/1hOma+chdPY012aGz6nhDBxOOYrI54YlD61jKsheE51IYWKdQXAKfsp0UCGR3Q+t69QuCm+lclZgatuoBWO+tt/WKekO3T4Y1HMx1aoDk/kAKYZwfZXPyCPQ/cIK0GUpS3H8Fa8uDIpaf5KqTj3DTV9qCcrtvrmrf4Qj8YiyN+TBp/wtLiMLtofjI+apTxBqQSGss7oB/JsuH1wacA5EGoKwO8tZAuMWLM6wuPjaSgVRlBezbGsYqKUAiDZkcXLOHco6AbEptMbK91t5Mgw0V8ActHRg+3LbuHo5P/tWwzdF+1Ht7J7EvJIzvz4lfuVGyUA4fh6uVHHeeJG2OrNtjrqr2zq5iSEECJ4tc7hNDnGR7wqHc3TPEi0YIicUNU5701v3wcKgxqiPrrfRS1ONR7SLqiTHijpdFBOZyFrh/bd0216w/6vUk8uZoq/3u0PjUkVG10ofb3farN4GUfOwvQBgSRkRjIe5K4DH1ZKnE0SQtV9UonzWPqU6jWGepPsOlTMiBkdoTeUhaeomAygbJjRuBsKzGGTDDaZwBpVvB0rBVIV5xge1NKVCSTnOyGYi2gYkmEcWgSdgWk+OOI=,iv:9PDXlSUYz+vl2EzVcwMHZTgyamXLNZU7C+XdcAEi9j4=,tag:bRkH77im+qHjDewMW6PsbA==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB4aENRQllpaWh3bnRJYjEw
|
|
||||||
eTFkM250eXBSVUFrQWJXSzJUTGgrNmJqL1NVCklRT0tRWTlRU0duYWNzVFExQllS
|
|
||||||
bWo0TndLVWl2VGllb00zR1c5ZERpWmsKLS0tIDh6Uk5hUFV4bmRkK0lHWWN6L2Jk
|
|
||||||
dlU3cXJlVFZYYi8yMm5kVUJveU91OGcKyin8Tr7OkCocRxf1dzWl/QsC4l2XW4dn
|
|
||||||
g/it6hJQx1P+23STw9pDZVPqEj4fOdqjnNoRqVCkM8wH4SXJfwrnlA==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-08-20T07:26:38Z"
|
|
||||||
mac: ENC[AES256_GCM,data:YIN51aoCGuFrJwxJIGbCf7vY/+S4uHR4HwQ6un084hMKnInh1uK96FxWrQBlcheqBDfoaXHXXmHAd14LVhrVEsj3R1cFPpxiQpqjwt+d+mON4YBeOrC/VcStAU8joKcLsc8H0PF41PGMcBmflQVDX30D/+am60hZ7FMnGavZqgA=,iv:N6tv/3gh9BJvZdWXAQqTwaceR5nLbiA4YwOz01uwbtg=,tag:6VG9ZkEVpE9xLCNy0HLYcw==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:seY=,iv:1q2bg1QBNDDQPYNu1S08MZ+Ix8WLthxObsrqTMJEVhY=,tag:OMh/stn4vVScOEAyviuurw==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:QwqgPMrm,iv:tv6rUcezEgQigvHVw05Mevt6BHDD5AARuoUmQpry3AU=,tag:m9RLNezJsLYGSRWnozfkoQ==,type:str]
|
|
||||||
metadata:
|
|
||||||
name: ENC[AES256_GCM,data:ZJtohnEDN2NBFrEhhvGuw4s=,iv:SU5LuA3Lqjh+q7nVxeVVtzeXiOxxVserNno5c7b47lM=,tag:YLprEk7oVjVsIDVANWDPrQ==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:LieEC+YWJzq+,iv:U00VXejzlpgOYS7Yxix/MnEW0PEvmwOuOXMUi2CWIps=,tag:QvFYZIexWhyvBYTruGMifQ==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:U41EbP+H,iv:CKDqec0jfVtbOtEyqQpV9CHJNCwhneYi0wqK8QU2vnY=,tag:CSJ04pkPslevQrpSjhXNHw==,type:str]
|
|
||||||
stringData:
|
|
||||||
id_forgejo: ENC[AES256_GCM,data:dgwezImUFb2r6MMZ2V1ZMuy32DAVZ9YMzlr9zYlXKEuzeCaWn29DuqgeP5PJQFFKwd37Y9jmzXwuKD5KrC9WCAzrS2VClxmJYHj2Bf842qtKa4x3SubHu4w6wp8v42wQYCU0Kygc5UgkCFYgOBgvpYCFKhiro6Mg/PBTWTIIgrLlSVwUWPIcaX6Sq4ZW4sy6CZ0H/pEMDAjU8drZ4KmGw7W4FshNynplMuP9ewbSs9yxtuAsKt4ezuUQBGHmAYdPHytC9HUzf67z/j6TF/zg6XgTe0AG116cNddAL8oNermCIkkKuTwb5xy1+3WRjjacC0K5Ire1fV5RyXtD/1JgEHJ8gbPK1w1V5fHZXX/ARvQFlP+uqxFYmPkCHBmK/R6/VvAghI/KYHlc76hdsAe0sd1UrPIHjVEtwgvDQl+4de9L9cHG3Qxj23ajlpISWA3tGADhwzXDxi0KYClckebrCKhizWFQLYXezALYklgY2yghG9jZHufV9jRjYSr9DLtwUmVf3oglAzKkWErNs8dWqEKYFx7HpFf0FiZh,iv:1aX4E+R1oFMm9eI1RENFIYCoatX8BdHo6PhrEGq3rtg=,tag:nwniLOOmpYd3Z6R92SvB3Q==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvRmtrdkpMOVhRdTVucVlr
|
|
||||||
QmlENGdDN2J1MUczSmRhNUpwYlBjWXhqbERzCnpLRFl1bVVtSFE3L1l4T3Rwd2ZQ
|
|
||||||
Z1MxTzJYZWovMmd2RTA3RFJVdVRRTEUKLS0tIDFQWHh2dmQxU1FUZnUxalhTdEd3
|
|
||||||
c3dUVzg2L2VOWFlmUG9XdFNWN2RvQkkKYqSmFMkDV/T7AOjKYQNJW85gUzFraRre
|
|
||||||
GhwAuPJ9oNQAhSRa5z2p6ghoUplSXtNZ6H2OzETRfOc4N2cHLljW/Q==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-08-18T04:18:28Z"
|
|
||||||
mac: ENC[AES256_GCM,data:ScGMBQqGm31fWDZurmlmk+rnlvyaQkalUcJ6ds/U5JJ2niTf7Llbruuqt2EzMQO09fT2kQqAIYr4cF61Czip6rDhpG8I3H3lBc2mOqUXbeCFyRL6mRpVYPOgkSzgyfXUfe8T++54UhQ0Clv/8Z4XB48oTs9detkUDzAfxP4azRE=,iv:edhsVviyjAm48Blm5CEQ0PBhcWVX97xi2hEbgDX9Dpg=,tag:Xnn85E99LibtB/LWwi5wcA==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:ECE=,iv:bISz4HovH++X7DW1Qj8Cw0L6+EvPB0+68hh7tfyW5C0=,tag:w+SjD7MsfeIuSf62n+Zl7Q==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:AxXaV3Mb,iv:xfJb354Rrjz3zLctW2i6hl40yh9EsfIqKxDvmn8jqnU=,tag:1xQ/RR3KhPLKCTWCYjebJg==,type:str]
|
|
||||||
metadata:
|
|
||||||
name: ENC[AES256_GCM,data:gdQ4OoMYKUPEbsUeA5OI4i1xnh8=,iv:IA2kaCyqQmkYpLolvcTF4aleh+yd/ImXJMhRMvpGCgo=,tag:8qyYYw4EhBKKPzEmpepAeQ==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:/mOSyXWmRg==,iv:cpeHUTlMqlJzJttGtuR3DoiMtvVqFmDS0/5Tl7K7c2M=,tag:q4GxWgWn2wQJxJHqnq4WQA==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:92OcrsuV,iv:Z1XBy6iZ6unGrK4/SSdDa58pbPL3022gU/f1AOM4uvc=,tag:cd4wWRq/ohm6BJ/Eo6HOIw==,type:str]
|
|
||||||
stringData:
|
|
||||||
REGISTRY_PAT: ENC[AES256_GCM,data:zDKruOXhzsIcFZTTp4r4r9rwHih9uf1cCp06KZ6eIXSIZDvR6aQY4A==,iv:5Vch5Z7xn1KkRxrgOs2p/M58nd6WhtcuUhoZDCi8LSY=,tag:14SAZIWGOxJMqLBtOorjYQ==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIaG5QVUYyNjhvaENPcUFy
|
|
||||||
K3RSSVM2N2hTeFhNK0M5YWZmODhzZU1HaHlFCmNGZWdFcFEvanVwcXpGSmVvRHVx
|
|
||||||
aG45c2lBY1RuSTYwbDZTVk1QeHNWOTQKLS0tIDg4TnVMNjNtaU1VQk5zQjUvU2hM
|
|
||||||
R245ZVdqc0ZWWVhJb3dOZ3lpU3JjUlUKawSg09ZPq8FKx5tvOVZZ+K4yh7eTQsUp
|
|
||||||
be8mWUpS0+eEmNqh35BwU3HrETMQFA6a1kjVp30JOMtqa5rbYlzF7w==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-08-23T23:14:53Z"
|
|
||||||
mac: ENC[AES256_GCM,data:yzi6woUaUCi6w9pG/eKnU7k/VfZgoXg/tW8p9joG8p7iabLWMdlXlx23m/CItw/NE0zeGWA5iZPPFZXOit2vN36VzK3kQNoFW6QbhvYLZi+78C/RWIdcpZkxB/tPIRG0vq8Q1SA+rwGjG/0xeAFh+R7k+YBTMd2XWBH3P7EI5T4=,iv:kf6MDaAVrtDvPIEjHMMLxXDSDRC3I1GpsPeJFUYppiw=,tag:NukVbGLa9EoMYmRsa4nBtA==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:D9Y=,iv:EH+zD6bogxh/h/Oe+RxDCtfO96tkc56ou14V+68nK7k=,tag:xTeuhSzqZyrS/ltqvtHcgw==,type:str]
|
|
||||||
data:
|
|
||||||
.dockerconfigjson: ENC[AES256_GCM,data:P8x3bhPbJTFvFIKE8WQY1P7KqPwrxNTYTNR+4/Z7nZSr2YQab/Db0SRcvFO9lr4ImdO6waZ8EZ6CI1RzIYx1WcjGXs8FzX1jpBXSw9EG69vRTZCqRRJR8c2yg7qmNFfSvcPAUhK9tciYCzzFWRHZEDAmQ52cox/sMdxE/YR61dhYCy1E9kPwTnaDmDj5Hu72mbLJDxIYpN3T2hVWlw02lHYHgFnuKsPb0Tf1lxn174j/gMMxUT6ynVhWSEExzHvTqQ0RZzuP+VXAh3N9HasbtfcAabt3FtEjYNZqYJ+OE8oiX/A6YjgPDqUOR3AJFZE3,iv:6YsyIHQc8xp8T8XUWhN/pBeaYVI/VdIHOe/w/hb5e6U=,tag:nmGFdkv2hLgj8Dp8/0MOkw==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:hOA36Sjr,iv:y0XHfUOUnut8z0yM2g7Beo3qiqxJhLLhffPpNlUhaec=,tag:45s3FjYCr83vYCSXDXu2/g==,type:str]
|
|
||||||
metadata:
|
|
||||||
creationTimestamp: null
|
|
||||||
name: ENC[AES256_GCM,data:yD8Fo/fncAA4qkaY7RjhRg==,iv:qwc4WId/kGwradgxFUwG5B5XIVYRlMY2HxsaGJAzrjw=,tag:O8GwNdDHMJreL609G/gykA==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:zg2o,iv:KDpM2L71/LDI4JLTQUtbcv5SV0IraAbKpNEBzSFn/rE=,tag:uFm+3EkgZN3NC2hUPXo9Pg==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:5/a+6lRp9Ea5rU/+gEMIgoDYs72xRWkfwefMcy+h,iv:jYirTXlb4rFwvb+nLcgB5X4x1Q/S+3LsVL+7ypS7mkQ=,tag:5Wc0DrMZxIpC0e39MzlAlw==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB2ekxNWW9VQ09RTUlMUzVy
|
|
||||||
QlVrRm12Smt2akRaYmkvMHRPRmJvOTdBTzE0CmdhVFNyWDJnWDV3eTFFUm1majk0
|
|
||||||
L01ZWTNZdmcwc1MwMHgzQmdVZy9KcjgKLS0tIHZLVGg5VmNRQ2ZNY1lIRHlzaUlo
|
|
||||||
TWVNelZRcHFveGRiNTVvNzJCNWtDSGMKyV3Puscgx3RqK65KSL6SYaTauxsBY3qd
|
|
||||||
CeFU928hcB86DwAG/Atq2Qtd7S9pzuzOVQmXRZxwpCDTTyRVhU7eVA==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-08-20T04:43:53Z"
|
|
||||||
mac: ENC[AES256_GCM,data:Leus5j38xJwJz3Ge9WggBjQSAh2ESlIMUnX9SylE4oIcAt71f8WadtSCOmnqT3ZK+uN8f+Huq6WetGHWUdLfZjZUngHQHWLoRP1xpTVvB5HwJK4F1ASvmL+u1rC88AG3JsZc3Kc2N1G+m6QruQK/9HSmkMi/HGXy658ijzErEzk=,iv:1ga2dKDqTalj9WnjVT6AubXsL7130CuJp3SbBkTb/64=,tag:YsD9metJdYiJmdVXpPuIsQ==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -1,10 +1,8 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
kind: Kustomization
|
kind: Kustomization
|
||||||
|
# All homelab SOPS-encrypted Secrets, decrypted in-line via the ksops generator.
|
||||||
# Disable hash suffix for all generated secrets (stable names)
|
# Each *.enc.yaml carries its own metadata.namespace, so no namespace transformer
|
||||||
generatorOptions:
|
# here (that would rewrite every Secret into one namespace). Renders exactly the
|
||||||
disableNameSuffixHash: true
|
# Secret objects — replaces the old argocd-cmp-cm SOPS plugin.
|
||||||
|
|
||||||
# SOPS-encrypted secrets via ksops generator
|
|
||||||
generators:
|
generators:
|
||||||
- secret-generator.yaml
|
- secret-generator.yaml
|
||||||
|
|||||||
@@ -1,25 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:wGA=,iv:Z2Gfzq3aJ9j4fYaeLQolgLb/XELTHrKX9at3vUsMLIw=,tag:yZLZFnltJln5jFoVsyyZ2A==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:bJkV4pNj,iv:0ZT7l0kw9qSoiEMZioRw1aBzzlxBkoXR+hOxoPI7zPU=,tag:EQczPq508Qw1vi/oLCeQpw==,type:str]
|
|
||||||
metadata:
|
|
||||||
name: ENC[AES256_GCM,data:sZwTVM41PPaidMwFNRo5RvU=,iv:BHfuwIHng7rkeLK3a69t8cI9QeSfD/3FEXqxby+gBxM=,tag:3zQx8APnE2ZpBKf/ZYCxOA==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:i7lpoEaZ1oXS,iv:jUYyDPYhf1TV51he/S5MlKPD19Vmz6wfE98y8qFEg3U=,tag:zgHDVlplmW/XPUAKdncfYQ==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:2khs1uIg,iv:ET6HcBGyv33fGFlFAl3dkJQB83naHGeaYuWvW1IFhvw=,tag:qIfMeaSwv//7cNWvy8O5dg==,type:str]
|
|
||||||
stringData:
|
|
||||||
PAPERLESS_SECRET_KEY: ENC[AES256_GCM,data:kj9DrQYL3cQGJz87FHYlFKy6Muu84Oy/6wnsWAh0w/3MmcTAsCzQvUtKLKKf1U61JTQ=,iv:UffVv78vMHDEWHRFdSKZ/6qyrVD02Nlk0CNxwWd1jTo=,tag:LNMKnFWw7QCjfcKzI1s8ig==,type:str]
|
|
||||||
PAPERLESS_ADMIN_USER: ENC[AES256_GCM,data:a3bx7z0=,iv:nrU6VXZE74PNXM+Lgg4K/3mhusaQhA2GAgl0GzqAQt8=,tag:x8nswTfkFHR6Ou+JSI9P3Q==,type:str]
|
|
||||||
PAPERLESS_ADMIN_PASSWORD: ENC[AES256_GCM,data:+SWyAq7D1mNt1TlFOlo2Z0biK8kEn097,iv:an2ZZuXNSig+7mJP7ogHFc5+QvUb+z9pzqcV4xvFLbA=,tag:OaRiVZsnajEjAlc9mDWo9w==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1Q2V6bVBUYmVRR1N5SCtF
|
|
||||||
eXdTV2dmWjhtMS9lRFEzS0wzbkd6Y3JLMVRVCncydFBjdmJkRCtVUXphR2w0SDlJ
|
|
||||||
cng2bi9MWlJzTEN2amJrYjRJN2VFcEEKLS0tIEY5cmw2RGhnbzUxZW9FaFJjQmVN
|
|
||||||
WWcvNlNiYWdwbnNSR1Q4alpDZmFqTFkKh9TOw8ERP9fpx2pKi/Q0b7+OkEv0UC7o
|
|
||||||
aAIK4Tzvi5dp6y9IWcu9l6PjDLeYWOJ5wr7QABaFNOz82hngxUleDA==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-08-25T16:04:48Z"
|
|
||||||
mac: ENC[AES256_GCM,data:OXPQXUyn/SDftKH5nRzhqEtnaOb4Gp7etmGojqV4Z01kHnABZ42PPIyiBb8z997eLNZsK6/1bi7nUNyVp4fIuXG4F52aEdMewofOocCHokRnNmz7jzhooK1gScJb2u0eHG3FL5iLONMaGgVpk7BLYO3e0xiDytWGe8BxcuDukPg=,iv:kh0jUwFvO6AUICy2Us1E7YOTEcp3L+ptrGdDWBSpWyc=,tag:qUPFmZ/rpljln37f/NRjjw==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
FORGEJO_TOKEN=273fdcffabbcbb5a191e8289c73d106063acefc6
|
|
||||||
LLM_API_TOKEN=s3VksXyw2z3sGbegnwjMDFnJ6CtNRd1a5CcnE5A4ET77toCcykNdunk6Oa2J
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
apiVersion: ENC[AES256_GCM,data:bnY=,iv:Fuc3aqncHQ+L16o7eLarPbOECD3o8Mk5c2r9pQBpy70=,tag:JPfEnbNb3wZXPdXafnJDqw==,type:str]
|
|
||||||
kind: ENC[AES256_GCM,data:WFlmi4Yg,iv:Zq/KQbgNcBVoo8ZsQ2H79ygyc8Dtkgxh4fCpEExfwSg=,tag:cWHP6Y5V+nZP2tFMJrOB8A==,type:str]
|
|
||||||
metadata:
|
|
||||||
name: ENC[AES256_GCM,data:iCXbhwvg3Zq6YL/4j0wAy7Y=,iv:8s+d/8lDVEL7bGdIF+GOtAxapKnmx8JTjLSO04hXF5I=,tag:LjzX40DjK+uRZPCXsMlmwQ==,type:str]
|
|
||||||
namespace: ENC[AES256_GCM,data:HRMdZdCbxORQ,iv:MvaIWoKWjJRA7/fce0KtXRkFH/7cn0OuIg2QwHEdQzM=,tag:NqztiGyfU3BaopWBKhx2eg==,type:str]
|
|
||||||
type: ENC[AES256_GCM,data:myBW86Za,iv:3x9ys5UzVhAuX8gvZO67B1e+Orw4Aqasv/lHBgUV0b4=,tag:yl18P6SaxVLJidwmRtd4aQ==,type:str]
|
|
||||||
stringData:
|
|
||||||
FORGEJO_TOKEN: ENC[AES256_GCM,data:SUoBpNKOItyNGY01EhKNlPH0fyN4N7g6bfU2jsGogpCMhP7NuRipoA==,iv:h77RtmYZjXxiHYw1pHynQHuVX1+yJDGHwsXLf+DbUYA=,tag:GHzoDNHP1GGqlN5eT/N7TQ==,type:str]
|
|
||||||
sops:
|
|
||||||
age:
|
|
||||||
- enc: |
|
|
||||||
-----BEGIN AGE ENCRYPTED FILE-----
|
|
||||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBzTVBsekl3TGgzQVRMUU9m
|
|
||||||
cTduS2NoZW5uZFNNMG13cFY2cGVsTnlXaXhrCjJjbzhLdHZ4ZWpUV3J0cDQ0eVlM
|
|
||||||
WDNxdzVoQ2ZzcGJSbTU3RVorcnczNVkKLS0tIFdtQTE4Umk2TDBzUmdKOXNkbjFi
|
|
||||||
Vk5vK2VuUHVsb3FQL21vcGU1UW5CT1kKFM8vVjji3Cg9dvfTr4Hx7BJC8JH5ovef
|
|
||||||
Dj6zkofhsNWPgP9T+mnQakj+C0RKmHOMJqfWP7vwCBkZoNosIJVlMw==
|
|
||||||
-----END AGE ENCRYPTED FILE-----
|
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
|
||||||
lastmodified: "2026-09-01T05:32:42Z"
|
|
||||||
mac: ENC[AES256_GCM,data:it24T9y9ixXo2aiL37k93vKFR+SRjjuI9DQdv0sWYtTogWnc7+uXBY4Zip/ouWyCse1muKKAGuek5c0XVrvSw4an9VkaXFczeunaZb6MOyVbVOkmJr+5xZFpZGjYcSkrhaWcVheedZ3iIFU5UWI7BBn/qQCf+HJ483cJqwtrV34=,iv:WprlWJdsMBNjqaA0O3ekfXMUpX5gC6OLYortQXYdTS4=,tag:rGqSSRTYTv2VR6AcRokO0A==,type:str]
|
|
||||||
unencrypted_suffix: _unencrypted
|
|
||||||
version: 3.13.2
|
|
||||||
@@ -7,12 +7,8 @@ metadata:
|
|||||||
exec:
|
exec:
|
||||||
path: ksops
|
path: ksops
|
||||||
files:
|
files:
|
||||||
- agent-pod-models.enc.yaml
|
|
||||||
- agent-pod-ssh-key.enc.yaml
|
|
||||||
- authentik-secrets.enc.yaml
|
- authentik-secrets.enc.yaml
|
||||||
- cloudflare-secrets.enc.yaml
|
- cloudflare-secrets.enc.yaml
|
||||||
- forgejo-registry-pat.enc.yaml
|
|
||||||
- forgejo-registry-pull.enc.yaml
|
|
||||||
- forgejo-runner-token.enc.yaml
|
- forgejo-runner-token.enc.yaml
|
||||||
- forgejo-secrets.enc.yaml
|
- forgejo-secrets.enc.yaml
|
||||||
- grafana-oidc-secrets.enc.yaml
|
- grafana-oidc-secrets.enc.yaml
|
||||||
@@ -23,7 +19,5 @@ files:
|
|||||||
- homelab-ca-secrets.enc.yaml
|
- homelab-ca-secrets.enc.yaml
|
||||||
- loki-secrets.enc.yaml
|
- loki-secrets.enc.yaml
|
||||||
- minio-secrets.enc.yaml
|
- minio-secrets.enc.yaml
|
||||||
- paperless-secrets.enc.yaml
|
|
||||||
- vault-secrets.enc.yaml
|
- vault-secrets.enc.yaml
|
||||||
- vault-unseal-keys.enc.yaml
|
- vault-unseal-keys.enc.yaml
|
||||||
- portfolio-secrets.enc.yaml
|
|
||||||
|
|||||||
@@ -5,9 +5,9 @@ metadata:
|
|||||||
namespace: ENC[AES256_GCM,data:wK6m,iv:KtA31Bo8aGE1HU8H9KWbMwt2NfywWfy77G/LaReaI1E=,tag:V9SSlPawRQw0x2utFNN1aw==,type:str]
|
namespace: ENC[AES256_GCM,data:wK6m,iv:KtA31Bo8aGE1HU8H9KWbMwt2NfywWfy77G/LaReaI1E=,tag:V9SSlPawRQw0x2utFNN1aw==,type:str]
|
||||||
type: ENC[AES256_GCM,data:6UPZ1ZTR,iv:FxN1ebrlJ4IO3eDGEYSvktSpMgAebcl0DO1WHh5O0+0=,tag:2bZN4zdbHzx0oR+bTJPPJg==,type:str]
|
type: ENC[AES256_GCM,data:6UPZ1ZTR,iv:FxN1ebrlJ4IO3eDGEYSvktSpMgAebcl0DO1WHh5O0+0=,tag:2bZN4zdbHzx0oR+bTJPPJg==,type:str]
|
||||||
stringData:
|
stringData:
|
||||||
key1: ENC[AES256_GCM,data:ke5EfsOsZHBJyAQoFfwhhWCQJQgnwcrBqL4CzPIfkx7bi9S7kWgWySdxcXQ=,iv:eYKxG8k+hizp2t2i/YMR2lQNJQFV+A21YyWnDc+kJ9w=,tag:Wa1Lx3EKvfpYfnLrBOWSIg==,type:str]
|
key1: ENC[AES256_GCM,data:dV2HOh7W1Pl0QDJaGvtEKpBppybQMiK5kxzyk05zkA352ZmJvE8Ppm5yWKk=,iv:grQ8v2o/LHpJnIZonjtgTHKcLUQIH8xl1FtWtEs4rEs=,tag:lANcccYT6Ouo/0IcoLw4uA==,type:str]
|
||||||
key2: ENC[AES256_GCM,data:JxiXgLKvDe5oiHlwIL/Cj8txHc7fVQ5VzBcQMU/ro9TSOcTGSe7Z3omQzwY=,iv:x5arcY2dey+npMpUxjdUPV+t94LEaMOXq3iOar88eT8=,tag:E/Z5zEDy87B1IAWkdjdNEg==,type:str]
|
key2: ENC[AES256_GCM,data:YSdpYL8h56PfUrvMFhBXhmBG2en0toLKEpYlJqwjAk/vI0jm7gq/TqGn034=,iv:GxthkNLhm3qHxjkZetiYl58Qa/K7G2ib6E+LWK16H8Q=,tag:q79UUoh6uCcMZJ0U18Ireg==,type:str]
|
||||||
key3: ENC[AES256_GCM,data:AsxyyBTHzt+SLru6ayi1UbgPPFmdIvowIDdcfc7N03Xt8V/KaJW3kbXExG8=,iv:lA+bERIBq4+bxx08ZZjP5tMHC9JQlGPW1EMCwEixSeg=,tag:xkuTAU3KTidFuhNMXkVFhA==,type:str]
|
key3: ENC[AES256_GCM,data:sO7Lao9qkmIOdARL+6FBcLo+e4z8LzMEzNrwcy5hv9uRuxppwAXrP0MO1vw=,iv:wSWe9h1HovRhV5YCZrmvcB2VEfRYQ+gasGO3uh8IpmQ=,tag:Bc5yaROQDTz1AhVnFDm/BA==,type:str]
|
||||||
sops:
|
sops:
|
||||||
age:
|
age:
|
||||||
- enc: |
|
- enc: |
|
||||||
@@ -19,7 +19,7 @@ sops:
|
|||||||
7RltNZF2SCxjIv5C2pqf3CgmRBaQGWgMybRRH5gdB87PLBKkPL3+HQ==
|
7RltNZF2SCxjIv5C2pqf3CgmRBaQGWgMybRRH5gdB87PLBKkPL3+HQ==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
||||||
lastmodified: "2026-08-26T23:33:09Z"
|
lastmodified: "2026-08-12T20:52:24Z"
|
||||||
mac: ENC[AES256_GCM,data:kr8XuOqXyhfrC84yBot5aoVQZZhQd9xYMUEs0XDwkmCgtG0iobADYH5XWPh72xSdWEtwxkZJL7r0UEyEhU9zjlUA5qHupOSBBt1SgZ+zstMvaOqUnlNn//p/DIJBpsiT/qmx64NpTLAiz6lm0796MozIMr8PTX+ubGLHi9tnUiY=,iv:jwP0MLCT7nG6m+nZeqNip9q3BcScpcDXmInnY95DicY=,tag:Cgg2IUZARkaK1gF2qUIthg==,type:str]
|
mac: ENC[AES256_GCM,data:kLjoBWmJ2bZ13EbuzgppahHKmCY/xtCi1R7xCEUbCP4FQfiVaC4qAbVftE4lTNlXa2mqCKsGPYZ/A3HV/4+a+it/pg0a+rj7E7DszhieWbZufHMJs+w4/Le8l8wFFE5aNW0wdvGTJZ0n4HBOdAkE1qN9ReaiaQgr0rk5ggOPG9g=,iv:MiDVUgcoKrP/gj4keqomo6OB12dmz9VoesTiHQTnBFM=,tag:cLK67Pym4BuvXLF1RUA8Gw==,type:str]
|
||||||
unencrypted_suffix: _unencrypted
|
unencrypted_suffix: _unencrypted
|
||||||
version: 3.13.2
|
version: 3.13.2
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: github-repo-creds
|
||||||
|
namespace: argocd
|
||||||
|
labels:
|
||||||
|
argocd.argoproj.io/secret-type: repo-creds
|
||||||
|
stringData:
|
||||||
|
type: git
|
||||||
|
url: [email protected]:[email protected]:Riotpiaole/riotpiao.homelab.com.git
|
||||||
|
sshPrivateKey: |
|
||||||
|
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||||
|
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
|
||||||
|
QyNTUxOQAAACAqZaKCvVj9z9JtQ8kyNpE42siEoEoUXTftc4zz+dAerQAAAKg5eEfbOXhH
|
||||||
|
2wAAAAtzc2gtZWQyNTUxOQAAACAqZaKCvVj9z9JtQ8kyNpE42siEoEoUXTftc4zz+dAerQ
|
||||||
|
AAAECTvujulFODUs/5miSpkwqqovKyuK3TSOSXFR8tNYOYKyplooK9WP3P0m1DyTI2kTja
|
||||||
|
yISgShRdN+1zjPP50B6tAAAAIXJvY2tsaWFuZ0BSb2NrZGVNYWNCb29rLVByby5sb2NhbA
|
||||||
|
ECAwQ=
|
||||||
|
-----END OPENSSH PRIVATE KEY-----
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user