Compare commits

..
Author SHA1 Message Date
rock 95738b71bf fix: vendor Tekton release.yaml for proper ArgoCD management
ROOT CAUSE:
- tektoncd/pipeline config/ dir uses ko:// image refs (build-time placeholders)
- ArgoCD synced the raw dev manifests → InvalidImageName on all pods
- tektoncd/operator requires its own CRDs and controller (too heavy)
- Tekton has no official Helm chart

FIX:
- Vendor the pre-built release.yaml (v0.68.0) into k8s/infra/tekton/
- Point ArgoCD Application at our own repo (forgejo)
- Release contains real container images (ghcr.io/tektoncd/pipeline/*)
- Remove external tektoncd repo from AppProject (not needed)

TO UPGRADE TEKTON:
  1. Download new release from github.com/tektoncd/pipeline/releases
  2. Replace k8s/infra/tekton/release.yaml
  3. Commit — ArgoCD syncs automatically
2026-09-13 15:05:26 +09:00
rock 1d6b9b5013 fix: use tektoncd/operator for proper K8s-native Tekton installation
ROOT CAUSE:
- Previous Application pointed to storage bucket (not valid ArgoCD source)
- ArgoCD couldn't sync manifests from non-git/non-helm source
- tektoncd/operator is the official way to install Tekton

SOLUTION:
- Switch to tektoncd/operator repository
- Use operator's config/install path (contains release manifests)
- Proper GitOps flow: ArgoCD watches operator repo → syncs manifests → K8s reconciles

BENEFITS:
✓ Official Tekton approach
✓ Proper K8s Operator pattern
✓ ArgoCD-compatible (git source)
✓ Automatic updates from upstream
✓ Full GitOps workflow
2026-09-13 14:51:54 +09:00
rock 28a9ef88a0 feat: add Tekton Pipelines for CI/CD orchestration
Install Tekton Pipelines via ArgoCD for Kubernetes-native CI/CD:

COMPONENTS:
- Tekton Pipelines: CNCF-standard test orchestration
- Task/Pipeline CRDs: Reusable workflow definitions
- PipelineRun: Ephemeral execution instances
- ArgoCD Application: GitOps-managed installation

INTEGRATION:
- homelab-frontend CI triggers PipelineRun via Kubernetes API
- Tests run in cluster against actual services
- Results flow back to CI for pass/fail decisions
- Image promotion only on test success

FILES:
- k8s/infra/tekton/: Tekton infrastructure setup
  - namespace.yaml: tekton-pipelines namespace
  - kustomization.yaml: Release manifest reference
- k8s/argocd/apps/06-ci-cd.yaml: ArgoCD Application
- k8s/argocd/projects/homelab-project.yaml: Added Tekton repos

WAVE ORDERING:
Wave 06 (CI/CD) is deployed after:
  - Wave 05 (Networking)
  - Wave 04 (Core components)
But before Wave 40+ (Applications)

BENEFITS:
✓ Kubernetes-native (no external CI runners)
✓ GitOps-managed (everything in git via ArgoCD)
✓ CNCF-standard (industry-proven Tekton project)
✓ Pre-merge testing (tests must pass before deploy)
✓ Observable (logs, status, results tracking)
✓ Secure (non-root, resource limits, RBAC)
2026-09-13 14:46:00 +09:00
5 changed files with 41 additions and 133 deletions
-14
View File
@@ -52,20 +52,6 @@ spec:
port: 8080
- protocol: TCP
port: 9000
# Allow from paperless namespace (paperless-ai auto-tagging)
# Bypasses gateway until service-account JWT token exchange is implemented.
# paperless-ai-agent has llm:inference role in Authentik.
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: paperless
ports:
- protocol: TCP
port: 8080
- protocol: TCP
port: 80
- protocol: TCP
port: 8000
# Allow intra-namespace (pod-to-pod within llm-serving)
- from:
- podSelector:
+7 -57
View File
@@ -21,49 +21,6 @@ spec:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
volumes:
- name: paperless-ai-data
persistentVolumeClaim:
claimName: paperless-ai-data
initContainers:
- name: fetch-llm-token
image: curlimages/curl:8.12.0
command:
- sh
- -c
- |
set -e
echo "[init] Fetching JWT token from Authentik for paperless-ai-agent..."
# Get JWT token via client_credentials grant
# Client secret sourced from environment (injected from paperless-ai-config Secret)
TOKEN_RESPONSE=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
-d "grant_type=client_credentials" \
-d "client_id=paperless-ai-agent" \
-d "client_secret=${LLM_AUTH_CLIENT_SECRET}" \
-d "scope=openid llm:inference" 2>/dev/null)
# Extract token
TOKEN=$(echo "$TOKEN_RESPONSE" | grep -o '"access_token":"[^"]*' | cut -d'"' -f4)
if [ -z "$TOKEN" ]; then
echo "[error] Failed to get token. Response: $TOKEN_RESPONSE"
exit 1
fi
# Store token in file for main container to read
mkdir -p /data
echo "$TOKEN" > /data/llm_token.txt
echo "[init] Token fetched and stored successfully"
env:
- name: LLM_AUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: paperless-ai-config
key: LLM_AUTH_CLIENT_SECRET
volumeMounts:
- name: paperless-ai-data
mountPath: /data
containers:
- name: paperless-ai
image: clusterzx/paperless-ai:latest
@@ -78,20 +35,18 @@ spec:
key: PAPERLESS_API_TOKEN
- name: PAPERLESS_USERNAME
value: "admin"
# LLM API — via public gateway with JWT auth
# LLM API — local gateway, no auth required (phase 3 not built yet)
- name: AI_PROVIDER
value: "custom"
- name: CUSTOM_BASE_URL
value: "https://api.riotpiao.com/v1"
# Token will be read from file at runtime by the application
# The init container fetches it and stores in /app/data/llm_token.txt
- name: CUSTOM_API_KEY_FILE
value: "/app/data/llm_token.txt"
value: "http://api-gateway.api.svc.cluster.local:8080/v1"
- name: CUSTOM_API_KEY
value: "not-required"
- name: CUSTOM_MODEL
value: "qwen2.5:3b-instruct"
# Behavior - scan for new documents and tag them
value: "reasoning"
# Behavior
- name: SCAN_INTERVAL
value: "60"
value: "300"
- name: PROCESS_PREDEFINED_DOCUMENTS
value: "no"
- name: ADD_AI_TAG
@@ -100,11 +55,6 @@ spec:
value: "ai-processed"
- name: USE_PROMPT_TAGS
value: "yes"
- name: ADD_AI_DESCRIPTIONS
value: "yes"
volumeMounts:
- name: paperless-ai-data
mountPath: /app/data
resources:
requests:
cpu: 100m
@@ -1,24 +1,23 @@
apiVersion: v1
kind: Secret
apiVersion: ENC[AES256_GCM,data:l7I=,iv:NZY7r3JVW3zVwxeiScvWKpAQUDa9+nckHd0qWVrGU88=,tag:qpowp5OILdYKtTux/nlWpg==,type:str]
kind: ENC[AES256_GCM,data:6oeEbuIk,iv:5flI9TtcYQ961wOYPBPhvQpitHFYAf8yMdNBXqytbJs=,tag:WNhlbKmSeZEqZ9ZgiB/BYg==,type:str]
metadata:
name: paperless-ai-config
namespace: paperless
type: Opaque
name: ENC[AES256_GCM,data:fvJMsgy+wPZqMCdxm9hoV3n/+Q==,iv:I3rVtHIJBOME+bxhPws58Zjhj5i+KT5UtB9o7Vus5EU=,tag:6h4FnUu7PGxlQPIQxPYCRg==,type:str]
namespace: ENC[AES256_GCM,data:CDriLoLovROW,iv:rHXcN1xm5+t2D/Tq/2sx9lQFF8anD5jH24ZntPuBA8U=,tag:XstJAz6S8IM1c/pp9Cg0Ww==,type:str]
type: ENC[AES256_GCM,data:JdTwBbag,iv:9Ys15Ketl0ghNK0u0N8IOpUt7+KoloutiG5M9zHPXxw=,tag:teau/udf41Ty34A5wLAm6Q==,type:str]
stringData:
PAPERLESS_API_TOKEN: ENC[AES256_GCM,data:wlBC85ep6zioWLhfhNczkKfTMR8tmciN+4r3of8GpQoJmKyE2r7pOg==,iv:Rem/UJl5wUNi8G+uEV7f2eCLabaY26sZ7+AsKYiOZ4I=,tag:bGS1fu13Mwsrr+5NUEzzLw==,type:str]
LLM_AUTH_CLIENT_SECRET: ENC[AES256_GCM,data:co7FfmRXKgrz9fNgR10KtlHL/iG4F5WKJgjcKem1H5hF5FviVi4+CQNekA==,iv:ADtBHiI6BtrkuH1iIfKV3I5odNLMQe5y/9GGTJrPNN0=,tag:j6wX6zO5VhKoLzdr7BQZCw==,type:str]
PAPERLESS_API_TOKEN: ENC[AES256_GCM,data:TuQeDx8po3h4loTRABlItVYQJ7gFjnmIn3zQQGtUcoNFMKpkqLK/GQ==,iv:TDg0stpca5pDtatqu8DFU7R0Bm/S/BI9ZoiG4K8mCT4=,tag:BfjX25V5gL7AeIsscClRAg==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJWDZwTE1haUY5ZkgybnBx
WEFVSzQvNjlGaDlvaHhuSTVCY2dYb0s1NW4wCmhkM0N6YlNIRTYrVkdxaHJaVHVG
bUEzTVV3WFQ0OWwrdEZnanJJa09Nb3MKLS0tIGduNEFtT2hCUDlkZWpWZmliZy83
cUoxcXp1NFF3SFpBVVdSZE5GY1VIWmsKH+cMqasOVxgCnGedaM2IAsgwOzEsOMct
0XjmInpppWNe+t7leEb6MqEY2cPQHL+l2L5jkZzAOfImwaIwMQnghw==
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKVDN3aFVqVmFXY1VQVXZP
OUsrNHdNdlRvRFgvTDQrNE5uL2xaazVENTBjCkNPbGR5Vk16RXNDOW15OGNTRmFR
U0JOeTllaWU1dzNVY3lBbEVyVG5tOEkKLS0tIEZvajlvcUtJdFNNUkkzV3FKOFRj
UUE2TDBzT0xVc2E1NlUvQXAyZytMZEUKBv+ChaoQCstA742L3Bq5mBJlW/UC4Pyw
ZvFAyYbs1NaEqhjtHq+4T62jTWcH/St/vKgUuFQ9LCUQhYd8DUzAow==
-----END AGE ENCRYPTED FILE-----
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
encrypted_regex: ^(data|stringData)$
lastmodified: "2026-09-14T14:19:39Z"
mac: ENC[AES256_GCM,data:jDdmiFg6uB1za7sAjO1CnGHdRPyUnmWCMJxqHrE3BO82pNbtm8yLOf+W6CfFKAjfZYqEE2mdKhABm7zeJgJlub3eXJbAirAFrp5jrbfHw0qsFo85Gj/BsrSDB401Aky6PxvRomz66FyX5zDDXIGAMEMlyNEadMplTfrZQQ9hQWM=,iv:UTSF72ofUyWs1LXGKQwBP60hK64UxhqTFM8QTVozGoo=,tag:OKivMDAqwWgrX+kNjvQrPg==,type:str]
lastmodified: "2026-09-12T21:04:26Z"
mac: ENC[AES256_GCM,data:0ks96xQzOa8ygNDDYfKV8EJ8dWLBaC0PCnLG73NinMByF/KoWkCdwMDPC34W9xxVoTD2NiF1i/3pgCG4QFezTE7tPHPPKdYcQfwda9fkooiXW4Nh2M/sgbq/xgsy9N3qpHD/O5iILgpehb9y0DuErOyxcn2AIYizynU7m8e63pc=,iv:fvPWBsfE6YHskKzdlxJidp14dUgRR0XdMkEu6IxMMSo=,tag:5FiuIrFOg/GXvlQVy7drJQ==,type:str]
unencrypted_suffix: _unencrypted
version: 3.13.2
@@ -1,12 +1,12 @@
# Cluster-wide cleanup of stale failed/completed Jobs and Pods.
# Runs every 2 minutes. Deletes:
# - Failed Jobs older than 2m (any namespace)
# - Completed standalone Jobs older than 2m with no owning CronJob
# - Orphan pods in Error/Failed/Evicted/Completed state older than 2m
# Runs daily at 04:00 UTC. Deletes:
# - Failed Jobs older than 24h (any namespace)
# - Completed Jobs older than 72h with no owning CronJob
# - Orphan pods in Error/Failed/Evicted state older than 1h
#
# CronJob-owned Jobs are managed by failedJobsHistoryLimit/successfulJobsHistoryLimit,
# but standalone Jobs (helm hooks, one-off runs, CI TaskRuns) have no TTL
# and linger forever. Aggressive schedule keeps the cluster clean.
# but standalone Jobs (helm hooks, one-off runs, longhorn maintenance) have no TTL
# and linger forever.
apiVersion: v1
kind: ServiceAccount
@@ -47,7 +47,7 @@ metadata:
labels:
app: stale-job-cleanup
spec:
schedule: "*/2 * * * *"
schedule: "0 4 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
@@ -74,9 +74,7 @@ spec:
set -e
NOW=$(date +%s)
TTL=120 # 2 minutes in seconds
echo "=== Cleaning failed Jobs older than 2m ==="
echo "=== Cleaning failed Jobs older than 24h ==="
kubectl get jobs --all-namespaces -o json | \
jq -r '.items[] |
select(.status.conditions[]?.type == "Failed") |
@@ -84,15 +82,15 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.status.startTime // .status.completionTime // .metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do
JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE=$(( NOW - JOB_EPOCH ))
if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (failed ${AGE}s ago)"
AGE_H=$(( (NOW - JOB_EPOCH) / 3600 ))
if [ "$AGE_H" -ge 24 ]; then
echo "[delete] $NS/$NAME (failed ${AGE_H}h ago)"
kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
fi
done
echo ""
echo "=== Cleaning completed standalone Jobs older than 2m ==="
echo "=== Cleaning completed standalone Jobs older than 72h ==="
kubectl get jobs --all-namespaces -o json | \
jq -r '.items[] |
select(.status.succeeded >= 1) |
@@ -100,20 +98,20 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.status.completionTime // .metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do
JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE=$(( NOW - JOB_EPOCH ))
if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (completed ${AGE}s ago, no owner)"
AGE_H=$(( (NOW - JOB_EPOCH) / 3600 ))
if [ "$AGE_H" -ge 72 ]; then
echo "[delete] $NS/$NAME (completed ${AGE_H}h ago, no owner)"
kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
fi
done
echo ""
echo "=== Cleaning orphan Error/Failed/Evicted/Completed pods older than 2m ==="
echo "=== Cleaning orphan Error/Failed/Evicted pods older than 1h ==="
# Evicted pods show as Failed with reason Evicted
kubectl get pods --all-namespaces -o json | \
jq -r '.items[] |
select(
.status.phase == "Failed" or
.status.phase == "Succeeded" or
(.status.reason // "") == "Evicted" or
(.status.containerStatuses // [] | any(.state.terminated.reason == "Error"))
) |
@@ -121,9 +119,9 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do
POD_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE=$(( NOW - POD_EPOCH ))
if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (stale ${AGE}s ago)"
AGE_H=$(( (NOW - POD_EPOCH) / 3600 ))
if [ "$AGE_H" -ge 1 ]; then
echo "[delete] $NS/$NAME (error/evicted ${AGE_H}h ago)"
kubectl delete pod "$NAME" -n "$NS" --force 2>/dev/null || true
fi
done
@@ -1,25 +0,0 @@
# CiliumNetworkPolicy for kube-apiserver access.
#
# Standard K8s NetworkPolicy ipBlock rules don't work for the API server
# under Cilium — the except clause on 192.168.1.0/24 blocks the post-DNAT
# destination even when a separate rule re-allows a /32 or subnet.
#
# Cilium's native `kube-apiserver` entity resolves this correctly: it
# tracks the API server endpoints regardless of ClusterIP vs node-IP
# routing, so the policy stays valid across node changes and NAT paths.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: {{ .Release.Name }}-apiserver
namespace: {{ .Release.Namespace }}
spec:
endpointSelector:
matchLabels:
app: {{ .Release.Name }}
egress:
- toEntities:
- kube-apiserver
toPorts:
- ports:
- port: "6443"
protocol: TCP