refactor(k8s): Reorganize into 5-layer structure with production kustomizations
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,299 @@
|
||||
# Ollama LLM Inference Service
|
||||
|
||||
CPU-only LLM inference server on talos-cp-1. Single model hot-loaded (DeepSeek-R1:70b), 42GB, 70Gi memory limit.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Access via port-forward
|
||||
```bash
|
||||
kubectl -n llm port-forward svc/ollama 11434:11434
|
||||
curl http://localhost:11434/api/tags
|
||||
```
|
||||
|
||||
### Debug pod (in-cluster)
|
||||
```bash
|
||||
kubectl run debug --rm -it -n llm --image=curlimages/curl \
|
||||
--labels="app.kubernetes.io/role=llm-debug" \
|
||||
--serviceaccount=llm-worker -- sh
|
||||
|
||||
# Inside pod
|
||||
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
|
||||
curl -H "Authorization: Bearer $TOKEN" \
|
||||
http://ollama.llm.svc.cluster.local:11434/api/tags
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
| Component | Value |
|
||||
|-----------|-------|
|
||||
| Service | ClusterIP `ollama.llm.svc.cluster.local:11434` |
|
||||
| Namespace | `llm` |
|
||||
| Node | talos-cp-1 (pinned via nodeAffinity) |
|
||||
| Memory request | 50Gi |
|
||||
| Memory limit | 70Gi |
|
||||
| Storage | 115Gi PVC (Longhorn) |
|
||||
| Model | `deepseek-r1:70b` (~42GB) |
|
||||
| Max loaded | 1 model |
|
||||
| Parallelism | 1 request at a time |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### List models
|
||||
```bash
|
||||
curl http://ollama.llm.svc.cluster.local:11434/api/tags
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"models": [
|
||||
{"name": "deepseek-r1:70b", "size": 42000000000, ...}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Generate (non-streaming)
|
||||
```bash
|
||||
curl -X POST http://ollama.llm.svc.cluster.local:11434/api/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "deepseek-r1:70b",
|
||||
"prompt": "Why is the sky blue?",
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
### Pull model
|
||||
```bash
|
||||
curl -X POST http://ollama.llm.svc.cluster.local:11434/api/pull \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name": "deepseek-r1:70b", "stream": false}'
|
||||
```
|
||||
|
||||
## Operations
|
||||
|
||||
### Check pod status
|
||||
```bash
|
||||
kubectl -n llm get pod -l app.kubernetes.io/name=ollama
|
||||
kubectl -n llm describe pod -l app.kubernetes.io/name=ollama
|
||||
```
|
||||
|
||||
### View logs
|
||||
```bash
|
||||
kubectl -n llm logs deployment/ollama -f
|
||||
```
|
||||
|
||||
### Monitor download progress (bootstrap)
|
||||
```bash
|
||||
kubectl -n llm logs -f job/bootstrap-models -c model-download
|
||||
```
|
||||
|
||||
### Restart deployment
|
||||
```bash
|
||||
kubectl -n llm rollout restart deployment/ollama
|
||||
```
|
||||
|
||||
## Storage
|
||||
|
||||
- **PVC:** `ollama-models-cache`, 115Gi, Longhorn StorageClass
|
||||
- **Mount:** `/root/.ollama/models` (Ollama model cache)
|
||||
- **Lifecycle:** RWO (Read-Write-Once), tied to talos-cp-1
|
||||
|
||||
### Resize PVC
|
||||
⚠️ PVCs can only expand, not shrink. Edit values.yaml and redeploy:
|
||||
|
||||
```yaml
|
||||
pvc:
|
||||
size: 120Gi # increase only
|
||||
```
|
||||
|
||||
```bash
|
||||
vsource .env && helmfile -f helmfile.yaml.gotmpl -l name=ollama apply
|
||||
```
|
||||
|
||||
## Networking
|
||||
|
||||
### NetworkPolicy
|
||||
- Default-deny ingress on Ollama pods
|
||||
- Allow from pods labeled `app.kubernetes.io/name: llm-worker` (port 11434)
|
||||
- Allow from pods labeled `app.kubernetes.io/role: llm-debug` (port 11434)
|
||||
|
||||
View policy:
|
||||
```bash
|
||||
kubectl -n llm get networkpolicy ollama
|
||||
```
|
||||
|
||||
Test access from external pod (should fail):
|
||||
```bash
|
||||
kubectl run test --rm -it --image=curlimages/curl -- \
|
||||
curl http://ollama.llm.svc.cluster.local:11434/
|
||||
# Connection timeout (correct)
|
||||
```
|
||||
|
||||
Test access from debug pod (should succeed):
|
||||
```bash
|
||||
kubectl -n llm logs job/bootstrap-models # verify bootstrap completed
|
||||
# Then run debug pod as shown above
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Helm values (`k8s/llm/charts/ollama/values.yaml`)
|
||||
|
||||
```yaml
|
||||
resources:
|
||||
requests:
|
||||
cpu: 8
|
||||
memory: 50Gi
|
||||
limits:
|
||||
cpu: 16
|
||||
memory: 70Gi
|
||||
|
||||
env:
|
||||
OLLAMA_MAX_LOADED_MODELS: "1"
|
||||
OLLAMA_NUM_PARALLEL: "1"
|
||||
OLLAMA_MAX_QUEUE: "32"
|
||||
OLLAMA_KEEP_ALIVE: "-1"
|
||||
OLLAMA_HOST: "0.0.0.0:11434"
|
||||
|
||||
preloadJob:
|
||||
enabled: true
|
||||
hotModels:
|
||||
- deepseek-r1:70b
|
||||
```
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Value | Purpose |
|
||||
|----------|-------|---------|
|
||||
| `OLLAMA_MODELS` | `/root/.ollama/models` | Model cache dir |
|
||||
| `OLLAMA_MAX_LOADED_MODELS` | `1` | Max concurrent models in RAM |
|
||||
| `OLLAMA_NUM_PARALLEL` | `1` | Parallel request threads |
|
||||
| `OLLAMA_MAX_QUEUE` | `32` | Request queue depth |
|
||||
| `OLLAMA_KEEP_ALIVE` | `-1` | Keep model resident (never unload) |
|
||||
| `OLLAMA_HOST` | `0.0.0.0:11434` | Bind address |
|
||||
|
||||
Tune `OLLAMA_NUM_PARALLEL` based on CPU cores. Current: 1 (conservative, CPU bottleneck).
|
||||
|
||||
## Model Management
|
||||
|
||||
### Current model
|
||||
- **Name:** `deepseek-r1:70b`
|
||||
- **Size:** ~42GB
|
||||
- **Quantization:** Default Ollama quant
|
||||
- **Status:** Downloaded during pod init via bootstrap job
|
||||
|
||||
### Change model
|
||||
|
||||
1. Edit `values.yaml`:
|
||||
```yaml
|
||||
preloadJob:
|
||||
hotModels:
|
||||
- deepseek-r1:32b # or any available model
|
||||
```
|
||||
|
||||
2. Redeploy:
|
||||
```bash
|
||||
kubectl -n llm delete job bootstrap-models --ignore-not-found
|
||||
vsource .env && helmfile -f helmfile.yaml.gotmpl -l name=ollama apply
|
||||
```
|
||||
|
||||
3. Monitor:
|
||||
```bash
|
||||
kubectl -n llm logs -f job/bootstrap-models -c model-download
|
||||
```
|
||||
|
||||
### Available models
|
||||
Ollama registry: https://ollama.com/library
|
||||
|
||||
Examples:
|
||||
- `deepseek-r1:70b` (reasoning, 42GB)
|
||||
- `deepseek-r1:32b` (faster, 20GB)
|
||||
- `llama3.1:70b` (general, 41GB)
|
||||
- `mistral:large` (26GB)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Pod stuck in `ContainerCreating`
|
||||
```bash
|
||||
kubectl -n llm describe pod -l app.kubernetes.io/name=ollama
|
||||
# Check Events section for PVC/image pull issues
|
||||
```
|
||||
|
||||
### Bootstrap job failing
|
||||
```bash
|
||||
kubectl -n llm logs job/bootstrap-models -c model-download --tail=50
|
||||
# Common: model not found in registry, disk full, network timeout
|
||||
```
|
||||
|
||||
### Model pull timeout
|
||||
```bash
|
||||
# Increase pod timeout (edit deployment directly)
|
||||
kubectl -n llm edit deployment ollama
|
||||
# Change readinessProbe.initialDelaySeconds, livenessProbe.periodSeconds
|
||||
```
|
||||
|
||||
### Out of memory
|
||||
Model size exceeds limit. Reduce `memory.limits` or choose smaller model.
|
||||
|
||||
```bash
|
||||
kubectl top pod -n llm # check actual usage
|
||||
```
|
||||
|
||||
### Cannot connect from other pods
|
||||
Verify NetworkPolicy:
|
||||
```bash
|
||||
kubectl -n llm get networkpolicy
|
||||
kubectl -n llm describe networkpolicy ollama
|
||||
# Add pod label: app.kubernetes.io/name: llm-worker or app.kubernetes.io/role: llm-debug
|
||||
```
|
||||
|
||||
## Secrets
|
||||
|
||||
Ollama pod receives MinIO credentials via Secret `ollama-minio` (created by helmfile presync):
|
||||
|
||||
```bash
|
||||
kubectl -n llm get secret ollama-minio -o jsonpath='{.data}' | jq
|
||||
```
|
||||
|
||||
Keys: `endpoint`, `bucket`, `access_key`, `secret_key`
|
||||
|
||||
Used by bootstrap job to upload model blobs to MinIO (future: auto-backup).
|
||||
|
||||
## Metrics & Observability
|
||||
|
||||
### Prometheus scrape (if enabled)
|
||||
ServiceMonitor: Not yet configured (see `k8s/monitoring/dashboards/services/`)
|
||||
|
||||
Metrics to add:
|
||||
- `ollama_requests_total` (counter)
|
||||
- `ollama_request_duration_seconds` (histogram)
|
||||
- `ollama_loaded_models` (gauge)
|
||||
|
||||
### Logs
|
||||
Pod logs via kubectl. No log aggregation to Loki yet.
|
||||
|
||||
```bash
|
||||
kubectl -n llm logs deployment/ollama -f --timestamps
|
||||
```
|
||||
|
||||
## Cleanup
|
||||
|
||||
### Delete Ollama completely
|
||||
```bash
|
||||
vsource .env && helmfile -f helmfile.yaml.gotmpl -l name=ollama destroy
|
||||
# Keeps PVC (data safety). To delete: kubectl -n llm delete pvc ollama-models-cache
|
||||
```
|
||||
|
||||
### Delete just the model cache (keep deployment)
|
||||
```bash
|
||||
kubectl -n llm delete pvc ollama-models-cache
|
||||
# Recreate: kubectl -n llm patch deployment ollama -p '{"spec":{"template":{"metadata":{"annotations":{"restart":"now"}}}}}'
|
||||
```
|
||||
|
||||
## See Also
|
||||
|
||||
- Helmfile: `helmfile.yaml.gotmpl` (llm release block)
|
||||
- Chart: `k8s/llm/charts/ollama/`
|
||||
- Namespace: `llm`
|
||||
- Bootstrap: `k8s/llm/bootstrap-models-job.yaml` (manual preload fallback)
|
||||
@@ -0,0 +1,6 @@
|
||||
apiVersion: v2
|
||||
name: ollama
|
||||
description: CPU-only Ollama LLM server with MinIO model registry
|
||||
type: application
|
||||
version: 0.1.0
|
||||
appVersion: "latest"
|
||||
@@ -0,0 +1,126 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ollama
|
||||
namespace: llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
app.kubernetes.io/part-of: llm
|
||||
spec:
|
||||
replicas: {{ .Values.replicaCount }}
|
||||
strategy:
|
||||
type: Recreate
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: ollama
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
app.kubernetes.io/part-of: llm
|
||||
spec:
|
||||
affinity:
|
||||
nodeAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
nodeSelectorTerms:
|
||||
- matchExpressions:
|
||||
- key: topology.kubernetes.io/zone
|
||||
operator: In
|
||||
values:
|
||||
- {{ .Values.nodeAffinity.zone }}
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Equal
|
||||
value: ""
|
||||
effect: NoSchedule
|
||||
|
||||
initContainers:
|
||||
- name: preload-model
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo "Starting Ollama server for model preload..."
|
||||
ollama serve &
|
||||
OLLAMA_PID=$!
|
||||
sleep 10
|
||||
{{- range .Values.preloadJob.hotModels }}
|
||||
echo "Preloading {{ . }}..."
|
||||
if ollama ls | grep -q "{{ . }}"; then
|
||||
echo "✓ {{ . }} already cached"
|
||||
else
|
||||
ollama pull {{ . }}
|
||||
fi
|
||||
{{- end }}
|
||||
echo "Model preload complete"
|
||||
kill $OLLAMA_PID || true
|
||||
wait $OLLAMA_PID 2>/dev/null || true
|
||||
volumeMounts:
|
||||
- name: models-cache
|
||||
mountPath: /root/.ollama/models
|
||||
env:
|
||||
- name: OLLAMA_HOST
|
||||
value: "127.0.0.1:11434"
|
||||
|
||||
containers:
|
||||
- name: ollama
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
ports:
|
||||
- containerPort: 11434
|
||||
name: http
|
||||
env:
|
||||
{{- range $key, $value := .Values.env }}
|
||||
- name: {{ $key }}
|
||||
value: "{{ $value }}"
|
||||
{{- end }}
|
||||
- name: OLLAMA_MODELS_MINIO_ENDPOINT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ollama-minio
|
||||
key: endpoint
|
||||
- name: OLLAMA_MODELS_MINIO_BUCKET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ollama-minio
|
||||
key: bucket
|
||||
- name: OLLAMA_MODELS_MINIO_ACCESS_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ollama-minio
|
||||
key: access_key
|
||||
- name: OLLAMA_MODELS_MINIO_SECRET_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ollama-minio
|
||||
key: secret_key
|
||||
resources:
|
||||
requests:
|
||||
cpu: {{ .Values.resources.requests.cpu }}
|
||||
memory: {{ .Values.resources.requests.memory }}
|
||||
limits:
|
||||
cpu: {{ .Values.resources.limits.cpu }}
|
||||
memory: {{ .Values.resources.limits.memory }}
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 11434
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 11434
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 5
|
||||
volumeMounts:
|
||||
- name: models-cache
|
||||
mountPath: /root/.ollama/models
|
||||
|
||||
volumes:
|
||||
- name: models-cache
|
||||
persistentVolumeClaim:
|
||||
claimName: ollama-models-cache
|
||||
@@ -0,0 +1,28 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: ollama-access
|
||||
namespace: llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: ollama
|
||||
policyTypes:
|
||||
- Ingress
|
||||
ingress:
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: llm-worker
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 11434
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/role: llm-debug
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 11434
|
||||
@@ -0,0 +1,92 @@
|
||||
{{- if .Values.preloadJob.enabled }}
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: ollama-preload
|
||||
namespace: llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama-preload
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: default
|
||||
restartPolicy: Never
|
||||
affinity:
|
||||
nodeAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
nodeSelectorTerms:
|
||||
- matchExpressions:
|
||||
- key: topology.kubernetes.io/zone
|
||||
operator: In
|
||||
values:
|
||||
- az-a
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Equal
|
||||
effect: NoSchedule
|
||||
initContainers:
|
||||
- name: model-cache-init
|
||||
image: ollama/ollama:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo "Starting Ollama server to cache models..."
|
||||
ollama serve &
|
||||
OLLAMA_PID=$!
|
||||
sleep 10
|
||||
echo "Caching hot-tier models..."
|
||||
{{- range .Values.preloadJob.hotModels }}
|
||||
echo "Checking if {{ . }} is cached..."
|
||||
if ollama ls | grep -q "{{ . }}"; then
|
||||
echo "✓ {{ . }} already cached, skipping"
|
||||
else
|
||||
echo "Pulling {{ . }}..."
|
||||
ollama pull {{ . }}
|
||||
fi
|
||||
{{- end }}
|
||||
echo "Model cache initialization complete"
|
||||
kill $OLLAMA_PID || true
|
||||
wait $OLLAMA_PID 2>/dev/null || true
|
||||
volumeMounts:
|
||||
- name: models
|
||||
mountPath: /root/.ollama
|
||||
env:
|
||||
- name: OLLAMA_HOST
|
||||
value: "127.0.0.1:11434"
|
||||
|
||||
containers:
|
||||
- name: cache-populate
|
||||
image: curlimages/curl:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo "Waiting for Ollama pod to be ready..."
|
||||
until curl -f http://ollama.llm.svc.cluster.local:11434/api/tags 2>/dev/null; do
|
||||
echo "Ollama not ready, waiting..."
|
||||
sleep 5
|
||||
done
|
||||
echo "Ollama is ready, populating local cache..."
|
||||
{{- range .Values.preloadJob.hotModels }}
|
||||
echo "Checking if {{ . }} is already cached..."
|
||||
if curl -s http://ollama.llm.svc.cluster.local:11434/api/tags | grep -q "{{ . }}"; then
|
||||
echo "✓ {{ . }} already cached, skipping"
|
||||
else
|
||||
echo "Caching {{ . }} locally..."
|
||||
curl -X POST http://ollama.llm.svc.cluster.local:11434/api/pull \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"{{ . }}","stream":false}'
|
||||
fi
|
||||
{{- end }}
|
||||
echo "Local cache population complete"
|
||||
|
||||
volumes:
|
||||
- name: models
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,14 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: ollama-models-cache
|
||||
namespace: llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: {{ .Values.pvc.storageClassName }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.pvc.size }}
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: ollama
|
||||
namespace: llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
ports:
|
||||
- port: {{ .Values.service.port }}
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app.kubernetes.io/name: ollama
|
||||
@@ -0,0 +1,12 @@
|
||||
apiVersion: storage.k8s.io/v1
|
||||
kind: StorageClass
|
||||
metadata:
|
||||
name: longhorn-llm
|
||||
labels:
|
||||
app.kubernetes.io/name: ollama
|
||||
provisioner: driver.longhorn.io
|
||||
parameters:
|
||||
numberOfReplicas: "1"
|
||||
staleReplicaTimeout: "2880"
|
||||
reclaimPolicy: Retain
|
||||
allowVolumeExpansion: true
|
||||
@@ -0,0 +1,40 @@
|
||||
replicaCount: 1
|
||||
|
||||
image:
|
||||
repository: ollama/ollama
|
||||
pullPolicy: IfNotPresent
|
||||
tag: "latest"
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
port: 11434
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 8
|
||||
memory: 60Gi
|
||||
limits:
|
||||
cpu: 16
|
||||
memory: 100Gi
|
||||
|
||||
pvc:
|
||||
enabled: true
|
||||
size: 115Gi
|
||||
storageClassName: longhorn-llm
|
||||
|
||||
nodeAffinity:
|
||||
zone: az-a
|
||||
|
||||
env:
|
||||
OLLAMA_MODELS: /root/.ollama/models
|
||||
OLLAMA_MAX_LOADED_MODELS: "2"
|
||||
OLLAMA_NUM_PARALLEL: "2"
|
||||
OLLAMA_MAX_QUEUE: "64"
|
||||
OLLAMA_KEEP_ALIVE: "-1"
|
||||
OLLAMA_HOST: "0.0.0.0:11434"
|
||||
|
||||
preloadJob:
|
||||
enabled: false
|
||||
hotModels:
|
||||
- ornith:35b
|
||||
- deepseek-r1:70b
|
||||
@@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: llm
|
||||
resources:
|
||||
- scripts/
|
||||
# Helm charts deployed via ArgoCD Helm source
|
||||
@@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources: []
|
||||
# Shell scripts, not K8s manifests
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/bin/bash
|
||||
|
||||
# setup-minio-bucket.sh
|
||||
# Creates MinIO bucket and Kubernetes secrets for Ollama LLM server
|
||||
# Runs as helmfile presync hook; all commands are idempotent
|
||||
# Dependencies: kubectl, access to minio-az-a pod in storage namespace
|
||||
# Environment: MINIO_ROOT_USER, MINIO_ROOT_PASSWORD (from Vault), AUTHENTIK_OLLAMA_CLIENT_ID, AUTHENTIK_OLLAMA_CLIENT_SECRET
|
||||
|
||||
set -e
|
||||
|
||||
echo "=== Step 1: Create and label llm namespace ==="
|
||||
kubectl create namespace llm --dry-run=client -o yaml | kubectl apply -f -
|
||||
kubectl label namespace llm \
|
||||
pod-security.kubernetes.io/enforce=baseline \
|
||||
pod-security.kubernetes.io/enforce-version=latest \
|
||||
--overwrite
|
||||
|
||||
echo "✓ llm namespace created/labeled"
|
||||
|
||||
echo ""
|
||||
echo "=== Step 2: Create MinIO bucket riotpiao-models ==="
|
||||
|
||||
# Configure mc host inside MinIO pod
|
||||
kubectl -n storage exec deployment/minio-az-a -- \
|
||||
mc config host add local http://localhost:9000 \
|
||||
"${MINIO_ROOT_USER}" "${MINIO_ROOT_PASSWORD}"
|
||||
|
||||
echo "✓ mc host configured"
|
||||
|
||||
# Create bucket (idempotent)
|
||||
kubectl -n storage exec deployment/minio-az-a -- \
|
||||
mc mb --ignore-existing local/riotpiao-models
|
||||
|
||||
echo "✓ MinIO bucket riotpiao-models created (or already exists)"
|
||||
|
||||
# Enable versioning for model rollback safety
|
||||
kubectl -n storage exec deployment/minio-az-a -- \
|
||||
mc version enable local/riotpiao-models
|
||||
|
||||
echo "✓ Versioning enabled on riotpiao-models bucket"
|
||||
|
||||
echo ""
|
||||
echo "=== Step 3: Create ollama-minio Secret (MinIO credentials) ==="
|
||||
|
||||
kubectl create secret generic ollama-minio -n llm \
|
||||
--from-literal=endpoint="http://minio-az-a.storage:9000" \
|
||||
--from-literal=bucket="riotpiao-models" \
|
||||
--from-literal=access_key="${MINIO_ROOT_USER}" \
|
||||
--from-literal=secret_key="${MINIO_ROOT_PASSWORD}" \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
echo "✓ Secret ollama-minio created/updated"
|
||||
|
||||
echo ""
|
||||
echo "=== Step 4: Create ollama-oidc Secret (Authentik credentials) ==="
|
||||
|
||||
kubectl create secret generic ollama-oidc -n llm \
|
||||
--from-literal=client_id="${AUTHENTIK_OLLAMA_CLIENT_ID}" \
|
||||
--from-literal=client_secret="${AUTHENTIK_OLLAMA_CLIENT_SECRET}" \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
echo "✓ Secret ollama-oidc created/updated"
|
||||
|
||||
echo ""
|
||||
echo "=== Verification ==="
|
||||
echo ""
|
||||
echo "Run these commands to verify:"
|
||||
echo " kubectl -n llm get secret ollama-minio ollama-oidc"
|
||||
echo " kubectl -n storage exec deployment/minio-az-a -- mc ls local/riotpiao-models"
|
||||
echo ""
|
||||
echo "Setup complete!"
|
||||
Reference in New Issue
Block a user