riotpiao-poimen/homelab
2
0
Fork 0
Code Issues 23 Pull Requests 1 Actions Packages Projects 1 Releases Wiki Activity

[Infra] Local PV on NVMe for model weight cold starts #40

New Issue
Open
opened 2026-09-11 01:27:34 +00:00 by poimen · 0 comments
poimen commented 2026-09-11 01:27:34 +00:00
Member
Copy Link
Copy Source

Downloading model weights from remote creates minutes of pod init lag. ornith pulls 35B model on every restart (ollama pull). Use Local PV on NVMe SSD with pre-cached weights. OLLAMA_MODELS=/mnt/models already set but storage is longhorn (network). Files: k8s/apps/llm-serving/ornith.yaml, k8s/infra/longhorn/

Downloading model weights from remote creates minutes of pod init lag. ornith pulls 35B model on every restart (ollama pull). Use Local PV on NVMe SSD with pre-cached weights. OLLAMA_MODELS=/mnt/models already set but storage is longhorn (network). Files: k8s/apps/llm-serving/ornith.yaml, k8s/infra/longhorn/
poimen added this to the LLM Production Hardening milestone 2026-09-11 01:27:34 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/auth
Authentication & authorization
 area/authentik
Authentik OIDC provider config
 area/downstream
Downstream service changes
 area/event-bus
Event routing and notification pipeline
 area/gateway
API gateway code (homelab-frontend)
 area/notifications
Gotify, SMTP, push notifications
 area/observer
Poimen goal-driven observation and pattern detection
 priority/high
Must have
 priority/medium
Should have
 status/blocked
Blocked by dependency
 status/done
Completed
 status/in-progress
Currently being worked on
 status/review
Ready for review
 status/todo
Not started
 type/docs
Documentation
 type/feature
New feature
 type/infra
Infrastructure change
No labels
Milestone
No items
No Milestone LLM Production Hardening
Projects
Clear projects
No projects
Assignees
Clear assignees
poimen rock (Rock)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: riotpiao-poimen/homelab#40