riotpiao-poimen/homelab-frontend
2
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gateway): route clef-flash to clef-serve #74

Merged
rock merged 1 commits from tmp-clef-route into main 2026-10-30 06:06:49 +00:00
Conversation 0 Commits 1 Files Changed 1
+1 -1
rock commented 2026-10-30 06:06:41 +00:00
Owner
Copy Link
Copy Source

Clef could not run under SGLang on V100 (needs CUDA 13, which dropped Volta kernels). It runs under plain transformers in fp16 on two cards, serving the model own systemone() endpoint.

Clef could not run under SGLang on V100 (needs CUDA 13, which dropped Volta kernels). It runs under plain transformers in fp16 on two cards, serving the model own systemone() endpoint.
rock added 1 commit 2026-10-30 06:06:42 +00:00
feat(gateway): route clef-flash to clef-serve
CI / CI (pull_request) Successful in 2m16s
Details
affb1caa63 
Restores the route removed when the model could not run. It runs now, on a different
runtime: SGLang needs v0.5.22+ which upstream publishes only on CUDA 13, and CUDA 13
dropped Volta kernels. clef-serve loads the same weights through plain transformers in
fp16, sharded over two V100s, and serves the model's own systemone() on /v1/systemone.

Weights measured at 17.75 GiB in BF16; fp16 costs the same bytes per parameter.
rock merged commit 1f906e95d8 into main 2026-10-30 06:06:49 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
area/auth
Authentication & authorization
 area/authentik
Authentik OIDC provider config
 area/downstream
Downstream service changes
 area/gateway
API gateway code
 priority/high
Must have
 priority/low
Nice to have
 priority/medium
Should have
 status/blocked
Blocked by dependency
 status/done
Completed
 status/in-progress
Currently being worked on
 status/review
Ready for review
 status/todo
Not started
 type/docs
Documentation
 type/feature
New feature
 type/infra
Infrastructure change
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
poimen rock (Rock)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: riotpiao-poimen/homelab-frontend#74