Admin Bot
d521d7a48b
Clef-flash is a decision model, not a chat model. It serves Clef's own System One / Jev API at /v1/systemone: a state plus a schema of typed questions in, one probability per allowed option out, single prefill pass, no decoding and no free-form text. Router previously dispatched by body `model` only for /v1/chat/completions, /v1/embeddings and /v1/rerank. A request to /v1/systemone therefore fell through to the configured routes, found none, and answered "no routes configured". Adding the path to the dispatch predicate is the whole fix: routeByModel already sets upstreamPath := path, so the upstream receives /v1/systemone unchanged. There is deliberately no /rerank-style rewrite. Config: `ornith:35b` is removed with the Ollama predictor, and its GPU now serves clef-flash. `qwen2.5:3b-instruct` already pointed at qwen-cpu and is unchanged. Callers of ornith:35b must move to `reasoning`, which keeps its serving name while running Qwen3-30B-A3B. Tests cover body dispatch, unknown_model parity with the other three paths, and that the upstream path is not rewritten.
11 lines
6.8 KiB
YAML
11 lines
6.8 KiB
YAML
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: api-gateway-config
|
|
namespace: api
|
|
labels:
|
|
app: api-gateway
|
|
type: Opaque
|
|
stringData:
|
|
config.yaml: "# PRODUCTION GATEWAY CONFIGURATION\n# ==========================================\n# All upstream services MUST use Kubernetes internal service DNS names\n# Format: <service>.<namespace>.svc.cluster.local\n# \n# This ensures:\n# - Communication within cluster network only (no external IP exposure)\n# - Pod-to-pod service discovery via internal DNS\n# - Security policy enforcement at network level\n# - Service-level load balancing via kube-proxy\n#\n# Routing Pattern:\n# PREFERRED: X-Service header routing (e.g., X-Service: workflow)\n# Legacy: Path-based routing (e.g., /workflow) - being deprecated\n#\nauth:\n enabled: true\n # ZITADEL is the only identity provider. CoreDNS rewrites\n # zitadel.riotpiao.com to ingress-nginx, so discovery and JWKS stay inside\n # the cluster and TLS is the wildcard certificate.\n issuer: \"https://zitadel.riotpiao.com\"\n # zitadelDomain enables the official ZITADEL Go SDK path: OIDC discovery,\n # remote JWKS cache, and op.AccessTokenVerifier.\n zitadelDomain: \"zitadel.riotpiao.com\"\n # Audience allow-list: a caller token must carry one of these client IDs.\n # Add a consumer here before it may call the LLM models.\n zitadelClientIds:\n - \"396757800874672511\" # api-gw\n - \"396757803559027071\" # paperless (paperless-ai-agent)\n - \"396757803877794175\" # poimen\n - \"396757801210282367\" # argocd\n - \"396757804213403649\" # queue-sqs\n # Two spellings accepted. ZITADEL does store role keys with a colon (this\n # project holds iam:read, memory:write and llm:inference), so the colon form\n # is primary and the hyphen form is the legacy alias.\n requiredCapability: \"llm-inference,llm:inference\"\n # token exchange (client_credentials) against the same instance\n tokenUrl: \"https://zitadel.riotpiao.com/oauth/v2/token\"\n clientId: \"396757800874672511\"\n # jwksUrl MUST be set. cmd/gateway/main.go:144 builds the dispatcher\n # validator only when cfg.Auth.Enabled and cfg.Auth.JWKSURL are both set, so\n # an unset value leaves jwtValidator nil and every adapter request passes with\n # only a warning in the log. The SDK discovers the keys on its own path, but\n # the adapter dispatcher does not use that path.\n # In-cluster endpoint, not the public host. The public host resolves through\n # the CoreDNS rewrite to ingress-nginx and the fetch timed out there, so the\n # validator answered 403 with a context-deadline error on every caller. The\n # ZITADEL service answers JWKS on 8080 and the NetworkPolicy already allows\n # egress from this namespace to iam:8080.\n # The public host, not the service. The ZITADEL service port 8080 answers 404\n # for /oauth/v2/keys, while https://zitadel.riotpiao.com/oauth/v2/keys answers\n # 200 from a pod in this namespace.\n jwksUrl: \"https://zitadel.riotpiao.com/oauth/v2/keys\"\nroutes: []\nmodels:\n# All model services use internal Kubernetes DNS (llm-serving namespace)\n- name: \"reasoning\"\n address: \"reasoning-predictor.llm-serving.svc.cluster.local:80\"\n path: \"/v1/chat/completions\"\n# clef-flash is a DECISION model, not a chat model. It takes a `state` plus a\n# schema of typed `questions` and returns one probability per allowed option in\n# a single prefill pass. There is no decoding and no free-form text.\n#\n# Its request and response shapes are Clef's own (System One / Jev), NOT\n# OpenAI's: no `messages`, no `choices`. A standard OpenAI client cannot call\n# it, and the public path is /v1/systemone rather than /v1/chat/completions.\n#\n# The dispatcher resolves this model by the body's `model` field, then forwards\n# to the REQUEST path. The `path` value below records intent and must stay in\n# sync with the dispatch list in internal/proxy/router.go.\n- name: \"clef-flash\"\n address: \"clef-flash-predictor.llm-serving.svc.cluster.local:80\"\n path: \"/v1/systemone\"\n# `ornith:35b` retired: the Ollama Llama-35B predictor is gone and its GPU now\n# serves clef-flash. Callers must move to `reasoning`, which keeps the same\n# serving name while running Qwen3-30B-A3B on the same hardware.\n- name: \"qwen2.5:3b-instruct\"\n address: \"qwen-cpu.llm-serving.svc.cluster.local:80\"\n path: \"/v1/chat/completions\"\n- name: \"nomic-ai/nomic-embed-text-v2-moe\"\n address: \"embeddings-predictor.llm-serving.svc.cluster.local:80\"\n path: \"/v1/embeddings\"\n- name: \"BAAI/bge-reranker-base\"\n address: \"reranker-predictor.llm-serving.svc.cluster.local:80\"\n path: \"/v1/rerank\"\nadapters:\n- serviceName: sqs\n upstream:\n url: http://management-service.sqs.svc.cluster.local:9090\n timeoutSeconds: 30\n auth:\n required: true\n resources:\n - name: send-message\n methods:\n - verb: POST\n upstreamPath: /sqs/send\n - name: receive-message\n methods:\n - verb: POST\n upstreamPath: /sqs/receive\n - name: list-queues\n methods:\n - verb: GET\n upstreamPath: /sqs/queues\n- serviceName: workflow\n upstream:\n url: grpc://temporal-frontend.temporal.svc.cluster.local:7233\n timeoutSeconds: 60\n auth:\n required: true\n resources:\n - name: execute\n methods:\n - verb: POST\n upstreamPath: /temporal.api.workflowservice.v1.WorkflowService/ExecuteWorkflow\n - name: describe\n methods:\n - verb: GET\n upstreamPath: /temporal.api.workflowservice.v1.WorkflowService/DescribeWorkflowExecution\n - name: list\n methods:\n - verb: GET\n upstreamPath: /temporal.api.workflowservice.v1.WorkflowService/ListWorkflowExecutions\n- serviceName: memory\n upstream:\n url: http://poimen-memory.poimen.svc.cluster.local:8080\n timeoutSeconds: 30\n auth:\n required: true\n resources:\n - name: query\n methods:\n - verb: POST\n upstreamPath: /memory/query\n - name: ingest\n methods:\n - verb: POST\n upstreamPath: /memory/ingest\n - name: skills\n methods:\n - verb: GET\n upstreamPath: /memory/skills\n- serviceName: s3\n upstream:\n url: http://minio.storage.svc.cluster.local:80\n timeoutSeconds: 30\n auth:\n required: true\n resources:\n - name: list-objects\n methods:\n - verb: GET\n upstreamPath: /\n - name: get-object\n methods:\n - verb: GET\n upstreamPath: /\n - name: put-object\n methods:\n - verb: PUT\n upstreamPath: /\n- serviceName: iam\n upstream:\n url: http://zitadel.iam.svc.cluster.local:8080\n timeoutSeconds: 30\n auth:\n required: true\n resources:\n - name: list-roles\n methods:\n - verb: GET\n upstreamPath: /api/v3/roles\n - name: list-users\n methods:\n - verb: GET\n upstreamPath: /api/v3/users\n - name: create-role\n methods:\n - verb: POST\n upstreamPath: /api/v3/roles\n"
|