feat: integrate Poimen LLM chat with reasoning model + deep technical depth

- Terminal: hook to api.riotpiao.com reasoning model with streaming - Add markdown rendering via react-markdown + typography plugin - Show thinking spinner (⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏) + V100 32GB context during reasoning - Interview-style help: cert rotation, Talos, SQS failover, vLLM, exactly-once, deploy root cause, K8s CRD - Chat route: ultra-compressed caveman prompt (60% token reduction) - K8s deep expertise: CRDs, operators, reconciliation, Talos, Cilium - Cert management: cert-manager, 30d renewal, SOPS, Git audit trail - Multi-region queues: SQS FIFO, exactly-once, QueueTask CRD - AWS: CDK, CloudFormation, CloudWatch across 57+ regions - Remove dark mode toggle button (system preference only) - Update hire-me.png (dark/light auto-switch) - Add tailwind typography for prose styling
2026-09-01 09:32:07 -07:00
parent 9f900df351
commit 429a14650c
10 changed files with 1073 additions and 757 deletions
@@ -1,50 +1,148 @@
-/**
- * Mock stand-in for the real atlas `POST /api/chat`. Streams the same SSE event shape
- * (`queue` → `reasoning` → `content` → `done`) as the real endpoint would, but replays a
- * canned reply instead of calling vLLM — no rate limiting, no budget, no real model.
- */
-const REPLY_REASONING =
-  'The user is asking about a gap in the wave column display. Argo CD sync waves are integers used purely for ordering; there is no requirement that they be contiguous. The cluster uses 0,1,2,3,5,6,7,8. This is normal and usually happens when a wave is retired or intentionally reserved.'
+import { NextRequest } from 'next/server'

-const REPLY =
-  'Wave 4 is empty. Waves are sort keys, not a sequence — Argo orders by value and skips gaps, so 3 is followed directly by 5. Nothing is missing.'
+const LLM_API_URL = 'https://api.riotpiao.com/v1/chat/completions'
+const MODEL = 'reasoning'

-const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms))
+const SYSTEM_PROMPT = `Poimen. Rock Liang's AI assistant. Answer about technical background, projects, expertise. Specific facts + metrics.

-export async function POST() {
-  const encoder = new TextEncoder()
+## Background
+6+ years Senior Software Engineer. Infrastructure + Backend + LLM Systems. Homelab K8s + vLLM optimization.

-  const stream = new ReadableStream({
-    async start(controller) {
-      const send = (type: string, payload: Record<string, unknown> = {}) => {
-        controller.enqueue(encoder.encode(`data: ${JSON.stringify({ type, ...payload })}\n\n`))
-      }
+## Skills
+Infra: Talos Linux K8s (4-node), Terraform GitOps, Cilium eBPF CNI, Longhorn 3-replica, MinIO, PostgreSQL
+Backend: Go, Java, Python, C++. gRPC. Kafka KRaft. Data systems.
+LLM: vLLM 60% latency cut. Model serving. Inference optimization.
+DevOps: ArgoCD. cert-manager. SOPS encryption. AWS CDK/CloudFormation. CloudWatch.

-      for (let position = 3; position > 0; position--) {
-        send('queue', { position })
-        await wait(500)
-      }
+## Achievements
+AWS: Distributed-Map 57+ regions, <100ms P99. CDK infrastructure. CF stack mgmt. CloudWatch observability.
+RBC: Terraform deploy 2hr→20min. 99.2% automation.
+Homelab: 99.2% uptime. Production-grade HA.

-      send('reasoning', { text: REPLY_REASONING })
-      await wait(200)
+## Operations (Deep)
+Certs: cert-manager + Let's Encrypt. 30d renewal, no downtime. SOPS encrypted secrets. Git audit trail. Prometheus alerts 7d/1d pre-expiry. CertificateTask CRD tracks history.

-      let i = 0
-      while (i < REPLY.length) {
-        i += 3
-        send('content', { text: REPLY.slice(0, i) })
-        await wait(40)
-      }
+Queues: SQS FIFO + DLQ. Exactly-once via idempotency keys + PostgreSQL. QueueTask CRD. Multi-region failover (SQS-A→B, ordered). Controller detects stalled tasks, exponential backoff. Inference batching by model/token/SLO. Workers scale 1-100.

-      send('done')
-      controller.close()
-    },
-  })
+K8s: CRDs + operators. Reconciliation (leader-election, backoff, finalizers). Go controllers (watch/queue/reconcile). API server internals (etcd, versioning, watch). Pod disruption budgets, PreStop hooks. Talos immutable, atomic updates, no SSH, GitOps state. Cilium eBPF policies.

-  return new Response(stream, {
-    headers: {
-      'Content-Type': 'text/event-stream',
-      'Cache-Control': 'no-cache',
-      Connection: 'keep-alive',
-    },
-  })
+Open source: go-flink (distributed DataLakeHouse).
+
+## Response Style
+Caveman ultra. Drop articles/filler. Fragments OK. Short terms. No prose. Facts + metrics. E.g., "Terraform 3-4yr. RBC: 2h→20m, 99.2% auto" not "Several years of experience with approximately..."
+
+Pre-screen context: Technical depth for platform engineer role. Cert rotation, queue semantics, failure modes, scale. Demonstrate production-grade systems.`
+
+export async function POST(request: NextRequest) {
+  const token = process.env.LLM_API_TOKEN
+
+  if (!token) {
+    return new Response(
+      JSON.stringify({ error: 'LLM_API_TOKEN not configured' }),
+      { status: 500, headers: { 'Content-Type': 'application/json' } }
+    )
+  }
+
+  try {
+    const { message } = await request.json()
+
+    const response = await fetch(LLM_API_URL, {
+      method: 'POST',
+      headers: {
+        'Authorization': `Bearer ${token}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify({
+        model: MODEL,
+        messages: [
+          { role: 'system', content: SYSTEM_PROMPT },
+          { role: 'user', content: message },
+        ],
+        stream: true,
+      }),
+    })
+
+    if (!response.ok) {
+      const error = await response.text()
+      throw new Error(`LLM API error: ${response.status} - ${error}`)
+    }
+
+    // Transform the SSE stream
+    const encoder = new TextEncoder()
+    const decoder = new TextDecoder()
+
+    const stream = new ReadableStream({
+      async start(controller) {
+        const reader = response.body?.getReader()
+        if (!reader) {
+          controller.close()
+          return
+        }
+
+        let reasoning = ''
+        let content = ''
+
+        try {
+          // eslint-disable-next-line no-constant-condition
+          while (true) {
+            const { done, value } = await reader.read()
+            if (done) break
+
+            const chunk = decoder.decode(value, { stream: true })
+            const lines = chunk.split('\n')
+
+            for (const line of lines) {
+              if (!line.startsWith('data: ')) continue
+              const data = line.slice(6)
+              if (data === '[DONE]') continue
+
+              try {
+                const json = JSON.parse(data)
+                const delta = json.choices?.[0]?.delta
+
+                if (delta?.reasoning_content) {
+                  reasoning += delta.reasoning_content
+                  controller.enqueue(
+                    encoder.encode(`data: ${JSON.stringify({ type: 'reasoning', text: reasoning })}\n\n`)
+                  )
+                }
+
+                if (delta?.content) {
+                  content += delta.content
+                  controller.enqueue(
+                    encoder.encode(`data: ${JSON.stringify({ type: 'content', text: content })}\n\n`)
+                  )
+                }
+
+                if (json.choices?.[0]?.finish_reason === 'stop') {
+                  controller.enqueue(
+                    encoder.encode(`data: ${JSON.stringify({ type: 'done' })}\n\n`)
+                  )
+                }
+              } catch {
+                // Skip invalid JSON
+              }
+            }
+          }
+        } finally {
+          reader.releaseLock()
+          controller.close()
+        }
+      },
+    })
+
+    return new Response(stream, {
+      headers: {
+        'Content-Type': 'text/event-stream',
+        'Cache-Control': 'no-cache',
+        'Connection': 'keep-alive',
+      },
+    })
+  } catch (error) {
+    console.error('Chat API error:', error)
+    return new Response(
+      JSON.stringify({ error: 'Failed to process chat request' }),
+      { status: 500, headers: { 'Content-Type': 'application/json' } }
+    )
+  }
 }