rock/poimen-memory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: remove knowledge/ from git tracking

Browse Source 
Knowledge lives in memory service (pgvector/OpenSearch) and vault,
not in git. Source markdown is ephemeral input to mem learn.
This commit is contained in:
rock
2026-08-29 22:50:23 -07:00
parent a412237095
commit 054386ca07
12 changed files with 1 additions and 1252 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -16,3 +16,4 @@ vault/
log/ log/
# tasks/ - Task board and acceptance criteria # tasks/ - Task board and acceptance criteria
CLAUDE.md CLAUDE.md
knowledge/
knowledge/andrej-karpathy.md
-55
View File
@@ -1,55 +0,0 @@
# Andrej Karpathy — Key Insights & Practices
## Software 2.0
- Traditional software (1.0): explicit rules written by programmers.
- Software 2.0: behavior learned from data via neural networks. Code = weights.
- Implication: datasets are the new source code. Data curation > clever algorithms.
- Debug by inspecting data, not stepping through logic.
## Training Neural Networks — A Recipe
1. **Become one with the data** — visualize, understand distributions, find patterns and anomalies before writing any model code.
2. **Set up end-to-end training/eval skeleton** — simplest possible model first. Get the pipeline working.
3. **Overfit first** — if model can't memorize a single batch, architecture is wrong.
4. **Regularize** — only add dropout, weight decay, augmentation after overfitting confirmed.
5. **Tune** — learning rate is the most important hyperparameter. Use LR finder.
6. **Squeeze** — ensembles, larger models, more data. Diminishing returns here.
## Most Common Neural Net Mistakes
- Not looking at data first.
- Forgetting to set model to eval mode (BatchNorm, Dropout change behavior).
- Forgetting to zero gradients.
- Using softmax with cross-entropy (use logits directly).
- Not normalizing inputs.
- Applying augmentation to validation set.
- Silent shape broadcasting bugs — always assert tensor shapes.
## LLM Insights (Post-GPT Era)
- LLMs are "operating systems" — CPU is the transformer, context window is RAM, training data is disk.
- Tokenization is a key bottleneck — BPE artifacts cause many failure modes.
- Temperature controls creativity vs precision. T=0 for factual, T>0 for creative.
- Chain-of-thought works because it gives the model "working memory" in the output tokens.
- Prompt engineering is programming in natural language. Be explicit, give examples.
## Build Nanograd / Micrograd Philosophy
- Understand backpropagation by implementing it from scratch.
- A neural net is just: forward pass → compute loss → backward pass → update weights.
- Autograd: track operations, build computation graph, reverse-mode differentiation.
- Every complex framework (PyTorch, JAX) is built on these same primitives.
## Practical ML Engineering
- Start simple: logistic regression baseline before deep learning.
- Measure everything: loss curves, gradient norms, weight distributions.
- Reproducibility: fix seeds, log hyperparameters, version datasets.
- Don't trust your code — trust your loss curve. If loss isn't going down, something is wrong.
- Data quality > model complexity. 10x data often beats 10x model size.
## Scaling Laws
- Performance scales predictably with compute, data, and parameters (Chinchilla scaling).
- Compute-optimal training: balance model size and training tokens.
- Emergent abilities appear at scale — capabilities that don't exist in smaller models.
## On AI Engineering
- The best AI engineers understand both ML and systems engineering.
- Inference optimization matters as much as training — quantization, batching, KV-cache.
- Eval is everything. If you can't measure it, you can't improve it.
- Build evaluation suites before building features.
knowledge/ast-grep.md
-78
View File
@@ -1,78 +0,0 @@
# AST-Grep (sg) — Structural Code Search & Transform
## Core Concept
- AST-grep searches/transforms code using Abstract Syntax Tree patterns, not regex.
- Pattern matches structural meaning, ignoring whitespace, comments, formatting.
- Works across: Rust, Go, Python, JS/TS, Java, C, C++, Ruby, Kotlin, Lua, CSS, HTML.
## CLI Usage
- `sg --pattern 'unwrap()' -l rust` — find all `.unwrap()` calls in Rust files.
- `sg --pattern 'println!($$$ARGS)' -l rust` — find all println macros with any args.
- `sg --pattern '$A.unwrap()' --rewrite '$A.expect("TODO")' -l rust` — rewrite unwrap to expect.
- `sg scan` — run lint rules from `sgconfig.yml`.
- `sg test` — test rules against fixtures.
## Pattern Syntax
- `$VAR` matches single AST node (identifier, expression, etc).
- `$$$VARS` matches zero or more nodes (variadic).
- `$$VAR` matches zero or one node (optional).
- Literal code matches itself: `if true { $$$BODY }` matches any `if true` block.
## Meta Variables
- `$A` in pattern captures node, available in `--rewrite` as `$A`.
- Named captures: same name must match same content. `$A == $A` matches `x == x` but not `x == y`.
- `$_` is anonymous — matches anything without capturing.
## Rule YAML Format
```yaml
id: no-unwrap
language: rust
rule:
pattern: $A.unwrap()
not:
inside:
kind: test_function
fix: $A.expect("handle error")
message: "Use .expect() instead of .unwrap() in production code"
severity: warning
```
## Composite Rules
- `all: [rule1, rule2]` — both must match.
- `any: [rule1, rule2]` — either matches.
- `not: rule` — negation.
- `matches: rule-id` — reference another rule.
- `inside: { kind: function_item }` — must be inside a function.
- `has: { pattern: $EXPR }` — must contain sub-pattern.
- `follows: { pattern: ... }` — must follow another pattern.
- `precedes: { pattern: ... }` — must precede another pattern.
## Kind Selectors
- `kind: function_item` — match AST node type directly.
- `kind: call_expression` — match function calls.
- Use `sg --debug-query='println!("hello")'` to see AST node kinds.
## Configuration (sgconfig.yml)
```yaml
ruleDirs:
- rules/
testConfigs:
- rules/tests/
```
## Advanced Patterns
- Find unused variables: `let $VAR = $EXPR;` where `$VAR` not referenced later.
- Find API migrations: `old_function($$$ARGS)` → `new_function($$$ARGS)`.
- Enforce patterns: ensure all error handling uses `?` not `.unwrap()`.
- Security: find `eval($EXPR)`, SQL injection patterns, hardcoded secrets.
## Integration
- CI/CD: `sg scan --json` for machine-readable output.
- Pre-commit hooks: `sg scan --rule rules/` on staged files.
- Editor: VSCode extension, LSP support.
- Programmatic: `@ast-grep/napi` Node.js binding for custom tools.
## vs Regex
- Regex: `unwrap\(\)` matches in comments, strings, docs. AST-grep: only actual code.
- Regex can't match nested structures. AST-grep handles `if { if { unwrap() } }`.
- AST-grep understands scope, types, structure. Regex is text-level.
knowledge/browser-use.md
-186
View File
@@ -1,186 +0,0 @@
# browser-use — AI Browser Automation
## What It Is
- AI agent that controls a real browser via CDP (Chrome DevTools Protocol).
- Describe task in natural language → agent clicks, types, navigates, extracts data.
- Python library for code-level automation OR CLI for agent integration (Claude Code, Cursor, etc).
- Open source, self-hosted. Optional cloud for stealth/scaling.
## Installation
```bash
# Python library
uv add browser-use
# or: pip install browser-use
# CLI (via uvx, no install needed)
uvx browser-use
# Install Chromium + dependencies
browser-use install
# Register skill for coding agents
browser-use skill install
```
## CLI Usage
```bash
# Run inline Python with pre-imported helpers
browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY
# Check local Chrome connection
browser-use --doctor
# macOS: approve remote debugging permission
browser-use mac-approve
# Auth for cloud browsers
browser-use auth login
```
## Python Library — Basic Agent
```python
import asyncio
from browser_use import Agent, ChatBrowserUse
async def main():
agent = Agent(
task="Find the price of iPhone 16 on Amazon",
llm=ChatBrowserUse(model='openai/gpt-5.5'),
)
result = await agent.run()
print(result)
asyncio.run(main())
```
## Structured Output
```python
from pydantic import BaseModel, Field
from browser_use import Agent, ChatBrowserUse
class ProductInfo(BaseModel):
title: str = Field(..., description='Product name')
price: float = Field(..., description='Price as number')
url: str = Field(..., description='Product URL')
agent = Agent(
task="Find the top 3 results for 'mechanical keyboard' on Amazon",
llm=ChatBrowserUse(model='bu-2-0-mini-preview'),
output_model_schema=ProductInfo,
)
result = asyncio.run(agent.run())
if result and result.structured_output:
print(result.structured_output)
```
## Connect to Existing Browser
```python
from browser_use import Agent, Browser, ChatBrowserUse
# Connect to local Chrome with CDP
browser = Browser(cdp_url='http://localhost:9222')
agent = Agent(browser=browser, llm=ChatBrowserUse(), task="...")
```
## CLI Helper Functions
Pre-imported in `browser-use` CLI context:
- `new_tab(url)` — open URL in new tab (first navigation per task)
- `goto_url(url)` — navigate current tab
- `page_info()` — get current page info
- `current_tab()` — get current tab info
- `list_tabs()` — list all open tabs
- `switch_tab(target)` — switch to existing tab
- `activate_tab(target)` — bring tab to foreground
- `click_at_xy(x, y)` — click at viewport coordinates
- `js(code)` — execute JavaScript in page
- `wait_for_load()` — wait for page load after navigation
- `ensure_real_tab()` — skip internal/stale tabs
- `cdp("Domain.method", ...)` — raw CDP command
- `start_recording(name)` / `stop_recording()` — record session
## Remote/Cloud Browsers
```bash
# Start remote browser (for parallel tasks or stealth)
browser-use <<'PY'
start_remote_daemon("my-task")
PY
# Use remote browser
BU_NAME=my-task browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY
# Stop when done (billed until stopped)
browser-use <<'PY'
stop_remote_daemon("my-task")
PY
```
## Page Interaction Workflow
1. Find elements via accessibility tree: `cdp("Accessibility.getFullAXTree")["nodes"]`
2. Get coordinates: `cdp("DOM.getBoxModel", backendNodeId=n)` → compute center
3. Click: `click_at_xy(x, y)`
4. Verify: `js(...)` or `page_info()` to confirm action
5. Fallback to raw HTML via `js(...)` only when AX tree lacks element
## When to Use browser-use vs curl
- **Use curl**: public pages, APIs, static content, docs
- **Use browser-use**: login-required pages, JS-rendered content, form filling, clicking, bot-protected sites, interactive workflows
## Environment Variables
- `BROWSER_USE_API_KEY` — API key for ChatBrowserUse or cloud
- `BU_NAME` — name for remote daemon
- `BU_CDP_URL` — custom CDP endpoint
- `BH_DOMAIN_SKILLS=1` — enable domain-specific skills
- `BH_AGENT_WORKSPACE` — path to agent workspace with helpers
## Recordings
```bash
# Enable/disable background recording
browser-use recordings enable
browser-use recordings disable
# View latest recording
browser-use recordings --latest
```
## Common Use Cases
- Fill job applications with resume data
- Extract structured data from websites (followers, products, prices)
- Compare prices across multiple sites
- Automate checkout flows
- Screenshot pages for verification
- QA testing of web applications
- Monitor website changes
## Integration with Verification Workflow
```bash
# After deploying an API, verify it works via browser
browser-use <<'PY'
new_tab("https://api.riotpiao.com/health")
info = page_info()
print(info) # Should show {"status": "ok"}
PY
# Verify UI renders correctly after deployment
browser-use <<'PY'
new_tab("https://myapp.riotpiao.com")
wait_for_load()
# Check page title
title = js("document.title")
print(f"Page title: {title}")
PY
```
## Gotchas
- First navigation per task must use `new_tab(url)`, not `goto_url(url)`
- Don't open duplicate tabs — check `list_tabs()` first
- Login walls: stop and ask user. Exception: SSO if already signed in
- `chrome://inspect/#remote-debugging` must be enabled for local Chrome
- CDP target order ≠ Chrome's visible tab-strip order
- Always call `wait_for_load()` after navigation
- Cloud browsers bill until stopped — always `stop_remote_daemon()` when done
knowledge/caveman-communication.md
-38
View File
@@ -1,38 +0,0 @@
# Caveman Communication — Ultra-Compressed Output
## Core Principle
- Cut output tokens 60-70% while keeping 100% technical accuracy.
- Speak like caveman: drop articles, filler, hedging. Keep nouns, verbs, code.
- Every word must carry information. If removing a word doesn't lose meaning, remove it.
## Rules
- No articles (a, an, the) unless ambiguous without them.
- No filler phrases: "I think", "it seems like", "you might want to", "basically".
- No hedging: "perhaps", "maybe", "it's possible that".
- No repetition of what user said. They know what they asked.
- No "Let me" or "I'll" — just do it.
- Code > prose. Show the fix, not a paragraph about the fix.
- Use symbols: → (leads to), ✓ (done/correct), ✗ (wrong), = (equals/means).
## Intensity Levels
- **Lite**: Drop filler, keep sentence structure. "Fix the port conflict by changing 8080 to 8081."
- **Full**: Drop articles, compress sentences. "Fix port conflict — change 8080→8081."
- **Ultra**: Telegraphic. Maximum compression. "port 8080→8081. done."
## Examples
Bad (verbose):
"I think the issue you're experiencing is that the Kubernetes service is trying to bind to port 8080, which is already in use by another process. You might want to try changing the port to 8081 in your deployment manifest."
Good (caveman):
"Port 8080 conflict. Change in deployment.yaml: containerPort: 8081. Restart pod."
## When NOT to Caveman
- User explicitly asks for detailed explanation.
- Teaching a new concept — clarity > brevity.
- Error messages and warnings — be precise.
- Documentation writing — full prose expected.
## Token Savings
- Average English: 1.3 tokens per word. 100 words ≈ 130 tokens.
- Caveman same content: ~35 words ≈ 46 tokens. 65% reduction.
- Over a session: saves thousands of output tokens → more context for actual work.
knowledge/curl-api-testing.md
-160
View File
@@ -1,160 +0,0 @@
# curl — API Testing & Verification
## Why Test With curl
- Every API endpoint must be testable with curl before marking a task done.
- curl is the ground truth — if curl can't hit it, the API doesn't work.
- Test locally first (`localhost`), then staging, then production.
- Save working curl commands as documentation for future reference.
## Basic Patterns
```bash
# GET request
curl -s http://localhost:8080/health | jq .
# POST with JSON body
curl -s -X POST http://localhost:8080/memory/ingest \
-H "Content-Type: application/json" \
-d '{"project": "poimen", "text": "test record", "kind": "L1"}'
# PUT update
curl -s -X PUT http://localhost:8080/resource/123 \
-H "Content-Type: application/json" \
-d '{"field": "new_value"}'
# DELETE
curl -s -X DELETE http://localhost:8080/resource/123
```
## Authentication
```bash
# Bearer token (JWT)
curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8080/memory/query
# Get JWT from Authentik
TOKEN=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
-d "grant_type=client_credentials" \
-d "client_id=$CLIENT_ID" \
-d "client_secret=$CLIENT_SECRET" | jq -r .access_token)
# API key header
curl -s -H "X-API-Key: $API_KEY" http://localhost:8080/endpoint
```
## Response Inspection
```bash
# Status code only
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/health
# Headers + body
curl -sv http://localhost:8080/health 2>&1
# Response time
curl -s -o /dev/null -w "%{time_total}s" http://localhost:8080/health
# Follow redirects
curl -sL http://localhost:8080/old-path
# Pretty print JSON
curl -s http://localhost:8080/health | python3 -m json.tool
```
## Error Testing
```bash
# Expect 401 — missing auth
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/memory/query
# Should return: 401
# Expect 400 — bad request body
curl -s -X POST http://localhost:8080/memory/ingest \
-H "Content-Type: application/json" \
-d '{"invalid": true}'
# Should return: 400
# Expect 429 — rate limit
for i in $(seq 1 200); do
curl -s -o /dev/null -w "%{http_code}\n" -H "Authorization: Bearer $TOKEN" \
http://localhost:8080/memory/query -d '{"query":"test"}'
done | sort | uniq -c
# Should see 429 after limit exceeded
# Expect 404 — not found
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/nonexistent
```
## Kubernetes Service Testing
```bash
# Port-forward to test in-cluster service
kubectl port-forward -n poimen svc/poimen-memory 8080:8080 &
curl -s http://localhost:8080/health
# Direct pod exec curl
kubectl exec -n poimen deploy/poimen-memory -- curl -s http://localhost:8080/health
# Test from inside cluster (debug pod)
kubectl run -it --rm curl-test --image=curlimages/curl -- \
curl -s http://poimen-memory.poimen.svc.cluster.local:8080/health
# Test ingress from outside
curl -sk https://api.riotpiao.com/health
```
## Verification Checklist
After completing any API work, verify with curl:
1. **Happy path** — correct input returns expected output and status code
2. **Auth required** — missing token returns 401, bad token returns 403
3. **Validation** — bad input returns 400 with error message
4. **Not found** — missing resource returns 404
5. **Idempotency** — same request twice returns same result (for POST with idempotency key)
6. **Rate limiting** — excessive requests return 429
7. **Content type** — response has correct Content-Type header
8. **Latency** — response time within SLA (< 500ms for queries, < 50ms for health)
## Scripted Verification
```bash
#!/bin/bash
# verify-api.sh — run after any API change
BASE="http://localhost:8080"
PASS=0; FAIL=0
check() {
local desc="$1" expected="$2" actual="$3"
if [ "$expected" = "$actual" ]; then
echo "✓ $desc"; ((PASS++))
else
echo "✗ $desc (expected $expected, got $actual)"; ((FAIL++))
fi
}
check "health returns 200" "200" \
"$(curl -s -o /dev/null -w '%{http_code}' $BASE/health)"
check "query without auth returns 401" "401" \
"$(curl -s -o /dev/null -w '%{http_code}' -X POST $BASE/memory/query)"
check "ingest with bad body returns 400" "400" \
"$(curl -s -o /dev/null -w '%{http_code}' -X POST $BASE/memory/ingest \
-H 'Content-Type: application/json' -d '{}')"
echo "---"
echo "$PASS passed, $FAIL failed"
[ $FAIL -eq 0 ] && exit 0 || exit 1
```
## Data Piping
```bash
# Ingest from file
curl -s -X POST http://localhost:8080/memory/ingest \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d @payload.json
# Query and filter with jq
curl -s -H "Authorization: Bearer $TOKEN" \
http://localhost:8080/memory/query \
-d '{"project":"poimen","query":"rust error handling"}' | \
jq '.results[] | {score, text}'
# Chain: query → pipe to next call
CHUNK_ID=$(curl -s ... | jq -r '.results[0].id')
curl -s http://localhost:8080/memory/chunk/$CHUNK_ID
```
knowledge/golang-skills.md
-85
View File
@@ -1,85 +0,0 @@
# Go (Golang) Skills
## Core Idioms
- Accept interfaces, return structs.
- Errors are values — check them explicitly. `if err != nil { return err }`.
- Don't panic in library code. Reserve panic for truly unrecoverable situations.
- Zero values are useful — `var m map[string]int` is nil but `var s []int` is usable.
## Error Handling
- Wrap errors with context: `fmt.Errorf("failed to open %s: %w", path, err)`.
- Sentinel errors: `var ErrNotFound = errors.New("not found")`. Check with `errors.Is(err, ErrNotFound)`.
- Custom error types: `type ValidationError struct { Field, Message string }`. Check with `errors.As()`.
- Never ignore errors: `_ = doSomething()` is a code smell. At minimum, log it.
## Concurrency
- "Don't communicate by sharing memory; share memory by communicating." — use channels.
- `go func()` launches goroutine. Always ensure goroutines terminate (context, done channel).
- `sync.WaitGroup` to wait for goroutine completion.
- `sync.Mutex` when channels are overkill (protecting a counter, map).
- `context.Context` for cancellation, timeouts, and request-scoped values. Always first parameter.
- `errgroup.Group` for parallel tasks with error propagation.
## Channel Patterns
- `ch := make(chan T)` unbuffered (synchronous). `make(chan T, n)` buffered.
- Fan-out: multiple goroutines read from one channel.
- Fan-in: multiple channels merged into one via select.
- Pipeline: chain of stages connected by channels.
- `select` with `case <-ctx.Done():` for cancellation.
- Close channels from sender side only. Never close from receiver.
## Interfaces
- Interfaces are satisfied implicitly — no `implements` keyword.
- Keep interfaces small: `io.Reader` has one method. `io.ReadWriteCloser` composes three.
- Define interfaces where they're used, not where they're implemented.
- `interface{}` (or `any`) is a code smell — prefer generics or specific interfaces.
- Type assertions: `v, ok := i.(ConcreteType)`. Type switch: `switch v := i.(type) { ... }`.
## Generics (Go 1.18+)
- `func Map[T, U any](s []T, f func(T) U) []U` — generic function.
- Constraints: `comparable`, `~int | ~float64`, custom interface constraints.
- Use generics for data structures and utility functions, not business logic.
## Project Structure
```
cmd/
myapp/main.go # entrypoint
internal/ # private packages
domain/ # business logic, no external deps
repository/ # data access
handler/ # HTTP handlers
pkg/ # public library code
```
- `internal/` enforced by Go compiler — cannot be imported outside module.
- One package per directory. Package name = directory name.
## Testing
- `func TestFoo(t *testing.T)` — test functions.
- Table-driven tests: `tests := []struct{ name string; input int; want int }{ ... }`.
- `t.Run(name, func(t *testing.T) { ... })` for subtests.
- `t.Parallel()` for concurrent test execution.
- `testify/assert` for cleaner assertions. `testify/mock` for mocking.
- `httptest.NewServer()` for HTTP integration tests.
- Benchmarks: `func BenchmarkFoo(b *testing.B) { for i := 0; i < b.N; i++ { ... } }`.
## HTTP Server
- `http.HandlerFunc` wraps functions as handlers.
- Middleware pattern: `func Logging(next http.Handler) http.Handler`.
- Use `chi` or `echo` for routing. Stdlib `http.ServeMux` improved in Go 1.22.
- Always set timeouts: `srv := &http.Server{ReadTimeout: 5*time.Second, WriteTimeout: 10*time.Second}`.
- Graceful shutdown: `signal.Notify` + `srv.Shutdown(ctx)`.
## Performance
- `pprof` for CPU/memory profiling: `go tool pprof http://localhost:6060/debug/pprof/profile`.
- `sync.Pool` for reducing GC pressure on frequently allocated objects.
- Pre-allocate slices: `make([]T, 0, expectedLen)`.
- String building: `strings.Builder` not `+` concatenation.
- Avoid interface boxing in hot paths.
## Common Gotchas
- Loop variable capture in goroutines (fixed in Go 1.22, but still common in older code).
- Nil interface vs nil pointer: `var p *MyType = nil; var i MyInterface = p; i != nil` is TRUE.
- Maps are not safe for concurrent access — use `sync.Map` or `sync.RWMutex`.
- Slice append may or may not create a new backing array — never hold stale slice references.
- `defer` evaluates arguments immediately, runs function at return.
- `init()` runs before `main()` — avoid side effects, prefer explicit initialization.
knowledge/poimen-memory-service.md
-237
View File
@@ -1,237 +0,0 @@
# Poimen Memory Service — Agent Integration Guide
## What Is This
Poimen Memory is a knowledge management system that stores, retrieves, and reasons over learned facts. Agents use it to remember solutions, patterns, and skills across sessions. It prevents re-learning the same things and surfaces relevant knowledge when needed.
## Architecture
- **Event Log** (JSONL): Immutable source of truth. All knowledge written here first.
- **pgvector** (Postgres): Vector store for semantic search (768-dim embeddings).
- **OpenSearch**: Lexical index for BM25 keyword search.
- **Hybrid Search**: 60% semantic + 40% lexical via RRF fusion.
- **Three-Tier Retrieval**: Tier 1 (exact signature) → Tier 2 (hybrid search) → Tier 3 (reference docs).
## When to Memorize (What's Worth Storing)
### DO Memorize
- **Bug fixes**: Error message → root cause → solution. Next time the same error appears, instant recall.
- **Configuration patterns**: Port conflicts, env var requirements, service dependencies.
- **Architecture decisions**: Why we chose X over Y, with trade-offs documented.
- **Tool usage patterns**: Commands that work, flags that matter, common gotchas.
- **Deployment procedures**: Steps to deploy, verify, rollback. Infra-specific knowledge.
- **API contracts**: Request/response formats, auth requirements, rate limits.
- **Design patterns**: Patterns that worked well in this codebase, anti-patterns to avoid.
- **Failure signatures**: Specific error outputs mapped to known fixes.
### DO NOT Memorize
- Temporary debugging output or scratch work.
- Information already in official docs (link to docs instead).
- Personal preferences or style opinions (put in CLAUDE.md instead).
- Secrets, tokens, passwords, API keys — NEVER.
- Exact code implementations (too large, changes too often — store the pattern, not the code).
- Trivial facts that any LLM already knows (e.g., "Rust uses cargo for builds").
### Quality Bar
Ask: "If I hit this problem again in 3 months, would this knowledge save me 30+ minutes?" If yes, memorize it. If not, skip it.
## CLI Usage — `mem learn`
### Ingest Markdown Knowledge
```bash
# Ingest a single file
mem learn knowledge/rust-fundamentals.md
# Ingest entire directory
mem learn knowledge/
# Preview chunks without writing (dry run)
mem learn knowledge/ --dry-run
# Custom project namespace
mem learn docs/ --project my-project
# Smaller chunks for fine-grained retrieval
mem learn knowledge/ --chunk-size 500
```
### How Chunking Works
- Splits on `## ` headings — each section becomes a chunk.
- If a section exceeds `--chunk-size`, it hard-splits at the boundary.
- Each chunk gets a SHA256 hash for deduplication.
- Chunks are stored as JSONL events in `log/<project>/learn/latest.jsonl`.
### Writing Good Knowledge Files
```markdown
# Topic Title
## Subtopic A
- Concise bullet points with actionable information.
- Include the WHY, not just the WHAT.
- Code examples with context, not bare snippets.
## Subtopic B
- Each ## section becomes one retrievable chunk.
- Keep sections focused — one concept per section.
- 200-500 chars per section is ideal for retrieval precision.
```
## CLI Usage — Other Commands
### Capture Failures (Auto-Learn from Errors)
```bash
# Record a command execution and its output
mem capture --cmd "cargo build" --exit 1 --output-file /tmp/build-error.log
# Extract failure signature
mem sig --tool cargo --file /tmp/build-error.log
```
### Resolve Lessons (Pair Failures with Fixes)
```bash
# After fixing a bug, derive the lesson
mem resolve --json
```
### Lookup Known Fixes
```bash
# Search memory for known fix
echo "error[E0308]: mismatched types" | mem lookup --tool cargo
# With minimum confidence threshold
mem lookup --tool kubectl --file /tmp/error.log --floor 0.7
```
### Materialize Skills
```bash
# Generate SKILL.md files from memory
mem materialize
```
## HTTP API — Programmatic Access
### Health Check
```bash
curl -s http://localhost:8080/health | jq .
# {"status": "ok", "timestamp": "..."}
```
### Ingest Records
```bash
curl -s -X POST http://localhost:8080/memory/ingest \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"project": "poimen",
"source": "transcript://session-123",
"kind": "L1",
"text": "Port 8080 conflict fixed by changing to 8081 in deployment.yaml",
"metadata": {"topic": "kubernetes", "session_id": "sess-123"}
}' | jq .
# Returns: 201 with chunk ID and SHA256
```
### Query Memory (Hybrid Search)
```bash
curl -s -X POST http://localhost:8080/memory/query \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"project": "poimen",
"query": "how to fix kubernetes port conflict",
"limit": 5,
"floor": 0.6
}' | jq '.results[] | {score, text}'
# Returns: ranked results with semantic + lexical scores
```
### Context Lookup (Three-Tier)
```bash
curl -s -X POST http://localhost:8080/memory/context \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"project": "poimen",
"tool": "cargo",
"task": "build",
"budget": 4096
}' | jq .
# Returns: tier-1 signature matches, tier-2 search results, tier-3 reference docs
```
### Browse Vault
```bash
curl -s "http://localhost:8080/memory/vault?project=poimen" \
-H "Authorization: Bearer $TOKEN" | jq '.files[] | .path'
```
### Rebuild from Log
```bash
curl -s -X POST http://localhost:8080/memory/rebuild \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"project": "poimen", "dry_run": true, "verify_parity": true}' | jq .
```
## Authentication
```bash
# Get JWT token from Authentik
TOKEN=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
-d "grant_type=client_credentials" \
-d "client_id=$CLIENT_ID" \
-d "client_secret=$CLIENT_SECRET" | jq -r .access_token)
# Use token
curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8080/memory/query ...
```
## Rate Limits
- Ingest: 100 requests/hour
- Query: 1000 requests/hour
- Context: 100 requests/hour
- Idempotency: 24-hour TTL (same ingest key → 409 Conflict)
## Agent Workflow — When to Query vs Ingest
### Before Starting a Task
```
1. Extract key terms from the task description
2. Query memory: mem lookup --tool <tool> or POST /memory/query
3. If results found with score > 0.7 → apply known solution
4. If no results → proceed with normal problem-solving
```
### After Completing a Task
```
1. Did I learn something new? (bug fix, config pattern, deployment step)
2. Would this save 30+ minutes if encountered again?
3. If yes → write a markdown note → mem learn note.md
4. If it was a failure→fix pair → mem capture + mem resolve
```
### During Debugging
```
1. Hit an error → mem sig --tool <tool> --file error.log
2. Check if signature matches known fix → mem lookup
3. If known → apply fix
4. If unknown → debug normally, then memorize the solution
```
## Knowledge Levels
- **L0**: Raw evidence (transcripts, logs). Automatically extracted.
- **L1**: Per-query memory (specific facts, fixes, patterns). Most useful for agents.
- **L2**: Project-level synthesis (cross-cutting themes, architecture summaries).
- **R**: Reference documents (Obsidian vault, external docs). Read-only fallback.
## Infrastructure
- **Cluster**: Kubernetes (poimen namespace)
- **Database**: CNPG pgvector (2 replicas)
- **Search**: OpenSearch (2-node HA)
- **Auth**: Authentik (JWT/OIDC)
- **Deployment**: ArgoCD from `forgejo.riotpiao.com/rock/poimen-memory`
- **Obsidian**: REST API at `obsidian-server.poimen.svc` (reference corpus)
## Gotchas
- Event log is immutable — you can't delete knowledge, only supersede it with newer facts.
- Rebuild from log is deterministic — same log → same vector store state.
- OpenSearch is fail-soft — if it's down, semantic search still works via pgvector alone.
- Embedding model is `sentence-transformers/all-MiniLM-L6-v2` (768-dim). Don't mix with other embedding models.
- `mem learn` appends to `latest.jsonl` — running it twice on the same files creates duplicate chunks (use SHA256 to detect).
knowledge/rust-fundamentals.md
-63
View File
@@ -1,63 +0,0 @@
# Rust Fundamentals
## Ownership & Borrowing
- Every value has exactly one owner. When owner goes out of scope, value is dropped.
- `&T` immutable borrow, `&mut T` mutable borrow. Cannot have `&mut` while `&` exists.
- Move semantics by default for non-Copy types. Clone for explicit deep copy.
## Lifetimes
- `'a` annotations tell compiler how long references live.
- Elision rules: single input lifetime → applied to all outputs. `&self` → output gets `'self` lifetime.
- `'static` means reference lives for entire program. String literals are `&'static str`.
## Error Handling
- `Result<T, E>` for recoverable errors, `panic!` for unrecoverable.
- `?` operator propagates errors. Use `anyhow::Result` for application code, `thiserror` for library errors.
- Never use `.unwrap()` in production — use `.expect("reason")` or proper error handling.
## Traits & Generics
- Traits define shared behavior: `trait Summary { fn summarize(&self) -> String; }`
- Trait bounds: `fn notify(item: &impl Summary)` or `fn notify<T: Summary>(item: &T)`
- `dyn Trait` for trait objects (dynamic dispatch), `impl Trait` for static dispatch.
- Blanket implementations: `impl<T: Display> ToString for T`
## Smart Pointers
- `Box<T>` heap allocation with single ownership.
- `Rc<T>` reference-counted shared ownership (single-threaded).
- `Arc<T>` atomic reference-counted (thread-safe). Use with `Mutex<T>` or `RwLock<T>`.
- `Cow<'a, T>` clone-on-write — borrows when possible, clones when mutation needed.
## Concurrency
- `Send` — type can be transferred across threads. `Sync` — type can be shared between threads.
- `tokio::spawn` for async tasks. `rayon` for data parallelism.
- Channels: `mpsc::channel()` for multi-producer single-consumer. `crossbeam` for advanced patterns.
- `async/await` — futures are lazy, must be `.await`ed or spawned.
## Pattern Matching
- `match` is exhaustive — must cover all variants.
- `if let Some(x) = option` for single-pattern matching.
- Destructuring: `let (a, b) = tuple;` and `let Point { x, y } = point;`
- Guards: `match x { n if n > 0 => ..., _ => ... }`
## Module System
- `mod foo;` loads from `foo.rs` or `foo/mod.rs`.
- `pub(crate)` visible within crate only. `pub(super)` visible to parent module.
- `use crate::module::Type` absolute path. `use super::Type` relative path.
- Re-exports: `pub use inner::Type;` to flatten module hierarchy.
## Iterators
- `.iter()` borrows, `.into_iter()` consumes, `.iter_mut()` mutable borrow.
- Lazy — nothing happens until consumed (`.collect()`, `.for_each()`, `.count()`).
- Chaining: `.filter().map().flat_map().take().collect::<Vec<_>>()`
- `impl Iterator for MyType { type Item = T; fn next(&mut self) -> Option<Self::Item> }`
## Macros
- `macro_rules!` for declarative macros. `#[derive(...)]` for derive macros.
- `proc_macro` for procedural macros (attribute, derive, function-like).
- `vec![1, 2, 3]` expands to `{ let mut v = Vec::new(); v.push(1); ... v }`
## Common Patterns
- Builder pattern: `MyStruct::new().with_field(val).build()`
- Newtype pattern: `struct UserId(u64);` for type safety without runtime cost.
- Type state pattern: use generics to encode state in the type system.
- Interior mutability: `Cell<T>`, `RefCell<T>` for single-threaded, `Mutex<T>` for multi-threaded.
knowledge/solid-dry-principles.md
-69
View File
@@ -1,69 +0,0 @@
# SOLID & DRY Design Principles
## Single Responsibility Principle (SRP)
- A class/module should have one and only one reason to change.
- Each module owns exactly one actor's requirements.
- Bad: `UserService` that handles auth, email, and database. Good: separate `AuthService`, `EmailService`, `UserRepository`.
- In Rust: one struct per concern. `ChunkProcessor` doesn't also handle HTTP routing.
## Open/Closed Principle (OCP)
- Software entities should be open for extension, closed for modification.
- Use traits/interfaces to allow new behavior without changing existing code.
- Strategy pattern: `trait Scorer { fn score(&self, doc: &Doc) -> f64; }` — add new scorers without modifying search.
- In Rust: trait objects or generics. `fn process<S: Strategy>(s: &S)` — new strategies don't touch `process`.
## Liskov Substitution Principle (LSP)
- Subtypes must be substitutable for their base types without breaking correctness.
- If `fn accept(animal: &dyn Animal)` works with `Dog`, it must work with `Cat` too.
- Violated when: subtype throws unexpected errors, ignores base contract, strengthens preconditions.
- In Rust: trait implementations must honor the trait's documented contract.
## Interface Segregation Principle (ISP)
- Clients should not be forced to depend on interfaces they don't use.
- Many small traits > one fat trait.
- Bad: `trait Repository { fn read(); fn write(); fn delete(); fn audit(); }` — read-only clients forced to see write methods.
- Good: `trait Readable`, `trait Writable`, `trait Auditable` — compose as needed.
- In Rust: supertraits for composition: `trait FullRepo: Readable + Writable + Auditable {}`
## Dependency Inversion Principle (DIP)
- High-level modules should not depend on low-level modules. Both should depend on abstractions.
- In Rust: accept `impl Trait` or `&dyn Trait`, not concrete types.
- `fn search(store: &dyn VectorStore)` — works with Postgres, OpenSearch, or in-memory mock.
- Constructor injection: `struct SearchEngine { store: Box<dyn VectorStore> }`
## DRY (Don't Repeat Yourself)
- Every piece of knowledge should have a single, unambiguous, authoritative representation.
- DRY is about knowledge, not code. Two functions with same code but different reasons to change are NOT duplication.
- Extract when: same logic appears 3+ times AND changes for the same reason.
- Wrong DRY: coupling unrelated code just because it looks similar. Right DRY: shared business rules in one place.
## WET (Write Everything Twice) — When DRY Goes Wrong
- Premature DRY creates coupling worse than duplication.
- Rule of three: duplicate is fine, triplicate means extract.
- Tests should be WET — readability > DRYness in test code.
- Configuration can be WET — explicit is better than magic shared config.
## KISS (Keep It Simple, Stupid)
- Simplest solution that works is usually the best.
- Avoid: premature abstraction, speculative generality, framework-itis.
- Measure complexity: if a new team member can't understand it in 15 minutes, simplify.
## YAGNI (You Aren't Gonna Need It)
- Don't build features until you actually need them.
- Speculative code rots — it's untested, unmaintained, and misleading.
- Exception: known architectural boundaries (API versioning, database migrations).
## Composition Over Inheritance
- Prefer composing objects over class hierarchies.
- In Rust: no inheritance. Composition is the default via struct fields + trait delegation.
- `struct HttpServer { router: Router, auth: AuthMiddleware, rate_limiter: RateLimiter }`
## Law of Demeter
- Only talk to your immediate friends. Don't chain: `a.b().c().d()`.
- Tell, don't ask: `order.ship()` not `order.get_warehouse().get_shipping().create_label()`.
- In Rust: expose methods that encapsulate internal structure.
## Practical Application
- Start concrete, extract abstractions when patterns emerge.
- Refactor in small steps with tests as safety net.
- Code review checklist: SRP violated? Unnecessary coupling? Duplicated knowledge? Over-engineered?
knowledge/tea-cli.md
-145
View File
@@ -1,145 +0,0 @@
# tea CLI — Gitea/Forgejo Command Line
## Setup
```bash
# Login to Forgejo instance
tea login add --name riotpiao \
--url https://git.riotpiao.com \
--token <your-token>
# Verify
tea whoami
```
## Repository Operations
```bash
# List repos
tea repos ls
# Clone
tea clone rock/poimen-memory
# Open in browser
tea open
```
## Issues & Tickets
```bash
# List issues
tea issues ls --repo rock/poimen-memory
tea issues ls --state open
tea issues ls --labels bug
# Create issue
tea issues create --title "Fix port conflict" --body "Port 8080 is in use"
# Close issue
tea issues close 42
# Comment on issue
tea comments create 42 --body "Fixed in commit abc123"
# Assign
tea issues edit 42 --assignees rock
```
## Pull Requests
```bash
# List PRs
tea pr ls --repo rock/poimen-memory
tea pr ls --state open
# Create PR from current branch
tea pr create --title "feat: add learn command" --base main
# Checkout a PR locally
tea pr checkout 15
# Merge PR
tea pr merge 15 --style squash
# Review PR
tea pr review 15 --approve
tea pr review 15 --request-changes --body "Fix the unwrap()"
```
## CI/CD — Actions & Workflows
```bash
# List workflow runs
tea actions runs list --repo rock/poimen-memory
tea actions runs list --repo rock/poimen-memory --limit 5
# View specific run
tea actions runs view <run-id> --repo rock/poimen-memory
# List workflows
tea actions workflows list --repo rock/poimen-memory
# Secrets management
tea actions secrets list --repo rock/poimen-memory
tea actions secrets create --repo rock/poimen-memory --name MY_SECRET --value "secret123"
# Variables
tea actions variables list --repo rock/poimen-memory
```
## Releases
```bash
# List releases
tea releases ls --repo rock/poimen-memory
# Create release
tea releases create --repo rock/poimen-memory \
--tag v1.0.0 \
--title "v1.0.0 — Production Release" \
--note "First production release"
```
## Wiki
```bash
# List wiki pages
tea wiki ls --repo rock/poimen-memory
# Create wiki page
tea wiki create --repo rock/poimen-memory \
--title "Setup Guide" \
--content "# Setup\n..."
```
## API Direct Access
```bash
# Raw API call (authenticated)
tea api /repos/rock/poimen-memory
tea api /repos/rock/poimen-memory/issues?state=open
# POST via API
tea api --method POST /repos/rock/poimen-memory/issues \
--body '{"title":"test","body":"test issue"}'
```
## Ticket Verification Workflow
After completing a task, verify the ticket is done:
```bash
# 1. Check CI passed
tea actions runs list --repo rock/poimen-memory --limit 1
# Should show: ✓ completed / success
# 2. Check issue is closed or PR merged
tea issues ls --repo rock/poimen-memory --state closed --limit 5
# 3. Verify the API endpoint works (curl the actual service)
curl -s http://localhost:8080/health | jq .status
# Should return: "ok"
# 4. Tag release if milestone complete
tea releases create --tag v1.x.x --title "Milestone X complete"
```
## Useful Flags
- `--repo owner/name` — specify repo (or use current git context)
- `--output simple` — machine-readable output
- `--output yaml` — YAML format
- `--output json` — JSON format for piping to jq
- `--limit N` — limit results
- `--state open|closed|all` — filter by state
- `--fields name,status` — select columns
knowledge/verify-done.md
-136
View File
@@ -1,136 +0,0 @@
# Verify Done — Confirming Task Completion
## Principle
A task is NOT done until it's verified with real data against the real service. Code review + CI green is necessary but not sufficient.
## Definition of Done Checklist
1. **Code compiles** — `cargo build` / `go build` passes with no errors
2. **Tests pass** — `cargo test` / `go test ./...` all green
3. **CI green** — `tea actions runs list` shows latest run succeeded
4. **API accessible** — `curl` the endpoint, get expected response
5. **Auth works** — requests without token return 401, with token return 200
6. **Error paths tested** — bad input returns proper error codes
7. **Deployed** — ArgoCD synced, pod running, ingress reachable
8. **Documented** — endpoint added to API docs or CLAUDE.md
## Verification Flow
```
Code change → Push → CI passes → ArgoCD deploys → curl test → Done
↓
If fails → fix → repeat
```
## API Endpoint Verification Template
For every new or changed endpoint, run:
```bash
ENDPOINT="https://api.riotpiao.com"
# 1. Is it alive?
curl -s -o /dev/null -w "%{http_code}" $ENDPOINT/health
# Expect: 200
# 2. Does the new endpoint exist?
curl -s -o /dev/null -w "%{http_code}" $ENDPOINT/new-endpoint
# Expect: NOT 404
# 3. Does auth gate work?
curl -s -o /dev/null -w "%{http_code}" $ENDPOINT/new-endpoint
# Expect: 401 (no token)
# 4. Does it return correct data?
curl -s -H "Authorization: Bearer $TOKEN" $ENDPOINT/new-endpoint | jq .
# Expect: meaningful JSON response
# 5. Does it handle bad input?
curl -s -o /dev/null -w "%{http_code}" -X POST $ENDPOINT/new-endpoint \
-H "Content-Type: application/json" -d '{}'
# Expect: 400
```
## CI Verification
```bash
# Check last CI run status
tea actions runs list --repo rock/poimen-memory --limit 1
# If failed, check logs
tea actions runs view <run-id> --repo rock/poimen-memory
# Check if ArgoCD synced
kubectl get application -n argocd poimen-memory-app -o jsonpath='{.status.sync.status}'
# Expect: Synced
# Check pod health
kubectl get pods -n poimen | grep poimen-memory
# Expect: Running, no restarts
```
## Deployment Verification
```bash
# Pod running and ready
kubectl get pods -n poimen -l app.kubernetes.io/name=poimen-memory
# Expect: 1/1 Running
# Service reachable internally
kubectl exec -n poimen deploy/poimen-memory -- curl -s http://localhost:8080/health
# Ingress reachable externally
curl -sk https://api.riotpiao.com/health
# Logs clean (no panics, no errors on startup)
kubectl logs -n poimen deploy/poimen-memory --tail=20
```
## Git Commit Verification
Before pushing, confirm:
```bash
# Commit message follows conventional commits
git log -1 --oneline
# Expect: feat: / fix: / docs: / refactor: prefix
# No secrets in diff
git diff --cached | grep -iE "password|secret|token|api_key"
# Expect: empty (no matches)
# No .env or key files staged
git diff --cached --name-only | grep -iE "\.env|\.key|\.pem"
# Expect: empty
```
## Memory System Specific Verification
```bash
# After ingest changes
curl -s -X POST https://api.riotpiao.com/memory/ingest \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"project":"test","text":"verify ingest","kind":"L1"}' | jq .
# Expect: 201 with chunk ID
# After query changes
curl -s -X POST https://api.riotpiao.com/memory/query \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"project":"test","query":"verify"}' | jq .
# Expect: 200 with results array
# After context endpoint changes
curl -s -X POST https://api.riotpiao.com/memory/context \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"project":"test","tool":"cargo","task":"build"}' | jq .
# Expect: 200 with tier, lessons, budget
# After rebuild changes
curl -s -X POST https://api.riotpiao.com/memory/rebuild \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"project":"test","dry_run":true}' | jq .
# Expect: 200 with records_processed count
```
## Anti-Patterns
- ❌ "CI passed so it's done" — CI doesn't test the real deployment
- ❌ "It works on my machine" — must work in-cluster
- ❌ Marking done without curl-testing the endpoint
- ❌ Skipping error path testing (400, 401, 404, 429)
- ❌ Not checking ArgoCD sync status after push
- ❌ Trusting `kubectl apply` over ArgoCD (let ArgoCD manage state)