feat: add 'mem learn' CLI for markdown knowledge ingestion

6 knowledge files: rust, SOLID/DRY, ast-grep, karpathy, golang, caveman
65 chunks ingested to log/knowledge/learn/latest.jsonl
Chunks on ## headings, SHA256 dedup, configurable chunk size

knowledge/andrej-karpathy.md

@@ -0,0 +1,55 @@
+# Andrej Karpathy — Key Insights & Practices
+
+## Software 2.0
+- Traditional software (1.0): explicit rules written by programmers.
+- Software 2.0: behavior learned from data via neural networks. Code = weights.
+- Implication: datasets are the new source code. Data curation > clever algorithms.
+- Debug by inspecting data, not stepping through logic.
+
+## Training Neural Networks — A Recipe
+1. **Become one with the data** — visualize, understand distributions, find patterns and anomalies before writing any model code.
+2. **Set up end-to-end training/eval skeleton** — simplest possible model first. Get the pipeline working.
+3. **Overfit first** — if model can't memorize a single batch, architecture is wrong.
+4. **Regularize** — only add dropout, weight decay, augmentation after overfitting confirmed.
+5. **Tune** — learning rate is the most important hyperparameter. Use LR finder.
+6. **Squeeze** — ensembles, larger models, more data. Diminishing returns here.
+
+## Most Common Neural Net Mistakes
+- Not looking at data first.
+- Forgetting to set model to eval mode (BatchNorm, Dropout change behavior).
+- Forgetting to zero gradients.
+- Using softmax with cross-entropy (use logits directly).
+- Not normalizing inputs.
+- Applying augmentation to validation set.
+- Silent shape broadcasting bugs — always assert tensor shapes.
+
+## LLM Insights (Post-GPT Era)
+- LLMs are "operating systems" — CPU is the transformer, context window is RAM, training data is disk.
+- Tokenization is a key bottleneck — BPE artifacts cause many failure modes.
+- Temperature controls creativity vs precision. T=0 for factual, T>0 for creative.
+- Chain-of-thought works because it gives the model "working memory" in the output tokens.
+- Prompt engineering is programming in natural language. Be explicit, give examples.
+
+## Build Nanograd / Micrograd Philosophy
+- Understand backpropagation by implementing it from scratch.
+- A neural net is just: forward pass → compute loss → backward pass → update weights.
+- Autograd: track operations, build computation graph, reverse-mode differentiation.
+- Every complex framework (PyTorch, JAX) is built on these same primitives.
+
+## Practical ML Engineering
+- Start simple: logistic regression baseline before deep learning.
+- Measure everything: loss curves, gradient norms, weight distributions.
+- Reproducibility: fix seeds, log hyperparameters, version datasets.
+- Don't trust your code — trust your loss curve. If loss isn't going down, something is wrong.
+- Data quality > model complexity. 10x data often beats 10x model size.
+
+## Scaling Laws
+- Performance scales predictably with compute, data, and parameters (Chinchilla scaling).
+- Compute-optimal training: balance model size and training tokens.
+- Emergent abilities appear at scale — capabilities that don't exist in smaller models.
+
+## On AI Engineering
+- The best AI engineers understand both ML and systems engineering.
+- Inference optimization matters as much as training — quantization, batching, KV-cache.
+- Eval is everything. If you can't measure it, you can't improve it.
+- Build evaluation suites before building features.