homelab/project-usage/networking-ingress.md

# Networking: Ingress, TLS & Service Discovery

**Ingress Controller:** `nginx-ingress` (Nginx)  
**Load Balancer:** Cilium LB-IPAM (eBPF-based)  
**TLS CA:** homelab-ca (self-signed, 10-year)  
**Namespace:** `ingress-nginx`

## When to Use

- **Public HTTPS endpoints** — External access via TLS
- **Hostname-based routing** — Multiple services on same IP
- **TLS termination** — Offload encryption/decryption
- **Service discovery** — Internal DNS (CoreDNS)

## Quick Start

**1. Create Ingress rule:**
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp
  namespace: myapp-ns
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"  # or homelab-ca
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - myapp.riotpiao.com
    secretName: myapp-tls
  rules:
  - host: myapp.riotpiao.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp-svc
            port:
              number: 8080
```

**2. Deploy:**
```bash
kubectl apply -f ingress.yaml

# Wait for cert issuance
kubectl get certificate -n myapp-ns
# Should show "Ready" after ~30s
```

**3. Test from client:**
```bash
# Add to /etc/hosts (or use WireGuard)
192.168.1.160 myapp.riotpiao.com

# Access
curl https://myapp.riotpiao.com
```

## Configuration

| Key | Value |
|-----|-------|
| Ingress class | `nginx` |
| Load balancer type | `LoadBalancer` (Cilium LB-IPAM) |
| TLS issuer | `homelab-ca` (ClusterIssuer) |
| TLS cert lifetime | 90 days (auto-renewed by cert-manager) |
| DNS | CoreDNS (in-cluster), external via `/etc/hosts` or DuckDNS |

## Common Patterns

**Ingress with path-based routing:**
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
  namespace: default
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - api.riotpiao.com
    secretName: api-tls
  rules:
  - host: api.riotpiao.com
    http:
      paths:
      - path: /users
        pathType: Prefix
        backend:
          service:
            name: users-svc
            port:
              number: 3000
      - path: /orders
        pathType: Prefix
        backend:
          service:
            name: orders-svc
            port:
              number: 3001
```

**Ingress with basic auth:**
```bash
# Generate htpasswd
htpasswd -c auth admin
# → prompted for password

# Create Secret
kubectl create secret generic basic-auth --from-file=auth -n default

# Create Ingress
```

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: protected
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
spec:
  ingressClassName: nginx
  rules:
  - host: protected.riotpiao.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-svc
            port:
              number: 8080
```

**Internal DNS (CoreDNS rewrite):**
```yaml
# k8s/coredns/coredns-configmap.yaml
# Rewrite:
# - grafana.riotpiao.com → grafana.logging (cluster-internal)
# - prometheus.riotpiao.com → prometheus-kube-prom-prometheus.monitoring
#
# Allows pods to use external URLs but resolve to internal Services
```

**Fixed LoadBalancer IP (Cilium LB-IPAM):**
```yaml
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  annotations:
    io.cilium/lb-ipam-ips: "192.168.1.160"  # fixed IP
spec:
  type: LoadBalancer
  selector:
    app: nginx-ingress
  ports:
  - port: 443
    targetPort: 443
    protocol: TCP
```

## TLS Certificate Management

**Automatic renewal (cert-manager):**
```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: myapp-cert
  namespace: myapp-ns
spec:
  secretName: myapp-tls
  duration: 2160h  # 90 days
  renewBefore: 360h  # renew 15 days before expiry
  commonName: myapp.riotpiao.com
  dnsNames:
  - myapp.riotpiao.com
  issuerRef:
    name: homelab-ca
    kind: ClusterIssuer
```

**Check certificate status:**
```bash
# List certs
k get certificate -A

# View cert details
k describe certificate -n myapp-ns myapp-cert

# View TLS Secret
k get secret -n myapp-ns myapp-tls -o json | jq '.data."tls.crt"' | base64 -d | openssl x509 -text

# Check expiry date
k get secret -n myapp-ns myapp-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -enddate
```

## Service Discovery

**Cluster-internal DNS:**
```bash
# From any pod, resolve via CoreDNS
nslookup grafana.logging.svc.cluster.local  # full FQDN
nslookup grafana.logging                     # short form (same namespace)
nslookup grafana                             # if in logging namespace

# Resolved to ClusterIP (internal only)
```

**External DNS (WireGuard VPN or port-forward):**
```bash
# Option 1: WireGuard tunnel
# Client connects to 10.6.0.1 (WireGuard server on talos-cp-1)
# All traffic tunneled to cluster

# Option 2: Port-forward from jump box
make pf-grafana  # localhost:3000 → grafana.logging:3000

# Option 3: Add to /etc/hosts (on home network)
192.168.1.160 grafana.riotpiao.com
```

## Monitoring

**Grafana dashboard:** `svc-nginx-ingress`

**Key metrics:**
- `nginx_requests_total` — total requests
- `nginx_request_duration_seconds` — latency histogram
- `nginx_ingress_upstream_requests_total{status=~"5.."}` — backend errors
- `nginx_ssl_expire_time_seconds` — cert expiry countdown

**Alert on cert expiry:**
```yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: cert-expiry
  namespace: ingress-nginx
spec:
  groups:
  - name: cert-expiry
    rules:
    - alert: CertificateExpiringSoon
      expr: nginx_ssl_expire_time_seconds < 86400 * 14  # < 14 days
      annotations:
        summary: "Certificate {{ $labels.host }} expires in {{ $value | humanizeDuration }}"
```

## Troubleshooting

**Certificate stuck in "Pending":**
```bash
# Check cert-manager logs
k logs -n cert-manager -f deploy/cert-manager

# Verify ClusterIssuer exists
k get clusterissuer

# Check ACME order (if using LetsEncrypt)
k describe certificate -n myapp-ns myapp-cert
```

**Ingress not exposing service (503 error):**
```bash
# Verify Service exists and has endpoints
k get svc -n myapp-ns
k get endpoints -n myapp-ns myapp-svc

# Check if pods are ready
k get pods -n myapp-ns

# Test pod directly (port-forward)
k port-forward -n myapp-ns pod/myapp-0 8080:8080
curl http://localhost:8080
```

**DNS resolution fails from pod:**
```bash
# Test from pod
k run -it --rm debug --image=busybox:1.28 --restart=Never -- \
  nslookup grafana.logging.svc.cluster.local

# If fails, CoreDNS may be unhealthy
k get pods -n kube-system -l k8s-app=kube-dns
k logs -n kube-system -l k8s-app=kube-dns
```

**TLS handshake error (cert not trusted):**
```bash
# Verify TLS cert Secret exists
k get secret -n myapp-ns myapp-tls

# Verify cert is correctly signed by homelab-ca
k get secret -n myapp-ns myapp-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text | grep -A 5 "Issuer:"

# If cert is self-signed (homelab-ca), add to client's trusted roots
# Or bypass cert verification (dev only):
curl -k https://myapp.riotpiao.com
```

See `/TROUBLESHOOTING.md` for full incident guide.