homelab/project-usage/networking-ingress.md

Networking: Ingress, TLS & Service Discovery

Ingress Controller: nginx-ingress (Nginx)
Load Balancer: Cilium LB-IPAM (eBPF-based)
TLS CA: homelab-ca (self-signed, 10-year)
Namespace: ingress-nginx

When to Use

• Public HTTPS endpoints — External access via TLS
• Hostname-based routing — Multiple services on same IP
• TLS termination — Offload encryption/decryption
• Service discovery — Internal DNS (CoreDNS)

Quick Start

1. Create Ingress rule:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp
  namespace: myapp-ns
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"  # or homelab-ca
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - myapp.riotpiao.com
    secretName: myapp-tls
  rules:
  - host: myapp.riotpiao.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp-svc
            port:
              number: 8080

2. Deploy:

kubectl apply -f ingress.yaml

# Wait for cert issuance
kubectl get certificate -n myapp-ns
# Should show "Ready" after ~30s

3. Test from client:

# Add to /etc/hosts (or use WireGuard)
192.168.1.160 myapp.riotpiao.com

# Access
curl https://myapp.riotpiao.com

Configuration

Key	Value
Ingress class	nginx
Load balancer type	LoadBalancer (Cilium LB-IPAM)
TLS issuer	homelab-ca (ClusterIssuer)
TLS cert lifetime	90 days (auto-renewed by cert-manager)
DNS	CoreDNS (in-cluster), external via /etc/hosts or DuckDNS

Common Patterns

Ingress with path-based routing:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
  namespace: default
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - api.riotpiao.com
    secretName: api-tls
  rules:
  - host: api.riotpiao.com
    http:
      paths:
      - path: /users
        pathType: Prefix
        backend:
          service:
            name: users-svc
            port:
              number: 3000
      - path: /orders
        pathType: Prefix
        backend:
          service:
            name: orders-svc
            port:
              number: 3001

Ingress with basic auth:

# Generate htpasswd
htpasswd -c auth admin
# → prompted for password

# Create Secret
kubectl create secret generic basic-auth --from-file=auth -n default

# Create Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: protected
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
spec:
  ingressClassName: nginx
  rules:
  - host: protected.riotpiao.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-svc
            port:
              number: 8080

Internal DNS (CoreDNS rewrite):

# k8s/coredns/coredns-configmap.yaml
# Rewrite:
# - grafana.riotpiao.com → grafana.logging (cluster-internal)
# - prometheus.riotpiao.com → prometheus-kube-prom-prometheus.monitoring
#
# Allows pods to use external URLs but resolve to internal Services

Fixed LoadBalancer IP (Cilium LB-IPAM):

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  annotations:
    io.cilium/lb-ipam-ips: "192.168.1.160"  # fixed IP
spec:
  type: LoadBalancer
  selector:
    app: nginx-ingress
  ports:
  - port: 443
    targetPort: 443
    protocol: TCP

TLS Certificate Management

Automatic renewal (cert-manager):

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: myapp-cert
  namespace: myapp-ns
spec:
  secretName: myapp-tls
  duration: 2160h  # 90 days
  renewBefore: 360h  # renew 15 days before expiry
  commonName: myapp.riotpiao.com
  dnsNames:
  - myapp.riotpiao.com
  issuerRef:
    name: homelab-ca
    kind: ClusterIssuer

Check certificate status:

# List certs
k get certificate -A

# View cert details
k describe certificate -n myapp-ns myapp-cert

# View TLS Secret
k get secret -n myapp-ns myapp-tls -o json | jq '.data."tls.crt"' | base64 -d | openssl x509 -text

# Check expiry date
k get secret -n myapp-ns myapp-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -enddate

Service Discovery

Cluster-internal DNS:

# From any pod, resolve via CoreDNS
nslookup grafana.logging.svc.cluster.local  # full FQDN
nslookup grafana.logging                     # short form (same namespace)
nslookup grafana                             # if in logging namespace

# Resolved to ClusterIP (internal only)

External DNS (WireGuard VPN or port-forward):

# Option 1: WireGuard tunnel
# Client connects to 10.6.0.1 (WireGuard server on talos-cp-1)
# All traffic tunneled to cluster

# Option 2: Port-forward from jump box
make pf-grafana  # localhost:3000 → grafana.logging:3000

# Option 3: Add to /etc/hosts (on home network)
192.168.1.160 grafana.riotpiao.com

Monitoring

Grafana dashboard: svc-nginx-ingress

Key metrics:

• nginx_requests_total — total requests
• nginx_request_duration_seconds — latency histogram
• nginx_ingress_upstream_requests_total{status=~"5.."} — backend errors
• nginx_ssl_expire_time_seconds — cert expiry countdown

Alert on cert expiry:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: cert-expiry
  namespace: ingress-nginx
spec:
  groups:
  - name: cert-expiry
    rules:
    - alert: CertificateExpiringSoon
      expr: nginx_ssl_expire_time_seconds < 86400 * 14  # < 14 days
      annotations:
        summary: "Certificate {{ $labels.host }} expires in {{ $value | humanizeDuration }}"

Troubleshooting

Certificate stuck in "Pending":

# Check cert-manager logs
k logs -n cert-manager -f deploy/cert-manager

# Verify ClusterIssuer exists
k get clusterissuer

# Check ACME order (if using LetsEncrypt)
k describe certificate -n myapp-ns myapp-cert

Ingress not exposing service (503 error):

# Verify Service exists and has endpoints
k get svc -n myapp-ns
k get endpoints -n myapp-ns myapp-svc

# Check if pods are ready
k get pods -n myapp-ns

# Test pod directly (port-forward)
k port-forward -n myapp-ns pod/myapp-0 8080:8080
curl http://localhost:8080

DNS resolution fails from pod:

# Test from pod
k run -it --rm debug --image=busybox:1.28 --restart=Never -- \
  nslookup grafana.logging.svc.cluster.local

# If fails, CoreDNS may be unhealthy
k get pods -n kube-system -l k8s-app=kube-dns
k logs -n kube-system -l k8s-app=kube-dns

TLS handshake error (cert not trusted):

# Verify TLS cert Secret exists
k get secret -n myapp-ns myapp-tls

# Verify cert is correctly signed by homelab-ca
k get secret -n myapp-ns myapp-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text | grep -A 5 "Issuer:"

# If cert is self-signed (homelab-ca), add to client's trusted roots
# Or bypass cert verification (dev only):
curl -k https://myapp.riotpiao.com

See /TROUBLESHOOTING.md for full incident guide.