homelab/project-usage/core-cli-tools.md

Core CLI Tools: Decision Guide

The core CLI is your primary tool for cluster auth, node operations, Vault secrets management, IAM administration, and object storage. This doc covers when to reach for core vs kubectl/helmfile, the two separate auth domains that power different commands, and the complete command inventory.

For exhaustive flag-level detail on each subcommand, see ~/workplace/core/USAGE.md.

---

The Two Auth Domains

The core CLI maintains two independent authentication systems that gate different command families. Confusing them causes authentication failures.

Domain 1: Node/Talos Operations (core auth)

Gates: Node-level commands, Talos service management, cluster status.

Login mechanism: core auth login-oob

• Out-of-band device-code flow via Authentik
• Token cached in ~/.core/token (24h expiry)
• Required before: core nodes, core status <ip>, core services <ip>, core logs <ip>, core log-svc <ip> <svc>, core dmesg <ip>, core config, core upgrade, core shutdown, core reboot, core node, core pods clean

Check auth state:

core auth status          # Show token expiry
core auth clear           # Force re-auth on next command

When to use:

• Troubleshooting node-level issues (crashes, disk space, network on a specific Talos node)
• Checking Talos service health (etcd, kubelet, scheduler, etc.)
• Upgrading cluster OS or managing node lifecycle
• Reading kernel logs (dmesg) or kubelet logs on specific nodes

Domain 2: Vault Secrets (core secrets)

Gates: Secrets management (read/write), secret listing, secret export.

Login mechanism: core secrets login

• Interactive login to Vault (opens browser, approves auth)
• Reads/writes to ~/.config/talos/secrets.toml
• Required before: core get <path> [--key KEY], core put <path> key=value, core secrets list, core secrets export, core secrets exec

Check auth state:

core secrets status       # Show Vault token / scopes
core secrets clear        # Force re-auth on next command

When to use:

• Reading or writing cluster secrets (API keys, database passwords, OAuth client secrets)
• Managing application credentials in Vault (centralized secret store)
• Rotating secrets for services (e.g., OAuth2 client secrets)
• Listing all secrets under a path
• Bootstrapping .env files for helmfile deployment (via vsource)

---

Auth Bug Fixes

Bug #1 (Fixed): core status <NODE> is a node-ops command, not an auth verifier. It does NOT verify that you're authenticated to Vault. To verify auth state, use the correct domain-specific command:

• For node access: core auth status
• For Vault access: core secrets status

Bug #2 (Fixed): core secrets login and core auth login-oob are NOT interchangeable. They gate completely different systems:

• core auth login-oob gates node/cluster operations
• core secrets login gates Vault secret read/write
• You may be authenticated to one domain and not the other. Check state separately.

---

When to Reach for core

Scenario	Tool	Why
Pod/deployment issue	kubectl	Pods, replicas, rollouts, events
Package release management	helmfile	Install/upgrade Helm charts
Node crashes, disk, kernel panic	core auth + node commands	Direct access to Talos node state
Service unreachable on Talos node	core status <ip>, core services <ip>	Talos service health
Database password rotation	core secrets + core put	Vault secret write
Need API key for app deployment	core secrets + core get	Fetch from Vault, inject via vsource
Create OAuth2 app in Authentik	core iam create-app	Manage federated identity apps
Add user to service (MinIO, Grafana, etc.)	core iam add-member	Group membership binding
S3 object upload/download	core bucket	MinIO operations

---

Command Inventory

Complete list of core subcommands, organized by domain. See ~/workplace/core/USAGE.md for usage flags and examples.

Authentication

# Node/Talos operations auth
core auth login-oob       # Authenticate with Authentik (device code flow)
core auth status          # Check token expiry / auth state
core auth clear           # Clear cached auth token

# Vault secrets auth
core secrets login        # Authenticate to Vault (interactive)
core secrets status       # Check Vault token / scopes / expiry
core secrets clear        # Clear Vault auth token

Cluster & Node Operations

# Node discovery and status
core nodes                # List all cluster nodes (IP, hostname, status)
core status <ip>          # Talos node overview (resources, uptime)
core services <ip>        # List Talos services (etcd, kubelet, etc.)

# Logs
core logs <ip>            # Stream kubelet logs on node
core log-svc <ip> <svc>   # Logs for specific Talos service (etcd, scheduler, etc.)
core dmesg <ip>           # Kernel logs (ring buffer) from node

# Node lifecycle
core upgrade <ip>         # Upgrade Talos OS on node
core shutdown <ip>        # Graceful shutdown
core reboot <ip>          # Reboot node

# Kubernetes context
core config kube-list     # List available kubeconfig contexts
core config kube-use <ctx>  # Switch to kubeconfig context (LAN vs WireGuard)

# Pod cleanup
core pods clean           # Delete Failed/Evicted/Terminating pods
core node <node-name>     # Get node details (includes pod stats)

Secrets Management (Vault)

# Read secrets
core get <path>                      # Fetch all fields under path
core get <path> --key KEY_NAME       # Fetch specific field

# Write secrets
core put <path> key=value [key2=value2 ...]  # Store secrets in Vault

# List and export
core secrets list                    # List all secret paths
core secrets export <path>           # Export secrets as shell-sourceable format

# Execute with secrets in environment
core secrets exec -- <command>       # Run command with secrets loaded in env

Convention: Field name = variable name (SCREAMING_SNAKE_CASE). Never use value=.

Example:

# Write
core put cluster/ANTHROPIC_API_KEY ANTHROPIC_API_KEY="sk-ant-..."

# Read
core get cluster/ANTHROPIC_API_KEY

IAM Management (Authentik)

# Groups
core iam list-groups              # List all groups
core iam create-group <name>      # Create new group

# OAuth2 Applications
core iam list-apps                # List all OAuth2 apps
core iam create-app <name> --slug <slug> --redirect-uri <uri>  # Create app
core iam describe-app <app>       # Show client ID, secret, URIs, scope claims
core iam rotate-secret <app>      # Rotate OAuth2 client secret

# User management
core iam add-member <group> <username>  # Add user to group
core iam bind-app <app> <group>   # Grant group access to application

# Cleanup
core iam delete-app <app>         # Delete OAuth2 application

Object Storage (MinIO)

# List and manage buckets
core bucket list                  # List all buckets
core bucket upload <bucket> <file> [<remote-path>]  # Upload file to S3
core bucket download <bucket> <remote-path> <file>  # Download file from S3
core bucket delete <bucket> <remote-path>  # Delete object from S3

Utilities

core help                         # Show command help
core pf grafana                   # Port-forward to Grafana (localhost:3000)
core pf prometheus                # Port-forward to Prometheus (localhost:9090)
core pf minio                     # Port-forward to MinIO console (localhost:9001)
core pf iam                       # Port-forward to Authentik (localhost:7000)

---

Access Control Tiers

The cluster uses a tiered access model based on authentication mechanism and network perimeter.

Tier A: OIDC + RBAC (Authentik-enforced)

Services with federated OIDC login and group-based role assignment.

Service	Auth Method	Group Claim	Role Binding
Grafana	Authentik OIDC	groups	Mapped to Admin / Viewer / Viewer
Argo CD	Authentik OIDC	groups	RBAC role binding (policy.csv)
MinIO	Authentik OIDC	groups / custom policy	Policy-based access (readwrite / readonly)
Forgejo	Authentik OIDC	email (OAuth login only)	No group enforcement (open git repo)
kmsvc	Vault JWT	sub / aud	Audience validation + service scope

Provisioning:

# After core auth login-oob, bootstrap IAM apps:
bash k8s/talos-iam/bootstrap-iam.sh

Tier B: Network-Perimeter Only (No Authentik Enforcement)

Services with no OIDC support (product limitation). Access is restricted to LAN/WireGuard perimeter only.

Service	Network Access	Use Case
Portainer	LAN + WireGuard only	Container UI, workload browsing
Longhorn	LAN + WireGuard only	Storage volume management
Temporal	LAN + WireGuard only	Workflow execution (auth TBD)

All three are reachable only via WireGuard/LAN-only Ingress rules. Zero remote access risk, but also zero federated identity. If remote Temporal access is needed, upgrade the chart's auth configuration or replace with an OIDC-compatible workflow platform.

Accessing Tier B services:

# From off-LAN, use WireGuard context
core config kube-use admin@homelab-cluster-1
core pf minio  # connects via 10.6.0.1:9001

---

Workflow: Rotate an OAuth2 Application Secret

# 1. Rotate secret in Authentik
SECRET=$(core iam rotate-secret grafana | jq -r '.client_secret')

# 2. Update Helm values
vi k8s/logging/grafana-values.yaml
# Set: GRAFANA_OIDC_CLIENT_SECRET="$SECRET"

# 3. Redeploy the app
helmfile apply -l app=grafana

# 4. Verify new secret is in use
core iam describe-app grafana | grep client_secret

---

Workflow: Add User to Service

# 1. Verify group exists (or create it)
core iam list-groups | grep minio-admins

# If not found:
core iam create-group minio-admins

# 2. Add user to group
core iam add-member minio-admins alice

# 3. Bind group to MinIO application
core iam bind-app minio minio-admins

# 4. User has access on next login
# (OIDC login to MinIO → Authentik → group check → MinIO policy applied)

---

Workflow: Rotate a Database Password (Vault)

# 1. Generate new password
NEW_PASS=$(openssl rand -hex 32)

# 2. Store in Vault
core put cluster/POSTGRES_ADMIN_PASSWORD POSTGRES_ADMIN_PASSWORD="$NEW_PASS"

# 3. Update database user
kubectl exec -n ddb pod/ddb-cluster-0 -- psql -U postgres -c \
  "ALTER USER postgres WITH PASSWORD '$NEW_PASS';"

# 4. Update Helm values with new password reference
# (Or if using helmfile hook: helmfile will re-run postInitApplicationSQL with new password)

# 5. Restart pods to pick up new secret
kubectl rollout restart -n <ns> deployment/<app>

---

Common Questions

Q: I'm getting "not authenticated" on core nodes. What do I do? A: Run core auth login-oob. Node commands use a different auth domain than secrets. After login, core nodes should work.

Q: I have a valid Vault token but core get fails. Why? A: Check that both auth domains are active:

core auth status        # Verify node auth is valid
core secrets status     # Verify Vault auth is valid

Both must succeed. If one is expired, re-auth that domain.

Q: Should I use core bucket or S3 CLI tools (aws-cli, s3cmd)? A: Use core bucket for simplicity (no AWS credentials). Use s3cmd/aws-cli if you need advanced sync or bandwidth control. All three talk to the same MinIO backend.

Q: Can I add a user without creating a group first? A: Groups are the unit of access control. Always create the group, then add users to it, then bind it to applications. Single-user bindings are not supported (by design).

Q: I rotated an OAuth2 secret but the app still fails to authenticate. What's next? A:

1. Verify the new secret is stored: core iam describe-app grafana
2. Check the deployment has the new secret: kubectl get secret -n logging grafana-oidc -o yaml | grep client_secret
3. Restart the pod: kubectl rollout restart -n logging deployment/grafana
4. Check logs: kubectl logs -n logging deployment/grafana | grep -i oauth

---

See Also

• ~/workplace/core/USAGE.md — Exhaustive command reference with flags and examples
• CLAUDE.md § Sign In (Device Code Flow) — Quick reference for core auth login-oob
• CLAUDE.md § Quick Shortcuts — One-liners for common tasks
• CLAUDE.md § Integration Checklist — New service onboarding