Implement Kubernetes-native CI/CD with Tekton Pipelines: ARCHITECTURE: - Tekton Task: Runs integration tests in container - Tekton Pipeline: Orchestrates test execution - ArgoCD Application: Manages Tekton installation - CI: Triggers PipelineRun, reads results, promotes image FLOW: 1. CI builds image:sha 2. CI creates PipelineRun with new image 3. Tekton controller watches PipelineRun 4. Task executes integration tests 5. Results written to PipelineRun status 6. CI reads status, promotes to :latest if pass 7. ArgoCD detects :latest change and deploys BENEFITS: ✓ Kubernetes-native (CRDs, no external dependencies) ✓ DRY (parameterized Task/Pipeline) ✓ SOLID (single responsibility, clean interfaces) ✓ GitOps (Tekton managed by ArgoCD) ✓ Observable (logs, status, results) ✓ Secure (non-root, resource limits) FILES: - k8s/tekton/task-integration-test.yaml: Task definition - k8s/tekton/pipeline-integration-test.yaml: Pipeline definition - k8s/tekton/kustomization.yaml: Kustomize management - k8s/tekton/README.md: Documentation - k8s/argocd-apps/tekton.yaml: ArgoCD Application - .gitea/workflows/ci.yaml: Updated CI to use Tekton NEXT: 1. Merge PR 2. ArgoCD syncs and installs Tekton 3. First git push triggers PipelineRun 4. Integration tests run in cluster 5. Results feedback to CI
This commit is contained in:
@@ -0,0 +1,130 @@
|
||||
# Tekton Integration Testing
|
||||
|
||||
Tekton Pipelines for running integration tests on API Gateway changes before merging to main.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Gitea CI (builds image:sha)
|
||||
↓
|
||||
Creates PipelineRun
|
||||
↓
|
||||
Tekton Controller (watches PipelineRun)
|
||||
↓
|
||||
Runs Task: integration-test
|
||||
↓
|
||||
Task runs tests in container
|
||||
↓
|
||||
Reports pass/fail to PipelineRun status
|
||||
↓
|
||||
CI reads status and promotes image (if pass)
|
||||
↓
|
||||
ArgoCD deploys new image
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
### Task: `integration-test`
|
||||
- **File**: `task-integration-test.yaml`
|
||||
- **Purpose**: Run integration tests in a container
|
||||
- **Inputs**: Image to test, timeout
|
||||
- **Outputs**: pass/fail result, message
|
||||
- **Security**: Non-root user, resource limits
|
||||
|
||||
### Pipeline: `integration-test-pipeline`
|
||||
- **File**: `pipeline-integration-test.yaml`
|
||||
- **Purpose**: Orchestrate integration test execution
|
||||
- **Tasks**: Runs the integration-test task
|
||||
- **Results**: Aggregates task results for CI consumption
|
||||
|
||||
## Usage
|
||||
|
||||
### Manual Trigger
|
||||
|
||||
```bash
|
||||
# Create a PipelineRun to test an image
|
||||
kubectl create -f - << 'YAML'
|
||||
apiVersion: tekton.dev/v1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
name: integration-test-manual
|
||||
namespace: api
|
||||
spec:
|
||||
pipelineRef:
|
||||
name: integration-test-pipeline
|
||||
params:
|
||||
- name: image
|
||||
value: forgejo.riotpiao.com/rock/api-gateway:abc123
|
||||
- name: test-timeout
|
||||
value: "5m"
|
||||
YAML
|
||||
|
||||
# Watch test progress
|
||||
kubectl logs -f -n api pipelinerun/integration-test-manual
|
||||
|
||||
# Check results
|
||||
kubectl get pipelinerun -n api integration-test-manual -o yaml
|
||||
```
|
||||
|
||||
### CI Trigger
|
||||
|
||||
CI automatically creates PipelineRun with:
|
||||
- Image tag: current commit SHA
|
||||
- Timeout: 5 minutes
|
||||
- Labels: PR ID, commit SHA for traceability
|
||||
|
||||
## Management
|
||||
|
||||
Tekton is managed by ArgoCD Application: `tekton-pipelines` (in `k8s/argocd-apps/tekton.yaml`)
|
||||
|
||||
To update:
|
||||
1. Edit manifest files
|
||||
2. Commit to git
|
||||
3. ArgoCD syncs automatically
|
||||
|
||||
Do NOT manually apply manifests - let ArgoCD manage everything.
|
||||
|
||||
## Monitoring
|
||||
|
||||
```bash
|
||||
# List all PipelineRuns
|
||||
kubectl get pipelineruns -n api
|
||||
|
||||
# Watch a specific run
|
||||
kubectl logs -f -n api pipelinerun/integration-test-<sha>
|
||||
|
||||
# Get detailed status
|
||||
kubectl describe pipelinerun -n api integration-test-<sha>
|
||||
```
|
||||
|
||||
## Results
|
||||
|
||||
PipelineRun status contains:
|
||||
- `status.conditions[0].reason`: Succeeded | Failed | Unknown
|
||||
- `status.taskRuns[*].status.taskResults`: Test outputs
|
||||
- Pod logs: Detailed test output
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **DRY**: Task and Pipeline are parameterized, reusable
|
||||
2. **SOLID**: Single responsibility (Task runs tests, Pipeline orchestrates)
|
||||
3. **GitOps**: Everything in git, managed by ArgoCD
|
||||
4. **Security**: Non-root containers, resource limits, no hardcoded values
|
||||
5. **Observability**: Clear logging, status tracking, result aggregation
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**PipelineRun stuck in Running**
|
||||
- Check pod logs: `kubectl logs -n api pod/<task-pod>`
|
||||
- Check gateway availability: `kubectl get pods -n api -l app=api-gateway`
|
||||
- Increase timeout in pipeline params
|
||||
|
||||
**Tests failing**
|
||||
- Check test logs: `kubectl logs -n api pipelinerun/<run-name>`
|
||||
- Verify gateway is ready and accessible
|
||||
- Check downstream services (memory, S3, etc.)
|
||||
|
||||
**Image not promoted**
|
||||
- CI only promotes if PipelineRun succeeds
|
||||
- Check PipelineRun status: `kubectl get pipelinerun <name> -n api -o yaml`
|
||||
- Review CI logs in Gitea for error details
|
||||
@@ -0,0 +1,44 @@
|
||||
# Tekton Pipelines Release manifest
|
||||
# Source: https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
|
||||
# This is managed by ArgoCD - do NOT manually apply
|
||||
# ArgoCD syncs this from git
|
||||
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: tekton-pipelines
|
||||
labels:
|
||||
managed-by: argocd
|
||||
|
||||
---
|
||||
# CRDs and RBAC are part of the full release manifest
|
||||
# Using a reference approach for cleaner GitOps
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: ApplicationSet
|
||||
metadata:
|
||||
name: tekton-pipelines
|
||||
namespace: argocd
|
||||
spec:
|
||||
generators:
|
||||
- list:
|
||||
elements:
|
||||
- name: tekton-pipelines
|
||||
template:
|
||||
metadata:
|
||||
name: tekton-pipelines
|
||||
namespace: argocd
|
||||
spec:
|
||||
project: default
|
||||
source:
|
||||
repoURL: https://github.com/tektoncd/operator
|
||||
targetRevision: main
|
||||
path: config/release
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: tekton-pipelines
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
@@ -0,0 +1,15 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: api-gateway-tekton
|
||||
|
||||
namespace: api
|
||||
|
||||
resources:
|
||||
- task-integration-test.yaml
|
||||
- pipeline-integration-test.yaml
|
||||
|
||||
commonLabels:
|
||||
app: api-gateway
|
||||
component: testing
|
||||
managed-by: argocd
|
||||
@@ -0,0 +1,35 @@
|
||||
apiVersion: tekton.dev/v1
|
||||
kind: Pipeline
|
||||
metadata:
|
||||
name: integration-test-pipeline
|
||||
namespace: api
|
||||
labels:
|
||||
app: api-gateway
|
||||
component: testing
|
||||
spec:
|
||||
description: Pipeline to run integration tests for API gateway
|
||||
params:
|
||||
- name: image
|
||||
type: string
|
||||
description: Container image to test (repo:tag)
|
||||
default: "forgejo.riotpiao.com/rock/api-gateway:latest"
|
||||
- name: test-timeout
|
||||
type: string
|
||||
default: "5m"
|
||||
description: Test execution timeout
|
||||
results:
|
||||
- name: test-result
|
||||
description: Overall test result (pass/fail)
|
||||
value: $(tasks.run-integration-tests.results.result)
|
||||
- name: test-message
|
||||
description: Test summary message
|
||||
value: $(tasks.run-integration-tests.results.message)
|
||||
tasks:
|
||||
- name: run-integration-tests
|
||||
taskRef:
|
||||
name: integration-test
|
||||
params:
|
||||
- name: image
|
||||
value: $(params.image)
|
||||
- name: timeout
|
||||
value: $(params.test-timeout)
|
||||
@@ -0,0 +1,85 @@
|
||||
apiVersion: tekton.dev/v1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: integration-test
|
||||
namespace: api
|
||||
labels:
|
||||
app: api-gateway
|
||||
component: testing
|
||||
spec:
|
||||
description: Run integration tests for API gateway
|
||||
params:
|
||||
- name: image
|
||||
type: string
|
||||
description: Container image to test (including tag)
|
||||
- name: timeout
|
||||
type: string
|
||||
default: "5m"
|
||||
description: Test timeout
|
||||
results:
|
||||
- name: result
|
||||
description: Test result (pass/fail)
|
||||
type: string
|
||||
- name: message
|
||||
description: Test summary message
|
||||
type: string
|
||||
steps:
|
||||
- name: run-tests
|
||||
image: $(params.image)
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65532
|
||||
allowPrivilegeEscalation: false
|
||||
env:
|
||||
- name: GATEWAY_URL
|
||||
value: "http://api-gateway:8080"
|
||||
- name: CI
|
||||
value: "true"
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "🧪 Starting integration tests..."
|
||||
echo "Image: $(params.image)"
|
||||
echo "Gateway: $GATEWAY_URL"
|
||||
echo ""
|
||||
|
||||
# Wait for gateway to be ready
|
||||
echo "Waiting for gateway service..."
|
||||
for i in $(seq 1 30); do
|
||||
if curl -s $GATEWAY_URL/healthz > /dev/null 2>&1; then
|
||||
echo "✓ Gateway is ready"
|
||||
break
|
||||
fi
|
||||
echo "Attempt $i/30: Waiting for gateway..."
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Run integration tests
|
||||
echo "Running integration tests..."
|
||||
if go test -v -tags=integration -timeout=$(params.timeout) ./internal/integration/...; then
|
||||
echo "pass" | tee $(results.result.path)
|
||||
echo "✓ All integration tests passed" | tee $(results.message.path)
|
||||
exit 0
|
||||
else
|
||||
echo "fail" | tee $(results.result.path)
|
||||
echo "✗ Some integration tests failed" | tee $(results.message.path)
|
||||
exit 1
|
||||
fi
|
||||
volumeMounts:
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
- name: home
|
||||
mountPath: /home/nonroot
|
||||
resources:
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 1Gi
|
||||
volumes:
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
- name: home
|
||||
emptyDir: {}
|
||||
Reference in New Issue
Block a user