Kubernetes Installation
Kubernetes Installation
Section titled “Kubernetes Installation”Deploy Mantis on Kubernetes for production-grade scalability and high availability.
Prerequisites
Section titled “Prerequisites”Kubernetes Cluster
Section titled “Kubernetes Cluster”| Requirement | Version |
|---|---|
| Kubernetes | 1.28+ |
| kubectl | 1.28+ |
| Helm | 3.12+ (optional) |
Infrastructure
Section titled “Infrastructure”The following services must be available (managed or self-hosted):
| Service | Purpose |
|---|---|
| PostgreSQL | Primary database |
| RabbitMQ | Message queue |
| Redis | Session and cache |
Namespace Setup
Section titled “Namespace Setup”# Create namespacekubectl create namespace mantis
# Set as default contextkubectl config set-context --current --namespace=mantisSecrets Configuration
Section titled “Secrets Configuration”Database Credentials
Section titled “Database Credentials”apiVersion: v1kind: Secretmetadata: name: mantis-database namespace: mantistype: OpaquestringData: url: 'postgres://mantis:secret@postgres.internal:5432/mantis?sslmode=require'JWT Secret
Section titled “JWT Secret”apiVersion: v1kind: Secretmetadata: name: mantis-jwt namespace: mantistype: OpaquestringData: secret: 'your-256-bit-secret-key-at-least-32-characters' token-lookup: '<output of openssl rand -base64 32>'Encryption Key
Section titled “Encryption Key”apiVersion: v1kind: Secretmetadata: name: mantis-encryption namespace: mantistype: OpaquestringData: master-key: '<output of openssl rand -base64 32>'TLS Certificates
Section titled “TLS Certificates”apiVersion: v1kind: Secretmetadata: name: mantis-tls namespace: mantistype: kubernetes.io/tlsdata: tls.crt: <base64-encoded-certificate> tls.key: <base64-encoded-private-key> ca.crt: <base64-encoded-ca-certificate>Create secrets:
kubectl apply -f secrets/ConfigMaps
Section titled “ConfigMaps”Mandible Configuration
Section titled “Mandible Configuration”apiVersion: v1kind: ConfigMapmetadata: name: mandible-config namespace: mantisdata: MANDIBLE__SERVER__PORT: '3000' MANDIBLE__JWT__ALGORITHM: 'HS256' MANDIBLE__JWT__ACCESS_TOKEN_EXPIRY_SECS: '86400' MANDIBLE__CORS__ALLOWED_ORIGINS: 'https://lens.example.com' # Log filtering uses the standard tracing EnvFilter RUST_LOG: 'info' MANDIBLE__GRPC__ORCHESTRATION_URL: 'https://thorax-headless.mantis.svc.cluster.local:50051' # The internal gRPC server (Tarsus/Thorax registration + persistence) is OFF by # default. Leave it off and Thorax cannot register or persist. MANDIBLE__INTERNAL_GRPC__ENABLED: 'true' MANDIBLE__INTERNAL_GRPC__PORT: '50052'Thorax Configuration
Section titled “Thorax Configuration”apiVersion: v1kind: ConfigMapmetadata: name: thorax-config namespace: mantisdata: THORAX__GRPC__PORT: '50051' MANTIS_TLS_AUTH_MODE: 'ca_signed' THORAX__CLUSTER__REDIS_URL: 'redis://:password@redis.internal:6379' MANTIS_LOG_LEVEL: 'info' # Session timeouts. Defaults are heartbeat_timeout_secs=90, stale_timeout_secs=300, # cleanup_interval_secs=60; set them here to override thorax.toml. THORAX__SESSION__HEARTBEAT_TIMEOUT_SECS: '90' THORAX__SESSION__STALE_TIMEOUT_SECS: '300' THORAX__SESSION__CLEANUP_INTERVAL_SECS: '60' # Queue (RabbitMQ) settings can be configured via env vars — the most # commonly needed keys are shown below; omit any that are set in thorax.toml. THORAX__QUEUE__HOST: 'rabbitmq.internal' THORAX__QUEUE__PORT: '5671' THORAX__QUEUE__VHOST: '/mantis'Deployments
Section titled “Deployments”Mandible Deployment
Section titled “Mandible Deployment”apiVersion: apps/v1kind: Deploymentmetadata: name: mandible namespace: mantis labels: app: mantis component: mandiblespec: replicas: 2 selector: matchLabels: app: mantis component: mandible template: metadata: labels: app: mantis component: mandible spec: containers: - name: mandible image: ghcr.io/mantisplatform/mantis-mandible:1.0.0 ports: - containerPort: 3000 name: http # Thorax connects here; without exposing 50052 nothing listens for it - containerPort: 50052 name: internal-grpc envFrom: - configMapRef: name: mandible-config env: - name: DATABASE_URL valueFrom: secretKeyRef: name: mantis-database key: url - name: MANDIBLE__JWT__SECRET valueFrom: secretKeyRef: name: mantis-jwt key: secret # Keys the lookup that finds a stored API key or OAuth2 token. Required # whichever JWT algorithm is in use; changing it invalidates every one. - name: MANDIBLE__SECURITY__TOKEN_LOOKUP_KEY valueFrom: secretKeyRef: name: mantis-jwt key: token-lookup - name: MANTIS_ENCRYPTION_KEY valueFrom: secretKeyRef: name: mantis-encryption key: master-key volumeMounts: - name: tls-certs mountPath: /etc/mantis/certs readOnly: true resources: requests: cpu: 250m memory: 256Mi limits: cpu: 1000m memory: 512Mi livenessProbe: httpGet: path: /api/v1/health/live port: http initialDelaySeconds: 10 periodSeconds: 10 readinessProbe: httpGet: path: /api/v1/health/ready port: http initialDelaySeconds: 5 periodSeconds: 5 volumes: - name: tls-certs secret: secretName: mantis-tlsThorax Deployment
Section titled “Thorax Deployment”apiVersion: apps/v1kind: Deploymentmetadata: name: thorax namespace: mantis labels: app: mantis component: thoraxspec: replicas: 2 selector: matchLabels: app: mantis component: thorax template: metadata: labels: app: mantis component: thorax spec: containers: - name: thorax image: ghcr.io/mantisplatform/mantis-thorax:1.0.0 ports: - containerPort: 50051 name: grpc envFrom: - configMapRef: name: thorax-config env: # Thorax TLS is unconditional -- without these three the pod fails # validation on startup ("No server certificate configured"). - name: MANTIS_TLS_CERT_PATH value: /etc/mantis/certs/thorax.crt - name: MANTIS_TLS_KEY_PATH value: /etc/mantis/certs/thorax.key - name: MANTIS_TLS_CA_CERT_PATH value: /etc/mantis/certs/ca.crt # Thorax reaches Mandible over mTLS gRPC, so the client-cert paths are # required, and Mandible's internal gRPC (:50052) must be enabled -- # it is OFF by default. Both the Mandible Deployment and its Service # on this page declare 50052. - name: THORAX__MANDIBLE__ENDPOINT value: 'https://mandible:50052' - name: THORAX__MANDIBLE__TLS_CERT_PATH value: /etc/mantis/certs/thorax.crt - name: THORAX__MANDIBLE__TLS_KEY_PATH value: /etc/mantis/certs/thorax.key - name: THORAX__MANDIBLE__TLS_CA_CERT_PATH value: /etc/mantis/certs/ca.crt - name: THORAX__CLUSTER__INSTANCE_ID valueFrom: fieldRef: fieldPath: metadata.name volumeMounts: - name: tls-certs mountPath: /etc/mantis/certs readOnly: true resources: requests: cpu: 500m memory: 512Mi limits: cpu: 2000m memory: 1Gi # Thorax probes itself. Its health listener binds loopback by default, where a # kubelet httpGet cannot reach it, and a grpc: or tcpSocket probe of 50051 is a # failed TLS handshake that the connection limiter counts against the prober. livenessProbe: exec: command: ['/usr/local/bin/thorax', '--health-check', 'live'] initialDelaySeconds: 20 periodSeconds: 20 timeoutSeconds: 8 readinessProbe: exec: command: ['/usr/local/bin/thorax', '--health-check'] periodSeconds: 10 timeoutSeconds: 8 volumes: - name: tls-certs secret: secretName: mantis-tlsLens Deployment
Section titled “Lens Deployment”apiVersion: apps/v1kind: Deploymentmetadata: name: lens namespace: mantis labels: app: mantis component: lensspec: replicas: 2 selector: matchLabels: app: mantis component: lens template: metadata: labels: app: mantis component: lens spec: containers: - name: lens image: ghcr.io/mantisplatform/mantis-lens:1.0.0 ports: - containerPort: 4173 name: http env: # Runtime config is MANTIS_API_URL; VITE_API_URL is only a # build-time fallback baked into the bundle and is ignored here. - name: MANTIS_API_URL value: 'https://api.mantis.example.com' resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 256Mi livenessProbe: httpGet: path: / port: http initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: / port: http initialDelaySeconds: 3 periodSeconds: 5Services
Section titled “Services”Mandible Service
Section titled “Mandible Service”apiVersion: v1kind: Servicemetadata: name: mandible namespace: mantisspec: selector: app: mantis component: mandible ports: - port: 3000 targetPort: http name: http # Thorax dials this port; a ClusterIP Service routes only the ports it declares, # so omitting it leaves every Thorax pod with connection-refused. - port: 50052 targetPort: internal-grpc name: internal-grpc type: ClusterIPThorax Service
Section titled “Thorax Service”apiVersion: v1kind: Servicemetadata: name: thorax namespace: mantisspec: selector: app: mantis component: thorax ports: - port: 50051 targetPort: grpc name: grpc type: ClusterIP---# Headless service for gRPC load balancingapiVersion: v1kind: Servicemetadata: name: thorax-headless namespace: mantisspec: selector: app: mantis component: thorax ports: - port: 50051 targetPort: grpc name: grpc clusterIP: NoneThorax External Service (for Tarsus agents)
Section titled “Thorax External Service (for Tarsus agents)”apiVersion: v1kind: Servicemetadata: name: thorax-external namespace: mantis annotations: # For cloud provider load balancers service.beta.kubernetes.io/aws-load-balancer-type: nlb service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcpspec: selector: app: mantis component: thorax ports: - port: 50051 targetPort: grpc name: grpc type: LoadBalancerLens Service
Section titled “Lens Service”apiVersion: v1kind: Servicemetadata: name: lens namespace: mantisspec: selector: app: mantis component: lens ports: - port: 4173 targetPort: http name: http type: ClusterIPIngress
Section titled “Ingress”Ingress Configuration (nginx)
Section titled “Ingress Configuration (nginx)”apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: mantis-ingress namespace: mantis annotations: nginx.ingress.kubernetes.io/ssl-redirect: 'true' nginx.ingress.kubernetes.io/proxy-body-size: '50m' cert-manager.io/cluster-issuer: 'letsencrypt-prod'spec: ingressClassName: nginx tls: - hosts: - mantis.example.com - api.mantis.example.com secretName: mantis-ingress-tls rules: - host: mantis.example.com http: paths: - path: / pathType: Prefix backend: service: name: lens port: number: 4173 - host: api.mantis.example.com http: paths: - path: / pathType: Prefix backend: service: name: mandible port: number: 3000Database Migrations
Section titled “Database Migrations”Migration Job
Section titled “Migration Job”apiVersion: batch/v1kind: Jobmetadata: name: mantis-migration namespace: mantisspec: template: spec: containers: - name: migration image: ghcr.io/mantisplatform/mantis-mandible:1.0.0 command: ['mandible', '--migrate-only'] env: - name: DATABASE_URL valueFrom: secretKeyRef: name: mantis-database key: url restartPolicy: OnFailure backoffLimit: 3Run migrations:
kubectl apply -f jobs/migration-job.yamlkubectl wait --for=condition=complete job/mantis-migration --timeout=120sHorizontal Pod Autoscaler
Section titled “Horizontal Pod Autoscaler”Mandible HPA
Section titled “Mandible HPA”apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: mandible-hpa namespace: mantisspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: mandible minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80Thorax HPA
Section titled “Thorax HPA”apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: thorax-hpa namespace: mantisspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: thorax minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70Pod Disruption Budgets
Section titled “Pod Disruption Budgets”apiVersion: policy/v1kind: PodDisruptionBudgetmetadata: name: mandible-pdb namespace: mantisspec: minAvailable: 1 selector: matchLabels: app: mantis component: mandible---# pdb/thorax-pdb.yamlapiVersion: policy/v1kind: PodDisruptionBudgetmetadata: name: thorax-pdb namespace: mantisspec: minAvailable: 1 selector: matchLabels: app: mantis component: thoraxNetwork Policies
Section titled “Network Policies”Restrict Internal Traffic
Section titled “Restrict Internal Traffic”apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: default-deny namespace: mantisspec: podSelector: {} policyTypes: - Ingress - Egress---# network-policies/allow-mantis-internal.yamlapiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: allow-mantis-internal namespace: mantisspec: podSelector: matchLabels: app: mantis policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: mantis # Without this the ingress controller cannot reach the lens/mandible Services # the Ingress above routes to -- default-deny drops it. Change the namespace # if your controller does not run in ingress-nginx. - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: ingress-nginx egress: # Cluster DNS. A default-deny egress policy blocks port 53 too, and every # FQDN on this page then fails to resolve. - to: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: kube-system ports: - protocol: UDP port: 53 - protocol: TCP port: 53 - to: - podSelector: matchLabels: app: mantis - to: - namespaceSelector: {} podSelector: matchLabels: app: postgres - to: - namespaceSelector: {} podSelector: matchLabels: app: rabbitmq - to: - namespaceSelector: {} podSelector: matchLabels: app: redisDeployment Order
Section titled “Deployment Order”Deploy resources in this order:
# 1. Create namespace and secretskubectl apply -f namespace.yamlkubectl apply -f secrets/
# 2. Create configmapskubectl apply -f configmaps/
# 3. Run migrationskubectl apply -f jobs/migration-job.yamlkubectl wait --for=condition=complete job/mantis-migration
# 4. Deploy serviceskubectl apply -f services/
# 5. Deploy applicationskubectl apply -f deployments/
# 6. Configure ingresskubectl apply -f ingress/
# 7. Set up autoscaling and policieskubectl apply -f hpa/kubectl apply -f pdb/kubectl apply -f network-policies/Verification
Section titled “Verification”Check Deployments
Section titled “Check Deployments”kubectl get deployments -n mantiskubectl get pods -n mantiskubectl get services -n mantisCheck Health
Section titled “Check Health”# Port forward to check health endpointskubectl port-forward svc/mandible 3000:3000 -n mantiscurl http://localhost:3000/api/v1/health
# Thorax. Its gRPC port requires a client certificate, so health is read from# the plain-HTTP listener instead; $METRICS_TOKEN is metrics.auth_token, when set.kubectl port-forward deploy/thorax 9090:9090 -n mantiscurl -H "Authorization: Bearer $METRICS_TOKEN" http://localhost:9090/healthView Logs
Section titled “View Logs”kubectl logs -l app=mantis,component=mandible -n mantiskubectl logs -l app=mantis,component=thorax -n mantisTroubleshooting
Section titled “Troubleshooting”Pods Not Starting
Section titled “Pods Not Starting”# Check pod eventskubectl describe pod <pod-name> -n mantis
# Check container logskubectl logs <pod-name> -n mantis --previousDatabase Connection Issues
Section titled “Database Connection Issues”- Verify secret contains correct URL
- Check network policies allow egress to database
- Ensure database is accessible from cluster
Certificate Issues
Section titled “Certificate Issues”- Verify secrets contain valid certificates
- Check certificate expiration dates
- Ensure CA certificate is correct
Next Steps
Section titled “Next Steps”- Manual Installation - Install without containers
- Upgrades - Upgrade procedures
- Scaling - Scale Thorax instances
