Troubleshooting

Start here when something isn’t working. Sections are organized by what you see, not what’s broken — pick the symptom that matches.

Commands use $NAMESPACE. For Helm, run export NAMESPACE=dreadnode, replacing dreadnode if you chose another namespace. For Embedded Cluster, run:

sudo ./dreadnode shell
export NAMESPACE=kotsadm

Do not run sudo kubectl inside the Embedded Cluster shell. See Access Kubernetes if kubectl tries to connect to 127.0.0.1:8080.

Diagnostic commands

These are useful regardless of the problem. Assume dreadnode as the release name throughout — substitute yours if different.

# All pods for the release
kubectl -n "$NAMESPACE" get pods -l app.kubernetes.io/instance=dreadnode

# Events (scheduling failures, image pull errors, probe failures)
kubectl -n "$NAMESPACE" get events --sort-by='.lastTimestamp'

# API logs
kubectl -n "$NAMESPACE" logs deploy/dreadnode-api

# API init container logs (migrations run here)
kubectl -n "$NAMESPACE" logs deploy/dreadnode-api -c migrations

# Health check (use http:// if the configured scheme is HTTP)
curl https://dreadnode.example.com/api/v1/health

Pods stuck in Pending

The pod can’t be scheduled. Check events:

kubectl -n "$NAMESPACE" describe pod <pod-name>

“no nodes available to schedule pods” or “Insufficient cpu/memory” — Your cluster doesn’t have enough allocatable resources. With bundled data stores and OpenSandbox enabled, the default small preset requests roughly 1.77 vCPU and 5.1 Gi. A 4 vCPU / 8 Gi node is therefore too close to the scheduler’s edge before sandbox workloads. Each active OpenSandbox pod requests another 0.25 CPU and 512 MiB by default. Use at least 8 vCPU / 16 Gi so Kubernetes, ingress, storage, and dynamic sandbox pods have headroom. See Resource sizing.

Runtime provisioning returns a scheduling timeout

When an OpenSandbox pod does not become ready before the lifecycle timeout, the API returns a 502 whose detail starts with OpenSandbox timed out waiting for the sandbox pod to become ready. The error includes that pod’s CPU and memory requests and lists likely causes. It is not a confirmed capacity verdict: image-pull failures, storage, taints, affinity, and other scheduler constraints can produce the same symptom.

Check the OpenSandbox server and recent namespace events immediately after reproducing the failure. The server removes the failed sandbox workload after its readiness timeout, so its pod and owned events may disappear quickly:

kubectl -n "$NAMESPACE" logs deploy/dreadnode-sandbox-server --since=10m
kubectl -n "$NAMESPACE" get events --sort-by='.lastTimestamp' | tail -50
kubectl describe node

If events report Insufficient cpu or Insufficient memory, add node capacity, reduce concurrent sandboxes, or lower the OpenSandbox pod requests after measuring the workload. In the Admin Console, open Config → Sandbox Runtime → Set OpenSandbox pod requests. For Helm, set dreadnode-api.config.opensandbox.resources.requests. The runtime’s configured resources remain pod limits, so lowering a request changes scheduling guarantees rather than the maximum burst.

Older releases can fail first with a bare ReadTimeout or Failed to provision sandbox. Upgrade before diagnosing those messages; they discard the server’s readiness response.

An OpenSandbox runtime URL returns 404, 502, or 503

Start with the wildcard Ingress, gateway pods, and exact runtime hostname returned by the API:

kubectl -n "$NAMESPACE" get ingress dreadnode-sandbox-server-gateway
kubectl -n "$NAMESPACE" get pods -l app.kubernetes.io/component=sandbox-gateway
kubectl -n "$NAMESPACE" logs deploy/dreadnode-sandbox-server-gateway --since=10m

export SANDBOX_HOST='<sandbox-id>-8787.sandbox.dreadnode.example.com'
curl -i "https://${SANDBOX_HOST}/api/health?probe=runtime-routing"

The wildcard DNS record must resolve *.sandbox.dreadnode.example.com to the ingress endpoint, and the selected TLS Secret must cover that wildcard. The gateway preserves the original HTTP or WebSocket path and query while routing the first DNS label to the sandbox and port.

• 404 — the sandbox ID does not exist, was deleted, or is ambiguous across namespaces. A 404 after reset is expected and confirms the route failed closed.
• 503 — the BatchSandbox exists but has no ready endpoint yet. Check its pod and namespace events.
• 502 — the gateway selected the sandbox but could not reach the runtime port. Check the sandbox pod, runtime process, and port 8787.
• Browser CSP error — redeploy after changing the sandbox domain. The frontend adds the configured HTTP and WebSocket wildcard sources to connect-src at runtime.

/api/health is public. Other runtime HTTP endpoints require the bearer token returned when the runtime starts; browsers exchange that token at /api/ws/ticket for a short-lived, single-use WebSocket ticket. Do not expose the authenticated dreadnode-sandbox-server lifecycle Service to fix a gateway problem.

“pod has unbound immediate PersistentVolumeClaims” — No StorageClass can provision the requested PVC. Check that a StorageClass exists:

kubectl get storageclass

If empty, install a storage provisioner (local-path, EBS CSI, Rook, etc.) before deploying Dreadnode. The preflight checks catch this, but only if you ran them.

The platform works but every runtime fails to connect

The runtime starts, the pod is Running, and the client still reports Could not connect to Dreadnode runtime server at https://<sandbox-id>-8787.sandbox.<your-domain>. The message names the underlying cause in parentheses — read that first, then match it below.

Runtimes are reached at a different authority than the platform, so a deployment whose platform connection is healthy can still fail on every runtime.

Cause in the message	What it means	Fix
Name or service not known, nodename nor servname provided	The wildcard sandbox authority does not resolve from the client	Add the *.sandbox.<your-domain> DNS record. A /etc/hosts entry cannot express a wildcard, so each new runtime gets a new name that fails
CERTIFICATE_VERIFY_FAILED, unable to get local issuer	The sandbox certificate chains to a CA the workstation does not trust	Install the organization root — see Trust an internal certificate
hostname mismatch, certificate is not valid for	The certificate does not cover the wildcard authority	Reissue the ingress certificate with *.sandbox.<your-domain> in its SAN list
ConnectTimeout, ReadTimeout	The name resolves but nothing answers, or the ingress is unreachable	Work through An OpenSandbox runtime URL returns 404, 502, or 503
HTTP 502, HTTP 503	The gateway answered but could not reach the runtime	Same section — this is a routing problem, not a client one

Confirm the same two facts from the workstation, using the hostname from the error:

export SANDBOX_HOST='<sandbox-id>-8787.sandbox.dreadnode.example.com'

# 1. Does the wildcard authority resolve?
getent hosts "$SANDBOX_HOST" || echo 'wildcard DNS record missing'

# 2. Does its certificate chain to a trusted root?
curl -sv --max-time 10 "https://${SANDBOX_HOST}/api/health" >/dev/null

If the plain curl fails and curl -k succeeds, the transport is fine and the problem is trust: install the organization root rather than disabling verification.

Pods in CrashLoopBackOff

The container starts and immediately exits. Check logs for the crashing container.

API pod: init container crash

The migrations init container runs alembic upgrade head before the API starts. If it fails, the pod shows Init:CrashLoopBackOff and the API never boots.

kubectl -n "$NAMESPACE" logs deploy/dreadnode-api -c migrations

connection refused or could not translate host name — The API can’t reach PostgreSQL. If using in-cluster Postgres, check that the dreadnode-postgresql StatefulSet has a Ready pod. If using an external database, verify the host, port, and network connectivity from inside the cluster.

password authentication failed or FATAL: role "..." does not exist — Wrong credentials. For in-cluster Postgres, the password lives in the dreadnode-postgresql Secret. If you deleted and recreated the Secret without deleting the PVC, the password on disk no longer matches. Delete the PVC and let both regenerate together.

ValidationError or missing required env — A required environment variable is missing or malformed. The API validates its config with Pydantic on startup. The error message names the exact field. Check the ConfigMap and Secrets for the API pod.

API pod: main container crash

If the init container succeeds but the main container crashes:

kubectl -n "$NAMESPACE" logs deploy/dreadnode-api

Look for Python tracebacks. The most common cause is a config value that passes validation but fails at runtime — a ClickHouse host that resolves but rejects connections, an S3 endpoint that times out, etc.

StatefulSet pods (PostgreSQL, ClickHouse, MinIO)

kubectl -n "$NAMESPACE" logs sts/dreadnode-postgresql
kubectl -n "$NAMESPACE" logs sts/dreadnode-clickhouse
kubectl -n "$NAMESPACE" logs sts/dreadnode-minio

The chart keeps bundled data-store Secrets across uninstall and reinstall, so a normal reinstall reuses the passwords already stored on the PVCs. If logs report an authentication failure, check whether someone manually deleted, recreated, or edited the Secret while preserving the PVC. Restore the original Secret from your backup.

Deleting a PVC is data destruction, not a password-recovery step. Use Reinstall from scratch only when you intend to discard the deployment.

Pods in ImagePullBackOff

The container runtime can’t pull the image.

kubectl -n "$NAMESPACE" describe pod <pod-name>

“unauthorized” or “authentication required” — The Replicated pull secret is missing or invalid. Check that the enterprise-pull-secret Secret exists in the namespace:

kubectl -n "$NAMESPACE" get secret enterprise-pull-secret

If missing, the license may not have been applied correctly. For Helm CLI installs, verify you logged in to the registry (helm registry login registry.replicated.com). For Embedded Cluster / KOTS, the license is injected automatically — check the Admin Console for license status.

If the Secret exists, verify that the failing pod references it:

kubectl -n "$NAMESPACE" get pod <pod-name> \
  -o jsonpath='{.spec.imagePullSecrets[*].name}{"\n"}'

OpenSandbox creates sandbox pods dynamically, so credentials on the server or controller Deployment do not automatically reach them. A sandbox pod that pulls from proxy.enterprise.dreadnode.io must list enterprise-pull-secret. If it does not, upgrade or redeploy Dreadnode so the OpenSandbox BatchSandbox template carries the registry credential.

A dockerRuntimeImage pointed at your own registry needs its own credential in global.imagePullSecrets — enterprise-pull-secret authenticates to the Dreadnode proxy and nothing else. See Trust a private CA in OpenSandbox.

“manifest unknown” or “not found” — The image tag doesn’t exist in the registry. This usually means the chart version and the published images are out of sync. Verify you’re installing a version that was promoted to your channel.

UI loads but API calls fail

You can see the Dreadnode login page, but interactions fail (login doesn’t work, pages show errors, network tab shows 404 or 502 on /api/* requests).

Check ingress routing. The frontend and API share a single hostname (<your-domain>). The ingress must route /api/* to the API service and / to the frontend service. If you see 404s on /api/*, the ingress isn’t routing correctly.

kubectl -n "$NAMESPACE" get ingress

Verify the API ingress has the correct host and paths configured.

Check the API pod is Ready. If the API pod isn’t passing health checks, the ingress controller won’t route traffic to it:

kubectl -n "$NAMESPACE" get pods -l app.kubernetes.io/name=dreadnode-api

Login fails silently

You enter credentials, the page reloads, but you’re not logged in. No error message.

Scheme mismatch. This is almost always caused by the configured URL scheme being https while you’re connecting over plain HTTP. The API sets Secure on authentication cookies when the scheme is HTTPS. Browsers silently refuse to store those cookies over HTTP.

Connect over HTTPS, or change the deployment to HTTP and redeploy:

Helm

Set global.scheme: http in the values overlay and run helm upgrade.

Admin Console

Set Config → Dreadnode Identity → URL Scheme to HTTP, then select Save config and Deploy.

KOTS ConfigValues

For a new automated installation:

apiVersion: kots.io/v1beta1
kind: ConfigValues
spec:
  values:
    scheme:
      value: http

Update an existing Embedded Cluster deployment through the Admin Console.

CORS mismatch. If you’re accessing the platform on a URL that doesn’t match global.domain (e.g., via IP address or a different hostname), the browser blocks cross-origin cookie writes. Access the platform on the exact domain you configured.

Requests fail with “Origin validation failed”

A state-changing request (login, or any save) returns HTTP 403 with a JSON body whose type is csrf_error. The API rejects unsafe requests whose browser origin does not match the application’s configured public URL.

The response names both URLs. The API log carries the same detail, so you can confirm it server-side without reproducing in a browser:

kubectl -n "$NAMESPACE" logs deploy/dreadnode-api | grep "CSRF origin check failed"
# CSRF origin check failed: method=POST path=/api/v1/... origin_source=Origin
# observed_origin=http://dreadnode.example.com allowed_origins=[https://dreadnode.example.com]

Compare observed_origin against allowed_origins. The comparison is exact, so all three of scheme, hostname, and port must match:

• Scheme differs (http:// observed, https:// allowed) — you reached the application over HTTP on an HTTPS-configured install. Connect over HTTPS; see TLS certificates for redirecting HTTP to HTTPS so this cannot happen.
• Hostname differs — you reached the application by IP address, or by a hostname that isn’t global.domain. Use the exact configured domain.
• Port differs — global.domain must include the port if users reach the application on a non-default one.

The result does not depend on whether the browser already holds session cookies, so a private window reproduces it identically. If it appears to work in one browser profile and not another, look for a difference in the URL rather than in cookie state.

Requests carrying an X-API-Key or Authorization header are exempt — the check exists for credentials a browser attaches automatically, which those are not. An SDK or CLI client seeing this error is being treated as a browser, which means it is sending an Origin or Referer header without a credential header.

Model deployment creation returns 503

If Admin → Model Deployments reports that LiteLLM model storage is not enabled, add this setting to your external LiteLLM configuration and restart it. The bundled LiteLLM chart already includes the setting.

general_settings:
  store_model_in_db: true

If the page reports that LiteLLM integration is disabled, enable both the proxy and API integration:

Helm

dreadnode-litellm:
  enabled: true
dreadnode-api:
  config:
    litellm:
      enabled: true

Run helm upgrade with the updated overlay.

Admin Console

Open Config → Inference Proxy, enable Enable bundled LiteLLM proxy, then save and deploy.

KOTS ConfigValues

apiVersion: kots.io/v1beta1
kind: ConfigValues
spec:
  values:
    litellm_enabled:
      value: "1"

Agent generation cannot reach the model proxy

Inspect the model route and its issued credential from the affected OpenSandbox pod. Replace <runtime-pod> with the UUID-named sandbox pod:

export RUNTIME_POD='<runtime-pod>'

kubectl -n "$NAMESPACE" exec "$RUNTIME_POD" -c sandbox -- bash -lc '
  set -eu
  printf "DREADNODE_LLM_BASE=%s\n" "$DREADNODE_LLM_BASE"
  curl -sS -o /tmp/models.json -w "HTTP %{http_code}\n" \
    -H "Authorization: Bearer $DREADNODE_LLM_API_KEY" \
    "$DREADNODE_LLM_BASE/models"
'
# DREADNODE_LLM_BASE=http://dreadnode-litellm:4000/v1
# HTTP 200

For bundled LiteLLM, an on-cluster OpenSandbox base should end in -litellm:4000/v1. A public /llm/v1 base in that pod means the deployment is running an older release or has an explicit OpenSandbox override. E2B runtimes should use the public route instead.

Use the response to isolate the failure:

• Could not resolve host — the configured OpenSandbox LiteLLM URL is not resolvable from the runtime cluster. Check the Service name or set a remote URL as described in Sandbox runtime.
• Connection timeout or refused — inspect the LiteLLM Service, endpoints, and runtime egress policy with kubectl -n "$NAMESPACE" get svc,endpoints dreadnode-litellm.
• HTTP 401 or 403 — the runtime received a credential that the proxy does not accept. Check the API and LiteLLM logs before rotating secrets.
• HTTP 200, but generation still fails — the sandbox-to-proxy route and credential work. Test the selected deployment in Admin → Model Deployments to diagnose its upstream provider.

To confirm a public-ingress hairpin problem, compare the internal request above with the public route from the same pod:

kubectl -n "$NAMESPACE" exec "$RUNTIME_POD" -c sandbox -- bash -lc '
  curl -sv --connect-timeout 5 --max-time 10 \
    "https://dreadnode.example.com/llm/v1/models" \
    -H "Authorization: Bearer $DREADNODE_LLM_API_KEY" \
    -o /dev/null
'

If the internal request returns 200 while this public request times out, the cluster cannot route pods through the public ingress. Keep the OpenSandbox model base on the internal route; do not change the public URL, because E2B and external clients still need it.

TLS issues

Start with the exact URL that fails: scheme, hostname, and port. Dreadnode application TLS uses https://<domain> on port 443. The Embedded Cluster Admin Console uses port 30000 and owns a separate certificate; changing the Dreadnode TLS Secret does not change the Admin Console certificate.

The application request path is:

client -> DNS/load balancer -> ports 80/443 -> ingress controller -> Ingress
       -> same-namespace kubernetes.io/tls Secret -> service

Work through that path in order. If a load balancer or service mesh terminates TLS before the cluster, inspect its certificate and routing instead, then use the upstream TLS termination configuration.

Set the diagnostic variables

For Embedded Cluster, enter the cluster shell from the directory that contains the installer:

sudo ./dreadnode shell
export NAMESPACE=kotsadm

Run kubectl without sudo inside this shell. For Helm, set NAMESPACE to the release namespace:

export NAMESPACE=dreadnode

Set the application hostname, TLS Secret resource name, and ingress IP:

export DOMAIN=dreadnode.example.com
export TLS_SECRET=dreadnode-tls
export INGRESS_IP=192.0.2.10

In the Embedded Cluster Admin Console, TLS Certificate Secret Name means the Kubernetes Secret resource name, such as dreadnode-tls. It is not a domain, certificate subject, filename, or AWS ARN.

Verify the Ingress and Secret wiring

Read the Ingress references before inspecting or changing the certificate:

kubectl -n "$NAMESPACE" get ingress \
  -o custom-columns='NAME:.metadata.name,CLASS:.spec.ingressClassName,HOSTS:.spec.rules[*].host,TLS_HOSTS:.spec.tls[*].hosts[*],TLS_SECRET:.spec.tls[*].secretName'

kubectl -n "$NAMESPACE" get secret "$TLS_SECRET" \
  -o jsonpath='{.metadata.name}{" "}{.type}{"\n"}'

Expect the configured ingress class, exact $DOMAIN and storage.$DOMAIN hosts, the same Secret on each relevant Ingress, and this Secret result:

dreadnode-tls kubernetes.io/tls

A missing or different reference means the ingress controller cannot select the intended certificate. For Embedded Cluster changes to URL Scheme, TLS Certificate Secret Name, ingress class, or redirect settings, select Save config, then Deploy. See TLS certificates for the configuration paths.

Compare the stored and served certificates

Inspect only the public certificate stored in the Secret:

kubectl -n "$NAMESPACE" get secret "$TLS_SECRET" \
  -o jsonpath='{.data.tls\.crt}' |
  base64 -d |
  openssl x509 -noout -subject -issuer -dates \
    -fingerprint -sha256 -ext subjectAltName

Never print, distribute, or ask another operator to share tls.key or the complete Secret YAML.

Inspect the certificate served through DNS, then bypass DNS and connect directly to the ingress IP. Both commands preserve the application hostname through Server Name Indication (SNI):

echo | openssl s_client \
  -connect "${DOMAIN}:443" \
  -servername "$DOMAIN" 2>/dev/null |
  openssl x509 -noout -subject -issuer -dates \
    -fingerprint -sha256 -ext subjectAltName

echo | openssl s_client \
  -connect "${INGRESS_IP}:443" \
  -servername "$DOMAIN" 2>/dev/null |
  openssl x509 -noout -subject -issuer -dates \
    -fingerprint -sha256 -ext subjectAltName

Matching stored and served SHA-256 fingerprints prove that the ingress controller serves the configured certificate. A browser warning can remain because certificate selection, chain validation, and client trust are separate checks. If the DNS and direct-IP results differ, inspect DNS A and AAAA records, load balancers, CloudFront, proxies, and other Dreadnode instances on the request path.

Validate the served chain

Show every certificate the server sends and require OpenSSL to validate the result:

echo | openssl s_client \
  -connect "${DOMAIN}:443" \
  -servername "$DOMAIN" \
  -showcerts -verify_return_error

echo | openssl s_client \
  -connect "${DOMAIN}:443" \
  -servername "$DOMAIN" \
  -showcerts 2>/dev/null |
  grep -c 'BEGIN CERTIFICATE'

tls.crt must contain the leaf certificate first, followed by intermediate certificates in order. Keep the private key in tls.key. Servers normally omit the root certificate because clients install it in their trust stores.

A public certificate that serves one certificate and reports unable to get local issuer certificate usually lacks an intermediate. Matching fingerprints do not rule this out: they prove the correct leaf is selected, not that the server provides a complete chain. For a private CA or self-signed certificate, follow Trust an internal certificate instead of copying private keys or disabling verification.

Validate replacement files before updating the Secret

Run these read-only checks against the proposed files:

openssl x509 -in fullchain.pem -noout \
  -subject -issuer -fingerprint -sha256 -ext subjectAltName

openssl pkey -in private-key.pem -check -noout

openssl x509 -in fullchain.pem -pubkey -noout |
  openssl pkey -pubin -outform DER |
  openssl sha256

openssl pkey -in private-key.pem -pubout -outform DER |
  openssl sha256

The first certificate in fullchain.pem must be the leaf for $DOMAIN and storage.$DOMAIN. The two public-key hashes must match. An intermediate placed first produces private key does not match public key; a newly issued leaf also requires its newly corresponding private key. Do not update the Secret when either check fails.

After all checks pass, apply the replacement idempotently:

kubectl -n "$NAMESPACE" create secret tls "$TLS_SECRET" \
  --cert=/absolute/path/to/fullchain.pem \
  --key=/absolute/path/to/private-key.pem \
  --dry-run=client -o yaml |
  kubectl apply -f -

secret/<name> configured means the update succeeded. A warning about a missing kubectl.kubernetes.io/last-applied-configuration annotation is benign for an imperatively created Secret and unrelated to TLS. Traefik watches Secret content changes automatically, so this update should not require a chart redeploy.

Verify the final transport

Confirm that port 80 redirects to HTTPS and that the API responds over port 443:

curl -I "http://${DOMAIN}/"
# Expect a method-preserving HTTPS redirect, normally HTTP 308 with Traefik.

curl "https://${DOMAIN}/api/v1/health"

Repeat the certificate and redirect checks with storage.${DOMAIN}. Confirm the SAN covers that exact hostname and that its stored and served fingerprints match.

Match the symptom to the next check

Symptom	Next check
TRAEFIK DEFAULT CERT	Check for a missing or wrong Secret reference, wrong namespace or SNI hostname, invalid certificate, or configuration that was not deployed. See Traefik’s default certificate.
Stored and served fingerprints differ	Check the ingress endpoint, stale Traefik state, DNS A and AAAA records, load balancer, CloudFront, proxy, and other instances.
Fingerprints match and OpenSSL reports unable to get local issuer certificate	Add the missing intermediate to the served chain, or install the intended private root in client trust stores.
OpenSSL reports self-signed certificate	Install the intended trust anchor on clients or replace the certificate with a publicly trusted one.
private key does not match public key	The leaf, key, or full-chain order is wrong. Correct the files and rerun both hashes; do not update the Secret.
Hostname mismatch	Replace the leaf with one whose SAN covers exact $DOMAIN and storage.$DOMAIN names. See Use an existing certificate.
Certificate expired or not yet valid	Compare the displayed validity dates with the client clock, then renew the certificate or correct the clock.
Browser still says Not secure after OpenSSL succeeds	Confirm https:// rather than http://, port 443 rather than Admin Console port 30000, the browser’s certificate identity and trust store, cached state, and mixed-content console warnings.
Browser succeeds but CLI fails	Install the CA in the OS trust store used by the Dreadnode process, then restart dn; managed browsers can use a separate trust policy.
API succeeds but trace or model traffic fails	Restart dn so each transport creates a fresh trust context. Configure OpenSandbox through its custom image extension.

Keep certificate verification enabled while diagnosing TLS failures.

S3 / MinIO issues

Presigned URL errors

The platform generates presigned S3 URLs for file downloads. If these fail, check that storage.<your-domain> resolves and is reachable from the user’s browser — presigned URLs point at the external S3 endpoint, not the internal one.

For in-cluster MinIO, verify the MinIO ingress exists and routes correctly:

kubectl -n "$NAMESPACE" get ingress dreadnode-minio

“Access Denied” or “NoSuchBucket”

The dreadnode-minio-bootstrap Helm hook creates python-packages, org-data, and user-data-logs after install and upgrade. If the hook failed while MinIO was unhealthy, inspect its logs:

kubectl -n "$NAMESPACE" logs job/dreadnode-minio-bootstrap

After MinIO is Ready, run the Helm upgrade again or redeploy through the Admin Console. The hook is idempotent and creates only missing buckets.

Support bundles

Support bundles collect logs, cluster state, and diagnostic information into a single archive you can share with us for debugging.

From the Admin Console (Embedded Cluster): Go to Troubleshoot and click Generate a support bundle.

From the Embedded Cluster CLI: From the directory that contains the installer, outside the Embedded Cluster shell:

sudo ./dreadnode support-bundle

From the CLI (Helm installs):

kubectl support-bundle --load-cluster-specs -n "$NAMESPACE"

This requires the troubleshoot kubectl plugin. The bundle spec is baked into the chart as a Secret with the troubleshoot.sh/kind: support-bundle label — the plugin discovers it automatically.

On an air-gapped install you need that binary staged before you disconnect, since it can’t be installed from inside the enclave. See Before you begin.

The bundle includes pod logs (up to 720 hours, 10,000 lines per pod), Helm release history, cluster resource state, your deployment configuration as the API reports it, and health checks against PostgreSQL, ClickHouse, object storage, and the inference proxy. Credentials are automatically redacted.

It also runs a set of checks over what it collected, so the summary printed at the end usually names the problem without anyone opening the archive. Read that first.

Sending us a bundle

If your Admin Console shows a Send bundle to vendor button under Troubleshoot, use it. The archive goes to us directly and there is nothing to transfer.

Otherwise send us the .tar.gz. On an air-gapped install, carry it out the same way you carried the install bundle in, and see Support bundles and telemetry for how often to collect one.

Missing license and version data in a Helm bundle

Some of the bundle is collected by running a command inside the Replicated SDK pod, which needs pods/exec with the create verb. Embedded Cluster and KOTS installs already have it. On a Helm install, whoever runs kubectl support-bundle needs it too, and without it those collectors fail without reporting anything: the bundle is produced as normal but arrives with no license or application version information.

If you run support bundles with a restricted account rather than a cluster administrator, add:

- apiGroups: ['']
  resources: ['pods/exec']
  verbs: ['create']