tokio-console

tokio-console

tokio-console is an interactive debugging tool for async Rust applications. It provides real-time visibility into Tokio tasks, resources, and runtime behavior. Mantis supports tokio-console on all three components (Mandible, Thorax, Tarsus) when built with the tokio-console feature.

Caution

tokio-console adds runtime overhead and is intended for development and debugging only. Do not enable it in production.

Default Ports

Each Mantis component uses a unique port to avoid conflicts:

Component	Default Port	Environment Variable
Mandible	6670	MANDIBLE__CONSOLE__PORT
Thorax	6669 ¹	THORAX__CONSOLE__PORT
Tarsus	6671	TARSUS__CONSOLE__PORT

¹ Thorax has no compiled-in port default — its ConsoleConfig starts at the sentinel port 0, and Thorax refuses to start on 0 once the console is enabled, so THORAX__CONSOLE__PORT=6669 (or port in thorax.toml) must be set explicitly.

Mandible (6670) and Tarsus (6671) carry theirs as a whole-field serde default, which fires only while the console key is absent entirely. Set any one console key — TARSUS__CONSOLE__ENABLED=true on its own, or a [console] table without port — and the port falls to the sentinel 0 instead, binding an ephemeral port. Set ENABLED and PORT together.

Setup

1. Install tokio-console CLI

cargo install tokio-console --version 0.1.14 --locked

2. Build with tokio-console Feature

The tokio-console feature must be enabled at compile time:

# Build a single component. RUSTFLAGS="--cfg tokio_unstable" is REQUIRED:
# console-subscriber needs the unstable tokio cfg, and the repo ships no
# .cargo/config.toml to supply it -- without it the build succeeds but the
# console layer is inert.
RUSTFLAGS="--cfg tokio_unstable" cargo build -p mandible --features tokio-console

# Or use the shipped recipes, which set the flag for you: all three, or one
just build-console
just build-console-thorax
just build-console-mandible
just build-console-tarsus

# Or build Docker images with console support
ENABLE_TOKIO_CONSOLE=true docker compose \
    -f docker-compose.yml -f docker-compose.debug.yml \
    build

3. Enable in Configuration

Configure via TOML or environment variables:

TOML configuration:

[console]
enabled = true
address = "127.0.0.1"    # Use 0.0.0.0 in Docker
port = 6670              # Component-specific port
retention_secs = 60      # How long to retain completed task data
publish_interval_ms = 1000  # Update frequency
# recording_path = "/tmp/mantis-console.dat"  # Optional: record for replay

Environment variables:

MANDIBLE__CONSOLE__ENABLED=true
MANDIBLE__CONSOLE__ADDRESS=127.0.0.1
MANDIBLE__CONSOLE__PORT=6670
MANDIBLE__CONSOLE__RETENTION_SECS=60
MANDIBLE__CONSOLE__PUBLISH_INTERVAL_MS=1000

To build and run one component from the repository with the console on and its port set:

just run-mandible-console
just run-thorax-console
just run-tarsus-console

4. Connect

# Connect to Mandible
tokio-console http://localhost:6670

# Connect to Thorax
tokio-console http://localhost:6669

# Connect to Tarsus
tokio-console http://localhost:6671

Docker Compose Setup

Use the debug compose overlay to expose console ports:

ENABLE_TOKIO_CONSOLE=true docker compose \
    -f docker-compose.yml -f docker-compose.debug.yml \
    --profile thumbprint up

This overlay:

• Publishes 6669 (thorax) and 6670 (mandible) on the host — those two services carry no profile and start either way
• Publishes 6671 (tarsus) and 6672 (tarsus-thumbprint) only with a profile flag: both are profiled services (dev, with-client, thumbprint), so without --profile the containers never start and neither port is opened
• Sets CONSOLE__ADDRESS to 0.0.0.0 for container accessibility
• Enables the console server on each component

What You Can See

Task View

The main view shows all active Tokio tasks:

• Task name and ID — Identify specific async operations
• State — Running, idle, or completed
• Polls — Number of times the task was polled by the runtime
• Busy time — Total CPU time spent in the task
• Idle time — Time spent waiting for I/O or other async operations
• Total time — Wall clock time since task creation

Resource View

Shows async resources (timers, I/O handles, semaphores):

• Type — Timer, TcpStream, Semaphore, etc.
• Location — Source code location where the resource was created
• Tasks waiting — Number of tasks blocked on this resource

Useful Investigations

Symptom	What to Look For
High latency	Tasks with high idle time relative to busy time
CPU spikes	Tasks with excessive poll counts or busy time
Deadlocks	Tasks stuck in idle state with no progress
Resource leaks	Growing number of resources without cleanup
Slow dispatching	Dispatch queue tasks with increasing poll counts

Configuration Reference

Parameter	Default	Description
enabled	false	Enable the console server
address	127.0.0.1	Bind address (use 0.0.0.0 for Docker)
port	Component-specific (assigned at startup; serde default is sentinel 0)	Listen port
retention_secs	60	Seconds to retain completed task data
publish_interval_ms	1000	Milliseconds between data pushes to clients
recording_path	None	File path to record session data for offline analysis

Recording and Replay

Record a debugging session for later analysis:

[console]
enabled = true
recording_path = "/tmp/mantis-console-recording.dat"

The file is written for offline analysis. tokio-console itself has no replay flag — it connects to a live endpoint — so the recording is read with whatever tooling understands the format, not by the CLI.

Security Considerations

• Bind to 127.0.0.1 (not 0.0.0.0) when running outside Docker to prevent network exposure.
• The console server provides unauthenticated access to runtime internals. Never expose console ports to untrusted networks.
• Use SSH tunneling if you need remote access:

ssh -L 6670:localhost:6670 user@mantis-server
tokio-console http://localhost:6670

Next Steps

• Logging — Configure log levels and filters
• Prometheus Metrics — Quantitative monitoring
• Connection Problems — Debug connectivity issues