Skip to main content

Tokio Runtime

TL;DR Tokio's multi_thread runtime runs a fixed pool of OS threads. Each thread has a local task queue. When a thread runs out of work, it steals tasks from other threads. I/O events are delivered via epoll/kqueue/IOCP, not busy-polling.

Runtime Flavors​

// Multi-thread (default for servers) — N OS threads, work-stealing
#[tokio::main]
async fn main() { ... }

// Equivalent explicit construction:
tokio::runtime::Builder::new_multi_thread()
    .worker_threads(num_cpus::get())
    .enable_all()
    .build()
    .unwrap()
    .block_on(async { ... });

// Current-thread — single OS thread, good for tests and CLI tools
#[tokio::main(flavor = "current_thread")]
async fn main() { ... }

Task Scheduling​

Each worker thread has:

  1. A local run queue (LIFO) — fast path for newly spawned tasks
  2. A global injection queue — external tasks go here first
  3. A steal queue — half-open ring buffer that other threads can steal from

When a thread's local queue is empty, it:

  1. Checks the global queue
  2. Steals from a random other thread's steal queue
  3. Polls for I/O events

The LIFO local queue gives cache-friendly locality: a task that just unblocked typically runs on the same thread that woke it.

I/O Driver​

Tokio wraps mio, which wraps OS-level async I/O:

Linux:   epoll
macOS:   kqueue
Windows: IOCP

When you call TcpStream::read().await, Tokio registers interest with the I/O driver. The task is parked. When the OS signals readiness, Tokio wakes the task's Waker, which queues it for polling.

Timer Wheel​

Tokio implements timers using a hierarchical timing wheel. tokio::time::sleep doesn't spawn a thread or use OS timers — it registers a deadline in the wheel. The I/O thread checks expired timers each tick.

tokio::time::sleep(Duration::from_millis(100)).await;
// efficient: no thread created, no system call until expiry

Blocking Work​

CPU-bound or blocking syscalls block the current OS thread, starving other tasks. Use the blocking pool:

let result = tokio::task::spawn_blocking(|| {
    // runs on a separate thread pool (default: 512 max threads)
    std::fs::read_to_string("big_file.txt")
}).await?;

spawn_blocking threads are distinct from worker threads. They're created on demand, reused, and capped by max_blocking_threads (default 512).

Metrics and Tuning​

tokio::runtime::Builder::new_multi_thread()
    .worker_threads(4)              // default: num CPUs
    .max_blocking_threads(128)      // default: 512
    .thread_name("adarsh-worker")
    .on_thread_park(|| {})          // hook when thread parks
    .enable_all()
    .build()?;

For observability: tokio-console provides a live view of task states, poll times, and waker activity.

[dependencies]
console-subscriber = "0.4"
// in main, before runtime starts
console_subscriber::init();

Common Mistakes​

Spawning too many tasks for CPU-bound work — tasks are cheap but not free. For CPU work, use rayon or spawn_blocking and join a fixed number of threads.

Not sizing the blocking pool — the default 512 means 512 simultaneous blocking threads. If you have a tight blocking loop, cap it.

select! without cancellation safety — cancelling a branch in select! drops the future. If the future holds state across yields (e.g., partially-written buffer), this is a bug. Check docs for cancellation safety.

→Async/AwaitHow Futures work and how .await suspends tasks →Arc<Mutex<T>>Sharing state between Tokio tasks