unchecked-io/docs/tech_fix.md

Here is the technical summary of our debugging session, formatted as a Markdown file for your documentation.

Markdown

Technical Post-Mortem: The "Thrashing" & Performance Optimization Log

Date: 2025-11-21 Project: UncheckedIO Topic: Debugging "Astronomical" Task Switching Delays & Laptop vs. Cloud Discrepancies

1. The Symptoms

• The "Astronomical" Delay: Increasing partition count (e.g., from 16 to 32) caused disproportionate slowdowns. Switching between tasks took seconds, not nanoseconds.
• The "Tracy" Paradox: The profiler showed "CPU Data Starved" and frequently crashed the runtime. When it crashed/detached, the library actually ran faster.
• Platform Discrepancy: The library performed significantly better on Google Colab (Server CPU) than on a high-spec Laptop (Consumer CPU) relative to competitors, despite the laptop having faster single-core speed.

2. Root Cause Analysis

A. The Database Villain: "The Missing Index"

• The Issue: The setup_db.sql script created the table but failed to index the id column.
• The Consequence: The "Parallel Coordinator" splits the job into partitions (e.g., WHERE id BETWEEN 0 AND 125000). Without an index, Postgres performed a Sequential Scan (Full Table Scan) for every single partition.
• The Impact: Running 16 parallel workers triggered 16 simultaneous Full Table Scans. This saturated memory bandwidth instantly, explaining the "CPU Data Starved" metric in Tracy.

B. The Concurrency Villain: "Pool Contention & Thrashing"

• The Issue: The connection pool was hardcoded to max_size(20) in src/parser.rs.
• The "Alive" Misconception: We assumed keeping connections "alive" allowed infinite parallelism. However, a connection is a single-lane bridge; it can only transport one query at a time.
• The Bottleneck: When spawning 800 tasks (partitions) against 20 connections:
  • 20 Tasks ran immediately.
  • 780 Tasks sat frozen in a RAM queue.
• The "Thrashing": The CPU spent ~90% of its time managing this queue—saving/restoring task states (Context Switching) and flushing L1/L2 caches—rather than parsing data. This overhead destroyed performance on the laptop (which has smaller caches than the Colab server).

C. The Red Herring: io_uring

• Hypothesis: We initially thought switching to io_uring (Linux asynchronous I/O) would solve the "waiting."
• Verdict: Incorrect. The bottleneck was not syscall overhead (which io_uring solves); it was Logical Contention (waiting for a DB connection) and Physical Latency (waiting for RAM/Network).

---

3. The Solutions

Fix #1: Create the Index (Database Side)

Change the operation from a Sequential Scan (reading 20M rows x 16 times) to an Index Scan (jumping directly to the data).

File: setup_db.sql

-- OLD
ANALYZE benchmark_table;

-- NEW
CREATE INDEX idx_benchmark_id ON benchmark_table(id);
ANALYZE benchmark_table;
Fix #2: Dynamic Pool Sizing (Rust Side)
Remove the artificial speed limit. Match the pool size to the hardware capabilities so tasks never wait in a queue.

File: src/parser.rs

Rust

// OLD
.max_size(20)

// NEW
use num_cpus; // Add to Cargo.toml
let num_threads = num_cpus::get();
.max_size(num_threads) // 1:1 mapping of Threads to Connections
Fix #3: Auto-Tuned "Blast Radius"
Instead of a hardcoded partition size or user guess, calculate the optimal partition count to balance 
load without causing overhead.

Strategy: 4 * CPU Cores partitions.

Why: Enough chunks to allow "Work Stealing" (fast threads take more work), but few enough to prevent
connection setup overhead.

Rust

pub fn calculate_optimal_blast_radius(min_id: i64, max_id: i64) -> i64 {
    let total_rows = max_id - min_id;
    let target_partitions = num_cpus::get() * 4; 
    total_rows / (target_partitions as i64)
}
4. Conclusion
The library's "Schema-on-Read" optimization (trusting the config) was working perfectly. 
The performance loss was due to infrastructure bottlenecks (DB Index and Connection Pool limits) 
that forced the CPU to manage queues instead of parsing data.