3.8 KiB
UncheckedIO — 10-Minute Technical Talk
Audience: IT/Cybersecurity group with programmers and ML engineers Goal: Plant the seed, demonstrate the mechanism, invite connections to LLM infrastructure teams
Slide 1 — Open with the hidden thing (0:00–0:45)
Lead with an Nsight Systems timeline screenshot. Don't explain it yet.
"This is what happens inside your GPU when a language model processes your request. The orange is waiting. The GPU costs $30,000 and it's waiting for data."
Let the image do the work. This audience will immediately want to know what they're looking at.
Slide 2 — The problem (0:45–2:00)
GPU starvation. Define it in dollar terms.
- Real LLM providers run at 30–50% actual GPU utilization on inference
- An 8-GPU H100 cluster costs ~$20/hr to rent
- At 40% utilization, $12/hr is wasted — the GPUs are idle waiting for data
- LLM providers run hundreds of clusters
- The waste is structural and nobody has fixed the root cause
Slides 3–4 — The mechanism (2:00–4:00)
Two diagrams, side by side.
Standard pipeline:
CPU tokenizes → Python tensor (pageable memory) → .to("cuda") → kernel buffer → GPU
^--- two OS-mediated copies
UncheckedIO pipeline:
Rust tokenizes in parallel → pinned memory (DMA-accessible) → direct DMA → DLPack → GPU
^--- one copy, no kernel involvement
The second diagram has fewer boxes and fewer arrows. That is the entire pitch.
Key terms to explain briefly:
- Pinned memory — memory registered with the OS so the GPU's DMA engine can read it directly
- DMA — the GPU pulls the data itself, no CPU involvement after the transfer starts
- DLPack — a standard zero-copy tensor exchange protocol; PyTorch accepts it natively with
torch.from_dlpack()
Slide 5 — The Nsight demo (4:00–7:00)
Show the recorded Nsight Systems timeline from the H100 benchmark run.
Both pipelines visible side by side:
- Point out the idle gaps (orange) in the standard pipeline
- Point out the absence of idle gaps in the UncheckedIO pipeline
- Show the DMA transfer lane — this is the "hidden thing" made visible
This is the centerpiece of the talk. Expect questions here. That's good.
Slide 6 — The number (7:00–8:00)
One slide. No prose.
| Hardware | Speedup |
|---|---|
| RTX 3060 (local) | 4.7x |
| H100 SXM (cloud) | [Wednesday result] |
Slide 7 — Market fit (8:00–9:00)
- This is infrastructure every LLM provider needs
- Built as an open-core library: PostgreSQL→Arrow is Apache 2.0 and already on PyPI
- GPU ingestion pipeline is the commercial layer
- AMD ROCm support (MI300X) is next — providers are actively evaluating MI300X to escape H100 pricing
Don't pitch deal specifics. This slide plants the seed.
Slide 8 — The close (9:00–10:00)
"I'm looking for connections to ML infrastructure teams and LLM providers. If you know someone building at that layer, I'd appreciate an introduction."
Direct. Not desperate. Leave room for questions.
Notes
What NOT to do:
- Don't go deep on the Rust implementation — the visual is the hook, not the code
- Don't mention the PostgreSQL→Arrow piece — it's a distraction in this context
- Don't volunteer that it's unfinished — if asked, "it's in active development and the core mechanism is proven"
For the well-connected engineer (separate talk, two weeks out): Don't reuse this version. Prep a deeper, tailored pitch after seeing what questions come up here.
For the NVIDIA contact: Wait until after the H100 benchmark. Send the Nsight trace + one paragraph. Leading with a measurement is different from leading with a claim.