bill/unchecked-io
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add roadmap, commercialization strategy, talk outline, and handoff notes

Browse Source 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bill Holcombe
2026-04-21 11:13:17 -06:00
parent f8ad40d302
commit 6f8b6440aa
4 changed files with 543 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

206
docs/commercialization-strategy.md Normal file
View File

@@ -0,0 +1,206 @@
# UncheckedIO — Commercialization Strategy
**Session date:** 2026-04-20
**Status:** Active planning
---
## Product Summary
Two components with different commercial treatment:
| Component | Language | License | Status |
|---|---|---|---|
| PostgreSQL → Arrow pipeline | Rust | Apache 2.0 (open source) | On PyPI, working |
| GPU zero-copy ingestion pipeline | Rust + CUDA | Commercial | Proof-of-concept, needs Phase 1 fixes |
The GPU pipeline is the commercial product. The Postgres piece is the credibility signal and open-source loss leader.
---
## The Problem Being Solved
GPU starvation — LLM providers run 30-50% actual GPU utilization on inference. The bottleneck is the data ingestion pipeline: CPU tokenizes, creates a Python tensor in pageable memory, `.to("cuda")` performs two OS-mediated copies (pageable → kernel buffer → GPU).
UncheckedIO replaces this with: Rust parallel tokenization → pinned memory (DMA-accessible) → single direct DMA → DLPack zero-copy handover to PyTorch.
**Current benchmark:** 4.7x faster than standard HuggingFace pipeline on RTX 3060.
**Pending:** H100 SXM benchmark (Wednesday, ~$2 on Lambda Labs). This number changes the pitch.
---
## Business Structure
**Entity:** New Mexico single-member LLC
- NM does not require member names in Articles of Organization
- No annual report requirement — less administrative overhead than most states
- LLC owns all IP and signs all contracts
- Registered agent handles public address
- Form this before any contract is signed
- Note: if the IP is eventually sold, the buyer's anonymity is their own concern — the LLC structure only needs to protect the original author
**Operating model:** Anonymous principal — LLC is the public face. The identity of the technical author is not disclosed to buyers or the public. Any investor or partner arrangement must include explicit confidentiality provisions.
---
## Go-to-Market Strategy
### Phase 1: Licensing (current goal)
License the GPU pipeline commercially. Annual per-cluster licenses. Open-source Postgres piece stays Apache 2.0 as a credibility and discovery channel.
**Why licensing before IP sale:**
A library generating $50K+/month in licenses is a business acquisition, not a proof-of-concept. The IP sale price increases significantly and the negotiating position improves. Licensing revenue demonstrates market validation.
### Phase 2: IP Sale (after traction)
Once licensing revenue establishes market proof, approach GPU cloud providers and LLM infrastructure companies for outright IP purchase. Source code + short-term handoff contract. Clean exit.
---
## Pricing
The ROI math for a GPU provider: 100 H100s at $175K/year each = $17.5M in GPU costs. A 10% utilization improvement = $1.75M saved. A $50K license is a 35x ROI.
**Do not anchor at $50K. That is the floor for small deployments.**
| Deployment size | Annual license | One-time source sale |
|---|---|---|
| Small (50–200 GPUs) | $25K–$75K | $100K–$200K |
| Medium (200–1,000 GPUs) | $75K–$200K | $300K–$600K |
| Large (1,000+ GPUs) | $200K–$500K | $750K–$1.5M |
**First client:** Accept below market rate in exchange for a private reference and permission to say "deployed at [company]" to the next prospect. That reference is worth more than the price difference.
---
## Target Buyers
**Primary (write the check):**
GPU cloud providers and LLM inference companies feel GPU starvation directly in their margins. They sell GPU utilization as a product. Any measurable improvement is direct revenue.
- CoreWeave, Lambda Labs — GPU cloud, sell utilization by the hour
- Together AI, Fireworks AI, Modal — LLM inference providers
- Anyscale — Ray-based ML infrastructure
**Secondary (may buy, slower process):**
- Hugging Face — owns the `tokenizers` Rust library this code uses, runs TGI serving stack. More plausible buyer than PyTorch/Meta.
- Databricks (MosaicML) — LLM training infrastructure
**Do not pitch:**
- PyTorch/Meta — open source project, they'd implement it themselves
- Hyperscalers (AWS, GCP, Azure) — too slow, too much process
**NVIDIA contact:**
Wait until H100 benchmark is complete. Send the Nsight trace + one paragraph. They may be interested as an ecosystem/marketing play ("runs X% faster on H100"). Do not approach before having the measurement.
---
## B2B Sales Process
### First contact (Zoom call)
The first call is discovery, not pitch. Goal: understand their specific infrastructure before presenting a solution.
**Ask first:**
- What GPU hardware? (H100, A100, MI300X — how many?)
- What serving stack? (vLLM, TGI, custom?)
- Current GPU utilization rate, if known?
- What caught their attention?
**Then:** "Based on what you're describing, here's specifically where this helps you..."
**On pricing:** Don't volunteer it first. If pressed: *"For a deployment your size I'd expect $50K–$150K annually. I want to understand your infrastructure better before giving you a specific number."*
**End of first call:** Set up follow-up with their infrastructure lead. Do not sign anything on the first call.
### NDA
Get an NDA signed before showing the GPU source code on any call. The open-source Postgres piece is fine to show. The GPU pipeline (`src/llm.rs`, `gpu-zero` branch) is not.
### Solo developer objection
Enterprise buyers will ask: *"What happens if you're unavailable?"*
**Answers:**
1. Source code sale model answers it directly — they own the code
2. Technical partner (friend's software business) provides continuity
3. Source code escrow clause: code held by third party, released if unavailable for 90 days — this is standard and removes most of the objection
---
## Maintenance and Support
**Reality for 1–2 enterprise clients:**
| Phase | Time commitment |
|---|---|
| Active development (first 3–6 months) | 20–40 hrs/week |
| Steady state | 5–10 hrs/week per client |
| Major CUDA/PyTorch version update | 2–4 days, happens 2–3x/year |
| Production incident | Drop everything, 1–3 days |
**Scope the support in the contract:** Cover current PyTorch LTS + one prior release. Anything newer is a paid upgrade. This prevents unbounded compatibility obligations.
One person can support 2–3 enterprise clients at steady state.
---
## Outsourcing
A friend with a software business can handle work that is not the core differentiator:
- Support ticket triage and first-response
- Documentation and integration guides
- Testing infrastructure (benchmark runs against new PyTorch/CUDA versions)
- Non-GPU features (Postgres pipeline, Python API polish)
**Do not outsource:** The GPU pipeline code, benchmarking methodology, CUDA/ROCm work — these are the differentiator.
**Rate:** Pass 60–70% of contract value for defined scope. On a $100K contract where the friend does 30% of the work: $18K–$21K to them. You take the larger share because you carry the business development risk and hold the client relationship.
**Critical:** Any work they do on the core library must be IP-assigned to the LLC in writing before the first engagement.
---
## Multi-Language Strategy (Rust + C++)
**Why both:**
- Rust is the development language — memory safety makes GPU pipeline bugs easier to catch
- C++ is the enterprise deliverable — the entire ML infrastructure ecosystem (cuDNN, TensorRT, vLLM C++ backend) is C++
- Having both is a differentiator from vibe-coded projects; it signals deep systems understanding
**How:**
- WhetstoneAI (separate project) can transpile language primitives automatically
- Library dependencies require manual substitution (see below)
- Rust is source of truth; C++ is a generated + manually doctored artifact per release
**Library substitutions for C++ version:**
| Rust dependency | C++ equivalent | Complexity |
|---|---|---|
| `cudarc` | Direct CUDA C++ calls (actually simpler) | Low |
| `rayon` | OpenMP `#pragma omp parallel for` | Low |
| `tokenizers` | SentencePiece (LLaMA) + tiktoken (GPT) | Medium — different API |
| `pyo3` | pybind11 | Medium — parallel structure |
| `tokio` | Not needed for GPU-only C++ version | N/A |
`parser.rs` (Postgres pipeline) is not worth transpiling — stays Rust only.
**Estimated work for C++ `llm.rs`:** 2–3 days after Whetstone handles primitives.
---
## NM State Funding Assessment
**Worth pursuing:**
- **NM Small Business Assistance Program (NMSBA)** — pairs small businesses with Sandia/LANL. The value is free access to HPC hardware (H100/A100 clusters), not the money. Solves the enterprise benchmark problem. Apply for this.
**Not worth pursuing (for now):**
- NMEDD money grants, angel tax credits — require registered business, matching funds, 3–6 month timelines. Time cost doesn't pencil out against a direct licensing deal.
**Key point:** Most state grant programs want an ongoing business, not a one-time IP sale. If the goal shifts toward a licensing business, revisit.
---
## Immediate Priorities
1. Fix Phase 1 correctness issues (see ROADMAP.md) — nothing else matters until these are done
2. Wednesday: H100 benchmark on Lambda Labs (~$2), run under `nsys profile`
3. Thursday/Friday: Incorporate H100 numbers into talk slides
4. Talk: 10-minute presentation to IT/Cybersecurity group (see docs/talk-10min.md)
5. Form Wyoming LLC before any serious sales conversation
6. After first client: evaluate C++ version via Whetstone transpilation

110
docs/talk-10min.md Normal file
View File

@@ -0,0 +1,110 @@
# UncheckedIO — 10-Minute Technical Talk
**Audience:** IT/Cybersecurity group with programmers and ML engineers
**Goal:** Plant the seed, demonstrate the mechanism, invite connections to LLM infrastructure teams
---
## Slide 1 — Open with the hidden thing (0:00–0:45)
Lead with an Nsight Systems timeline screenshot. Don't explain it yet.
> "This is what happens inside your GPU when a language model processes your request.
> The orange is waiting. The GPU costs $30,000 and it's waiting for data."
Let the image do the work. This audience will immediately want to know what they're looking at.
---
## Slide 2 — The problem (0:45–2:00)
**GPU starvation.** Define it in dollar terms.
- Real LLM providers run at 30–50% actual GPU utilization on inference
- An 8-GPU H100 cluster costs ~$20/hr to rent
- At 40% utilization, $12/hr is wasted — the GPUs are idle waiting for data
- LLM providers run hundreds of clusters
- The waste is structural and nobody has fixed the root cause
---
## Slides 3–4 — The mechanism (2:00–4:00)
Two diagrams, side by side.
**Standard pipeline:**
```
CPU tokenizes → Python tensor (pageable memory) → .to("cuda") → kernel buffer → GPU
^--- two OS-mediated copies
```
**UncheckedIO pipeline:**
```
Rust tokenizes in parallel → pinned memory (DMA-accessible) → direct DMA → DLPack → GPU
^--- one copy, no kernel involvement
```
The second diagram has fewer boxes and fewer arrows. That is the entire pitch.
Key terms to explain briefly:
- **Pinned memory** — memory registered with the OS so the GPU's DMA engine can read it directly
- **DMA** — the GPU pulls the data itself, no CPU involvement after the transfer starts
- **DLPack** — a standard zero-copy tensor exchange protocol; PyTorch accepts it natively with `torch.from_dlpack()`
---
## Slide 5 — The Nsight demo (4:00–7:00)
Show the recorded Nsight Systems timeline from the H100 benchmark run.
Both pipelines visible side by side:
- Point out the idle gaps (orange) in the standard pipeline
- Point out the absence of idle gaps in the UncheckedIO pipeline
- Show the DMA transfer lane — this is the "hidden thing" made visible
This is the centerpiece of the talk. Expect questions here. That's good.
---
## Slide 6 — The number (7:00–8:00)
One slide. No prose.
| Hardware | Speedup |
|----------|---------|
| RTX 3060 (local) | 4.7x |
| H100 SXM (cloud) | [Wednesday result] |
---
## Slide 7 — Market fit (8:00–9:00)
- This is infrastructure every LLM provider needs
- Built as an open-core library: PostgreSQL→Arrow is Apache 2.0 and already on PyPI
- GPU ingestion pipeline is the commercial layer
- AMD ROCm support (MI300X) is next — providers are actively evaluating MI300X to escape H100 pricing
Don't pitch deal specifics. This slide plants the seed.
---
## Slide 8 — The close (9:00–10:00)
> "I'm looking for connections to ML infrastructure teams and LLM providers.
> If you know someone building at that layer, I'd appreciate an introduction."
Direct. Not desperate. Leave room for questions.
---
## Notes
**What NOT to do:**
- Don't go deep on the Rust implementation — the visual is the hook, not the code
- Don't mention the PostgreSQL→Arrow piece — it's a distraction in this context
- Don't volunteer that it's unfinished — if asked, "it's in active development and the core mechanism is proven"
**For the well-connected engineer (separate talk, two weeks out):**
Don't reuse this version. Prep a deeper, tailored pitch after seeing what questions come up here.
**For the NVIDIA contact:**
Wait until after the H100 benchmark. Send the Nsight trace + one paragraph. Leading with a measurement is different from leading with a claim.