68 lines
3.6 KiB
Markdown
68 lines
3.6 KiB
Markdown
|
# Handoff — 2026-04-20
|
|||
|
|
|
|||
|
|
## What Was Decided This Session
|
|||
|
|
|
|||
|
|
- **Commercialization path:** Licensing first, IP sale after traction. Wyoming LLC before any contract.
|
|||
|
|
- **Language strategy:** Rust is source of truth. C++ generated via Whetstone + manual library substitution as enterprise deliverable. Both versions is a deliberate differentiator.
|
|||
|
|
- **Primary buyers:** GPU cloud providers and LLM inference companies (CoreWeave, Lambda Labs, Together AI, Fireworks AI, Modal). Not PyTorch/Meta.
|
|||
|
|
- **Hugging Face** is a secondary buyer worth a conversation — they own the `tokenizers` crate this code already uses.
|
|||
|
|
- **NVIDIA contact:** Approach after H100 benchmark. Send Nsight trace + one paragraph. Not before.
|
|||
|
|
- **NM funding:** Apply for NMSBA (Sandia/LANL hardware access). Skip money grants for now.
|
|||
|
|
- **Outsourcing:** Friend's software business for non-core work. 60-70% of contract value for defined scope. IP assignment clause required in writing first.
|
|||
|
|
|
|||
|
|
## Documents Created This Session
|
|||
|
|
|
|||
|
|
| File | Purpose |
|
|||
|
|
|---|---|
|
|||
|
|
| `ROADMAP.md` | Full technical development plan, Phases 1–5 |
|
|||
|
|
| `docs/talk-10min.md` | 10-minute presentation structure for Wednesday |
|
|||
|
|
| `docs/commercialization-strategy.md` | Pricing, B2B sales process, structure, outsourcing |
|
|||
|
|
| `HANDOFF-2026-04-20.md` | This file |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## This Week — Time-Sensitive (in order)
|
|||
|
|
|
|||
|
|
### Before Wednesday
|
|||
|
|
- [ ] Fix Phase 1.1 — dynamic buffer allocation (`src/llm.rs`, `PinnedBatcher::new`)
|
|||
|
|
- [ ] Fix Phase 1.2 — buffer isolation / DLPack overwrite bug
|
|||
|
|
- [ ] Fix Phase 1.3 — thread-safe engine pool (`TokenizerEngine` takes `&mut self`)
|
|||
|
|
- [ ] Fix Phase 1.4 — padding for variable-length sequences
|
|||
|
|
|
|||
|
|
### Wednesday
|
|||
|
|
- [ ] Rent H100 on Lambda Labs (~$2 for 30 min)
|
|||
|
|
- [ ] Run `benchmark_gpu.py` under `nsys profile --stats=true python benchmark_gpu.py`
|
|||
|
|
- [ ] Capture Nsight `.nsys-rep` file and download it
|
|||
|
|
- [ ] Note the speedup number — this is the headline for the talk
|
|||
|
|
|
|||
|
|
### Wednesday/Thursday
|
|||
|
|
- [ ] Update `docs/talk-10min.md` Slide 6 with the H100 result
|
|||
|
|
- [ ] Open Nsight Systems GUI locally, load the `.nsys-rep` file, screenshot the timeline showing both pipelines
|
|||
|
|
|
|||
|
|
### Thursday (if someone gives you a contact after the talk)
|
|||
|
|
- [ ] Do not show GPU source code without NDA signed first
|
|||
|
|
- [ ] First call is discovery — ask about their infrastructure before pitching
|
|||
|
|
- [ ] Pricing range if pushed: $50K–$150K annually depending on scale
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Next Session Priorities (after this week)
|
|||
|
|
|
|||
|
|
1. **H100 setup script** — a single paste-into-terminal script for the cloud instance (was going to write this, ran out of session). Needs: Rust install, maturin build with `--features gpu`, nsys profile run, output capture.
|
|||
|
|
2. **Phase 1 fixes** — if not completed before Wednesday, finish these
|
|||
|
|
3. **vLLM integration** — Phase 2.1, this is the demo that sells the concept
|
|||
|
|
4. **NM anonymous LLC** — straightforward to form locally, no member names in public filings, no annual report requirement
|
|||
|
|
5. **C++ version scoping** — after Phase 1 is solid, assess Whetstone transpilation of `llm.rs`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Key Technical Context for Next Session
|
|||
|
|
|
|||
|
|
- Repo is cloned at `/home/bill/Documents/unchecked-io/`, currently on `gpu-zero` branch
|
|||
|
|
- GPU feature is compiled with `maturin develop --release --features gpu`
|
|||
|
|
- The four Phase 1 bugs are documented in detail in `ROADMAP.md` Phase 1 section
|
|||
|
|
- `src/llm.rs` is the file to fix — `PinnedBatcher` struct, `batch_encode_to_gpu` method
|
|||
|
|
- `src/lib.rs` `TokenizerEngine` wraps `PinnedBatcher` and is the Python-facing class
|
|||
|
|
- chrono-tz is pinned at `=0.10.4` — do not change this, it was hard-won
|
|||
|
|
- `gpu-zero` branch is private; `main` branch is open source Apache 2.0
|