Inference workers
Scalable training
50%+ MFU
98% GPU utilization
Configuration & assembly
Configuration
recipes · validation · overrides
Trainer setup
model · data · optimizer · hooks · resume
Data pipeline
Model & objectives
Training loop
Dataset readers
trajectories · metadata
Data distributor
rank assignment · staging
Batch assembly
process · collate
Data loader
stream mixing · prefetch
Input preparation
encoding
tensor assembly
Model execution
forward computation
Training objectives
losses · regularization
Training loop
forward → backward
gradient accumulation
cross-rank consensus
steps · samples
training state
Optimization
optimizer · scheduler
gradient clipping
Distributed execution
FSDP / HSDP · context parallel · precision · compilation
Training lifecycle
Validation & rollouts
evaluation
generated samples
Telemetry & profiling
metrics · traces · memory
Checkpoint manager
save / restore
full training state
Native PyTorch · modular recipes · stateful recovery
batch
invoke
loss / metrics
weights
update
hooks / state
Robot serving
within 100 ms end to end
Orchestrator
Session gateway
connections
request validation
Input preprocessing
decode · tokenize
CPU pool
Session scheduler
batching · sessions
resource ownership
Data plane
shared inputs / outputs
Control plane
plans / completions
Multi-GPU execution
Model execution
CUDA graph replay
captured execution
shape reuse
GPU state caches
persistent
device-resident state
Hybrid
parallelism
TP · CP · EP
observations
actions
metrics / health
outputs / session updates
inputs / outputs
Model delivery
Model export
checkpoint conversion
Model package
weights · configuration
Model loader
device placement · initialization
deploy
load weights
training kernels
inference kernels
Shared GPU kernel layer
Attention kernels
Forward / Backward
Fused tensor kernels
normalization · elementwise fusion
Matrix & encoder kernels
grouped GEMM · encoding operators