OpenJet Adds Concurrent Coding-Agent Inference on One GPU
AI summary
OpenJet released an update that serves multiple coding agents concurrently on a single GPU using a custom CUDA megakernel for Qwen3.8-27B Q4_K_M. Its developer reported 64–72 tokens per second per agent with four agents on an RTX 3090 for short prompts, and 52–55 tokens per second with 23K contexts, though no independent measurements were provided.
In-depth insight
The release offers a concrete implementation for sharing model-weight reads and a paged KV pool across concurrent local coding agents, but its reported throughput remains developer-claimed.