Nebius

Headlines · 2

1. 2h
   Nebius, NVIDIA and Uber on squeezing more value from every unit of compute

   At a panel hosted by The Information's AI Agenda Live, executives from Nebius, NVIDIA and Uber discussed how they are optimizing compute as agentic workflows threaten to multiply token consumption. Nebius CRO Marc Boroditsky said the company tailors optimizations in its Token Factory inference platform to each customer's goals—speed, reliability, quality or price—using techniques such as speculative decoding, where smaller models predict likely next tokens that a larger model verifies in bulk, tuned with the customer's own traffic. NVIDIA's Dion Harris pointed to a Dynamo feature that locates already-computed data in a cluster's KV cache so teams avoid recalculating it, while Uber infrastructure VP Mattie Toia said its token costs have stabilized despite growing agentic use, helped by caching pushed down to individual sub-agents and by giving engineers visibility into their own usage; Uber also built an internal "context graph" of roughly 24 million nodes that cut query times from 20 minutes to 30 seconds.

   The Information · 🔥 9
2. 2d
   Nebius’ Eigen AI acquisition puts inference efficiency at center

   Nebius acquired inference-optimization startup Eigen AI for consideration exceeding $1 billion, bringing its roughly 20-person team into the cloud provider. Eigen AI founder Hanrui Wang now leads Nebius’s Token Factory, which covers inference, post-training and agent systems, while the company positions efficient open-model serving as a core business.

   MIT科技评论中文 · 🔥 0

Experience and discussion from the community

Share my Nebius experienceAsk about Nebius