Databricks details how its AI research team uses AI Runtime for agent-led training
Databricks published a post describing how its AI research team uses its own AI Runtime (AIR) to train and evaluate the models behind products such as Genie, Smart Routing and AI Functions. The team cites AIR capabilities including GPU preflight checks and health monitoring, cordoning off unhealthy nodes, the AIR CLI with custom Docker containers, automatic MLflow experiment tracking, governed data access through Unity Catalog FUSE mounting, and workload prioritization and preemption to share GPUs across projects. Databricks says agent-led "autoresearch" workflows have driven an 8x increase in the number of training jobs for its AI Functions research, with models ranging from a few billion to nearly a trillion parameters trained on NVIDIA H100 or B300 GPUs, and the largest runs using hundreds of GPUs over days or weeks.
It shows how agent-driven experimentation is raising the bar for fast job launches, programmatic interfaces and reliable GPU infrastructure in frontier model research.