AI Runtime

Headlines · 1

1. 3d
   Databricks cuts Qwen 3.8 27B fine-tuning time 6x on AI Runtime

   In a blog post, Databricks describes how systematic performance work on its AI Runtime cut fine-tuning time for Qwen 3.8 27B on a fixed dataset from 7 hours 44 minutes to 1 hour 17 minutes — about 6x — from a baseline throughput of 9,396 tok/s. The team notes that the initial run already showed 98% average GPU utilization, but that metric only measures whether the GPU is doing anything; the real bottleneck was hundreds of thousands of small ops that the CPU could not enqueue fast enough. Installing fused kernels such as flash-linear-attention and causal-conv1d brought training down to 2 hours 5 minutes (3.7x), and switching to 1D packed tensors with correct position_ids to cut padding waste delivered the final 6x gain, with example code published in an AI Runtime cookbook.

   Databricks Blog · 🔥 0

Experience and discussion from the community

Share my AI Runtime experienceAsk about AI Runtime