Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Learned predictors improve LLM fine-tuning duration estimates

AI summary

Dharma-AI R&D analyzed 272 production supervised fine-tuning jobs totaling about 7,382 GPU-hours on NVIDIA H200 clusters. In five-fold cross-validation, its CatBoost predictor achieved R² of 0.98 and a mean absolute error of 1.71 hours, compared with R² of 0.31 and 12.41 hours for the tested FLOP-based baseline. The study covers only SFT workloads on H200 infrastructure, and the full paper is planned for arXiv.

In-depth insight

Dharma-AI R&D says more accurate pre-dispatch duration estimates could help shared-cluster operators plan GPU placement and scheduling, although the reported results are limited to one organization's H200 SFT workloads.

Predicting end-to-end fine-tuning time on shared GPU clusters, from 272 production runs.

A preview of our upcoming paper, Predicting End-to-End Fine-Tuning Time for Generative Language Models. The full version will be on arXiv soon.

GPUs are among the most expensive and scarcest resources in AI today. Anyone running a shared cluster has to decide, every day, which jobs go on which GPUs and in what order. That plan is only as good as the estimate of how long each job will take. When a job finishes hours early, GPUs sit idle until the next one is dispatched. When it runs hours late, everything queued behind it slips, experiments wait and delivery dates move.

Better duration estimates turn this into planning instead of guesswork: what runs, when, and on how many GPUs, with less idle capacity in between. Yet the estimate most teams use is a back-of-the-envelope FLOP calculation. It says nine hours, and the job finishes the next afternoon.

We wanted to know how far off that math really is, and whether we could do better before a job is even dispatched. So we took 272 real supervised fine-tuning (SFT) jobs from our production pool of NVIDIA H200 GPUs, about 7,382 GPU-hours in total, and compared analytical estimates against learned predictors.

#Dharma-AI R&D#NVIDIA H200#CatBoost#Hugging Face Transformers#Unsloth#Calculon#XGBoost

Original · 8 min readRead the original
Group chat

No comments yet. Start the conversation.