Databricks Model Serving

Headlines · 2

1. Oct 21
   Databricks AI Runtime launches with expanded NVIDIA collaboration

   Databricks announced at GTC Berlin that AI Runtime is generally available, expanding its collaboration with NVIDIA for customizing open models with enterprise data. Developers can use NVIDIA NeMo, NGC containers, and Nemotron cookbooks to train and fine-tune models on serverless NVIDIA GPUs, then evaluate them and deploy them through Databricks Model Serving. Databricks also says AI Runtime supports distributed training, governed data access through Unity Catalog, experiment tracking with MLflow, and custom containers.

   Databricks Blog · 🔥 0
2. Oct 9
   TinyFish runs its web-native agent model on Databricks AI Runtime

   TinyFish is developing a specialized web-native model for enterprise agents that work across the live web, and has consolidated its post-training, evaluation and production serving on Databricks: managed GPUs via AI Runtime, evaluation with MLflow, and deployment through Model Serving. The team started with supervised fine-tuning and is moving toward Direct Preference Optimization. On its internal go/no-go benchmark, the model completed 1,691 of 2,178 web-agent tasks for a 77.6% task-success rate, with a separate LLM judge reviewing the request, action history and final result; partial completion or incorrect answers count as failures. Over a recent 15-day period, TinyFish recorded 294,781 agent runs involving the model across 6.4 million tool steps, at an estimated serving cost of $0.088 per run and $0.0040 per tool step — covering only that model, not the full agent stack. TinyFish already serves its search reranker and query rewriter on Databricks and uses Unity Gateway to centralize access to external foundation models.

   Databricks Blog · 🔥 0

Experience and discussion from the community

Share my Databricks Model Serving experienceAsk about Databricks Model Serving