Unity Catalog

Headlines · 3

1. 1d
   Databricks details how its AI research team uses AI Runtime for agent-led training

   Databricks published a post describing how its AI research team uses its own AI Runtime (AIR) to train and evaluate the models behind products such as Genie, Smart Routing and AI Functions. The team cites AIR capabilities including GPU preflight checks and health monitoring, cordoning off unhealthy nodes, the AIR CLI with custom Docker containers, automatic MLflow experiment tracking, governed data access through Unity Catalog FUSE mounting, and workload prioritization and preemption to share GPUs across projects. Databricks says agent-led "autoresearch" workflows have driven an 8x increase in the number of training jobs for its AI Functions research, with models ranging from a few billion to nearly a trillion parameters trained on NVIDIA H100 or B300 GPUs, and the largest runs using hundreds of GPUs over days or weeks.

   Databricks Blog · 🔥 4
2. 3d
   Databricks: AI agent ROI gap tied to missing business context, not speed

   A Databricks blog interview with staff member Amber Roberts examines why enterprises are adopting AI agents quickly but struggling to prove ROI. She argues the buyers and the day-to-day users are often different people, and that buyers deploy under competitive pressure first, then spend six months to a few years learning which use cases actually work — the trial-and-error phase many organizations are in now. She says business context matters more than raw task speed, that the most common mistake is scaling before a single use case works well, and that traditional metrics like hours saved and tickets closed should be paired with AI-specific checks such as a third-party LLM judging correctness, hallucinations and tone. She also describes using two agents to contact and follow up with conference speakers, with all 137 speakers confirmed.

   Databricks Blog · 🔥 0
3. Oct 8
   Databricks Lakebase branches databases per coding agent

   A Databricks blog post explains how Lakebase Postgres database branching supports coding-agent workflows: using copy-on-write, a full database branch can be created from a parent in under a second regardless of size, and branches scale to zero so idle ones incur no compute cost. The walkthrough gives each agent and each pull request an isolated ephemeral branch, combining Git worktrees, a post-checkout hook, GitHub Actions and Drizzle migrations, applying migrations to both a preview app and the main branch and posting a schema-diff comment on the PR. The author notes that Lakebase branches are not merged back to the main branch like Git; schema changes are promoted through migrations in code, and the speed and cost claims come from Databricks' own blog.

   Databricks Blog · 🔥 0

Experience and discussion from the community

Share my Unity Catalog experienceAsk about Unity Catalog