Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Berth v0.4.1 brings alpha GPU scheduling for local AI workloads

AI summary

Berth v0.4.1 is presented as an alpha, Go-based GPU resource manager for sharing local AI, image-generation, voice, and gaming workloads. The Reddit project post says it provides leases, priority queues, engine switching, monitoring, multi-GPU management, an OpenAI-compatible gateway, and integrations including Claude Code and Codex; these capabilities were not independently verified.

In-depth insight

Berth could give local AI users a unified way to manage VRAM and hand off GPU workloads, but its alpha status and single-source presentation limit confidence in the reported feature set.

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off. Instead of letting every application compete for VRAM, Berth manages access through leases, priorities, and a request queue. When something else needs the GPU, it waits for the current task to finish, releases the memory, and hands control to the next application. What does it do? GPU sharing: Lets AI models, image generation, voice engines, games, and other GPU applications share the same hardware without fighting over VRAM. Automatic scheduling: Manages requests by priority, queues competing applications, and prevents unnecessary switching between engines. Faster model switching: Can park supported engines in system RAM or use NVIDIA CUDA checkpointing to avoid repeatedly loading large models from disk. Game management: Automatically releases GPU resources for games and restores AI workloads when you’re finished. GPU monitoring: Tracks VRAM usage, temperature, power, utilization, running processes, and applications using memory without a Berth lease. Model management: Discovers local models across GGUF, Hugging Face, LM Studio, Ollama, and other model directories. Supports downloading, verifying, and organizing models. Unified gateway: Provides an OpenAI-compatible API for starting and switching between engines, alongside ComfyUI workflow support. Web dashboard: Provides a live view of GPU usage, active leases, waiting requests, engine activity, telemetry, and historical data. Multi-GPU support: Manages multiple cards independently, with separate queues, leases, and gateway endpoints. Agent integration: Includes installable skills for Claude Code, Codex, OpenCode, Hermes Agent, and other tools so they can manage GPU workloads through Berth. VM passthrough: Includes VFIO tooling for handing a GPU to a virtual machine and reclaiming it afterward. Berth runs as a single Go binary with no cloud account or additional monitoring service required. Linux NVIDIA is supported, with experimental AMD Linux and Windows support. The project is currently in alpha (v0.4.1). Note: i added a claude code session showing handoff request. You can do the same with a local model on the same hardware, trip is just longer depending on your gear. submitted by /u/Distinct-Pie2389 [link] [comments]

#Berth#NVIDIA CUDA#llama.cpp#ComfyUI#Claude Code#Codex

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.