OpenWebUI
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Berth v0.3 parks local AI engines in system RAM to speed GPU switching
Berth, a system service its developer describes in a r/LocalLLaMA post, watches the GPU and swaps and loads whichever engine the user wants, so that VRAM is dedicated to each running function without manual swaps or leftover processes holding memory. The developer has released v0.3, which parks engines in system RAM instead of unloading them, so switching back to a parked engine skips the reload from disk. They are looking for testers with 64, 96 or 128 GB of RAM and a 24 GB card, multi-GPU machines, Windows with an NVIDIA card, and AMD on Linux, noting that their own 62 GB workstation cannot measure parking at larger sizes because Strata alone pins 40 GB.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my OpenWebUI experienceAsk about OpenWebUI
Nobody has shared their experience with OpenWebUI yet.