Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

A Local AI User’s Journey Through Text, Image, and Audio Tools

AI summary

In a r/LocalLLaMA post, /u/revennest recounts moving from Ollama, LM Studio, and Msty to Oobabooga Text Generation-WebUI, later trying KoboldCPP, Llamafile, llama.cpp, Open WebUI, and SearXNG. The author says Oobabooga’s usability and deep customization made them stay with local AI, and reports running Llama 3.1 8B at roughly 1 token per second on an i5-6400T mini PC. The post also covers later experimentation with Forge, ComfyUI, Unsloth Studio, and audio.cpp, but these remain one user’s personal experience rather than a comparative test.

In-depth insight

The firsthand account shows how ease of use, prompt-processing features, and customization influenced one user’s progression through local AI tooling, including text, image, and audio workflows.

Here is my timeline

Text Generation

  • Ollama but quite have bad experience with it.
  • LM studio which still feel half-bake at that time.
  • Msty which easy to use like plug and play but not much I could do custom (or I just don't know at that time).
  • Oobabooga Text Generation-Webui this is the one that make me stick to AI, easy to use and deep custom; I like its function feature that take prompt as method/processed then input value in other prompt and then it just output an outcome, no chat history.

I run 1 - 4 with my mini pc with i5 6400T, 24GB DDR4 with onboard gpu... Llama 3.1 8B generate so slow at that time like 1 tok/s but I really enjoy using it with that very limited and the roleplay start from here, just my mini pc and Oobabooga Text Generation-Webui.

Also some missing thing I forgot at that time, I even mess around with windows WSL then docker for Open WebUI and SearXNG.

I finally can help but want it more so arise old gen pc, at that time DDR3 and thing relate to it is very cheap so I brought Xeon E3-1270 V2 with a Chinese mainboard and 32 GB DDR3 while I cancel the sell order of my GTX 1080 Ti.

#Ollama#LM Studio#Msty#Oobabooga Text Generation-WebUI#KoboldCPP#Llamafile#llama.cpp#Open WebUI#SearXNG#Llama 3.1 8B#Forge#ComfyUI

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.