Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Garlic LLM runs Qwen3.5-9B offline on a jailbroken PS4

AI summary

Leonardo Alegría released Garlic LLM, an open-source project that runs the 9-billion-parameter Qwen3.5-9B model offline on the GPU of an exploited PS4. It repacks weights into custom 4-bit and 3-bit formats to about 4.5 GiB and uses Qwen3.5's MTP head for speculative decoding, with Alegría reporting 31–39 tokens per second for text chat and up to 43 tokens per second for code generation. Tom’s Hardware also reported that the developer tested a base or Slim PS4 on firmware 11.52, while noting that the prompt and test method were not specified and the PS4 Pro remained untested.

In-depth insight

Garlic LLM demonstrates a concrete open-source route for running a 9B-parameter model entirely on the GPU of an older game console, using custom quantization and speculative decoding without cloud services.

I made this, so a quick note: it’s my open-source project. Qwen3.5-9B runs on a stock PS4 at 31–39 tok/s. The MTP head that ships with Qwen3.5 made the biggest difference: it drafts up to two tokens and the full model checks them in one pass, which took it from 22 to 31–39 tok/s on this old GPU. Weights are re-packed into custom 4- and 3-bit formats to fit about 4.5 GiB. Your old PS4 can now run a 9B AI, offline, on its own GPU. It talks, listens, and writes code. No cloud, no subscription, no Linux. Free and open source. Code: https://github.com/leonardo-alegria/garlic-llm Direct download link: https://huggingface.co/garlic-llm/garlic-llm-ps4 submitted by /u/leonardo_alegria [link] [comments]

#Garlic LLM#Qwen3.5-9B#PlayStation 4#GoldHEN#Chatterbox#Parakeet TDT 0.6B v3#OpenOrbis PS4

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.