Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

unsloth

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 50

  1. Developer Runs DeepSeek-V4.1-Flash-1017 Q2 at 41 Tokens per Second

    A community developer reported running the DeepSeek-V4.1-Flash-1017 Q2 GGUF natively on Windows with one RTX 3090 and 192 GB of RAM, without WSL. On that system, the developer measured an average of 41 tokens per second and next-token agreement with DeepSeek's API of 90.2% on general text and 92.4% on code and mathematics. The developer also claimed the quantization fits the full 1M-token context and said the interface engine supports several published quantization sets.

    r/LocalLLaMA · 🔥 0
  2. Loading a Q8 ngram with Qwen 3.8 Flash Next in Strata

    A Reddit author describes side-loading a Q8 ngram into Strata with the --ple-gguf option or, in Docker, a STRATA_SET entry. The author reports that an unmodified Unsloth Q8 ngram paired with the IQ4_XS quantization produces an architecture-mismatch error. Adding the general.architecture metadata value “strata-ple” to the GGUF file allows Strata to load it. The author did not verify whether the Q8 ngram improves output quality, and says their memory-use and prefill/decode-speed checks were not thorough. They also report, based on what they had heard, that a BF16 ngram is not noticeably different while requiring about 50 GB more storage.

    r/LocalLLaMA · 🔥 0
  3. Unsloth offers a direct setup path for NVIDIA RTX Spark

    A report from UnslothAI says users can get started with Unsloth on NVIDIA RTX Spark by running a provided .exe file. It also says Daniel Hanchen and NaderLikeLadder got hands-on with the system to discuss running and training AI models locally, exploring reinforcement learning and making it easier to start building with Unsloth.

    UnslothAI (X) · 🔥 0
  4. Swift 1.5 benchmark shows coding-agent token savings on Strix Halo

    A r/LocalLLaMA user compared UkisAI’s Swift 1.5 fine-tune of Qwen3.8-Flash-Next with Unsloth’s quantization across Halogen, gufo and llama.cpp on an Asus ROG Flow Z13 with 128 GB of memory and a 70 W power limit. Swift produced 56,000–63,000 output tokens across 11 Pi coding-agent tasks, versus 75,000–99,000 for Qwen3.8-Flash-Next, and the gufo Swift setup completed them in 24 minutes compared with 33 minutes for gufo with Unsloth’s UD-Q4_K_XL. The results were mixed on math and LiveCodeBench, and the author cautioned that they came from a single run, so differences of one or two solved problems could reflect chance.

    r/LocalLLaMA · 🔥 0
  5. TensorSharp adds MiniMax-M3.1-Flash support and reports 8× A40 results

    A TensorSharp developer reports adding MiniMax-M3.1-Flash support on the first day of its open-weight release, including GGUF inference, dedicated CUDA kernels for MiniMax Sparse Attention, text and image inputs through an OpenAI-compatible API, tool calling, multi-GPU execution, and DSpark speculative decoding. In early tests on eight NVIDIA A40 GPUs, the developer reports 64K executor-prefill throughput of 2,567 tok/s versus 262 tok/s on a tested llama.cpp branch, but says the results came from one host, one quantization, and an uncommitted build without public raw records.

    r/LocalLLaMA · 🔥 0
  6. Muse Spark Open and Qwen3.8-Flash-Next benchmark comparison

    A Reddit analysis of Artificial Analysis results gives Muse Spark Open a score of 39 and Qwen3.8-Flash-Next a score of 40, comparing open-weight models by total parameter count. The post also reports Unsloth UD-Q4_K_XL GGUF sizes of about 506 GB for Muse Spark Open and 111 GB for Qwen3.8-Flash-Next. It relays Meta’s recommendation to run Muse Spark Open locally on two M5 Ultra Mac Studios with 512 GB of memory each.

    r/LocalLLaMA · 🔥 0
  7. Reddit user runs 748B DeepSeek-V4.1-Flash on a used €2,000 PC at 7.2 tok/s

    A Reddit user posting as /u/Vladowski on r/LocalLLaMA reported running DeepSeek-V4.1-Flash-1017, a 748B-parameter model, on a used PC bought for under €2,000 with an RTX 3090 24 GB, 128 GB DDR4 and an Intel i9-9900X. The run used an Unsloth UD-IQ1_S GGUF taking 233 GB on disk and a llama.cpp build from open PR #30817 with CUDA 12.8, with the model's 110 GB Engram table kept on NVMe and read row by row on demand, which the user says is how it fit. The user reported generation at 7.2 tok/s and said the answer contained a physics error, adding that they saw little practical use for the setup while Qwen models work well on the same hardware.

    r/LocalLLaMA · 🔥 0
  8. Developer runs DeepSeek V4.1 Flash IQ1_S on RTX 5090 at 53.6 tok/s

    A developer posting on r/LocalLLaMA reports running DeepSeek V4.1 Flash 1017 at Unsloth's UD-IQ1_S quantization in Basalt, an inference engine they are developing, on an RTX 5090 plus an RTX 5060 Ti 16 GB and 128 GB of DDR5. With all 107 GiB of experts held in RAM and the 5090 used as an expert cache, the setup reached 53.6 tok/s decode and about 610 tok/s prefill, versus roughly 20 tok/s for a llama.cpp PR #30817 baseline on the same hardware and quantization. Letting the 5060 Ti compute some experts raised decode to 63.5 tok/s, but the author reports no perplexity or KL-divergence quality measurements yet, only greedy-match agreement with llama.cpp on test prompts.

    r/LocalLLaMA · 🔥 0
  9. Unsloth posts GGUF quantizations of Ornith-1.5-122B for local use

    Ornith announced on X, in a post reposted by UnslothAI, that Unsloth GGUF files for Ornith-1.5-122B are now available, hosted on Hugging Face in the unsloth/Ornith-1.5-122B-GGUF repository. Ornith credited UnslothAI for the conversion work, and the availability was also flagged in an r/LocalLLaMA post. The same promotional thread describes Ornith-1.5-122B as offering "no tradeoff between strong and fast" and "frontier-level capabilities with exceptional inference efficiency," though the supplied reports include no independent benchmarks for those claims.

    UnslothAI (X) · 🔥 0
  10. Ornith releases 1.5-122B with an expanded self-improvement loop

    Ornith introduced Ornith-1.5-122B, extending its earlier model by jointly optimizing task generation, scaffold construction, and solution rollouts. The developer says the model continuously creates training tasks, discovers solution strategies, and improves its policy through reinforcement learning, with official and Unsloth GGUF files linked in the announcement.

    r/LocalLLaMA · 🔥 0
  11. Unsloth releases 1-bit GGUF builds for DeepSeek V4.1-Flash-1017

    Unsloth released GGUF builds of DeepSeek V4.1-Flash-1017 for local use, including a 1-bit configuration that it says retains about 88.2% accuracy while reducing model size from 508GB to 250GB. The company also claims strong benchmark rankings, but the reports provide no independent measurements, and the stated RAM requirement varies between 64GB and 128GB across its posts.

    UnslothAI (X) · 🔥 0
  12. Notebooks released for fine-tuning d1-3B decision model

    UnslothAI reposted a thread by user leonie describing two notebooks: one walks through preparing training data, the other through fine-tuning the d1-3B decision model. The post says the workflow uses UnslothAI and Hugging Face TRL, and both notebooks live in the Liquid4All cookbook repository. The post also claims the whole process takes under 30 minutes on a free-tier GPU, a vendor claim with no independent measurement provided.

    UnslothAI (X) · 🔥 0
  13. Nvidia's RTX Spark laptops launch with up to 128GB memory for local AI agents

    Nvidia said on Oct. 16 that RTX Spark Windows laptops are available with up to 128GB of unified memory and up to 1 petaflop of FP4 AI performance, sold by Asus, Dell, HP, Lenovo, Microsoft and MSI starting at $2,599. Each laptop runs on a single superchip pairing a Blackwell RTX GPU with a Grace CPU, with native CUDA on Arm; Nvidia says users can install agents such as GitHub Copilot, Hermes, OpenClaw and Perplexity from first start-up without a command line. Microsoft, which says it is optimizing Windows for local AI models and a new Search interface, describes Windows as a platform for hybrid intelligence, and Lenovo has begun selling its YOGA Pro 15 RTX Spark in China from 29,999 yuan.

    TechRepublic · 🔥 0
  14. Soup CLI v0.76.0 refuses config settings that never took effect

    Soup, an open-source LLM fine-tuning CLI, shipped v0.76.0, which now refuses to load settings that were previously accepted but never actually applied — including training.quantization_aware: true, training.use_longlora and backend: unsloth on several tasks — so configs that loaded before may now fail, and the project advises running soup doctor --config soup.yaml first. The tool pitches one-command, single-YAML fine-tuning and an opt-in, still-BETA layer-streaming mode (stream_layers: true) that it says can LoRA fine-tune an 8B model on a 4 GB laptop GPU. The README also records a silent wrong-gradient defect on NF4 above roughly 165 MiB per layer that has been repaired, and notes that throughput figures from before and after the fix are not directly comparable.

    GitHub Trending · Python · 🔥 0
  15. Hugging Face Transformers tops GitHub Trending

    Hugging Face's Transformers library appeared on GitHub Trending's daily list. The project positions itself as the model-definition framework for state-of-the-art text, vision, audio, video and multimodal models, for both inference and training, and its documentation says there are over 1M Transformers model checkpoints on the Hugging Face Hub. It says the library interoperates with training frameworks such as Axolotl, Unsloth, DeepSpeed, FSDP and PyTorch-Lightning, and with inference engines and modeling libraries including vLLM, SGLang, TGI, llama.cpp and mlx, and requires Python 3.10+ and PyTorch 2.6+.

    GitHub Trending(每日) · 🔥 0
  16. Unsloth ships desktop app and Studio web UI

    The open-source Unsloth project is trending on GitHub's Python list, and its README now describes three ways to use it: the native Unsloth Desktop app for Windows, macOS and Linux, the Unsloth Studio web UI, and code-based Unsloth Core. The project says it can run and train LLMs plus diffusion, embedding and audio models, naming Qwen3.8, DeepSeek-V4, Gemma 4, GLM-5.3-Flash, Kimi K3 and MiniMax-H3, and its Unsloth Start command wires local models into agents such as Claude Code and Codex. The README also claims 2x faster training with up to 70% less VRAM and no accuracy loss — figures that come from the project's own documentation.

    GitHub Trending · Python · 🔥 0
  17. Unsloth trains a decision model on Qwen3.5-2B in about 2 minutes on one DGX Spark

    Unsloth shared an Ivan Fioravanti demo showing a "decision model" trained in roughly two minutes on a single NVIDIA DGX Spark using Unsloth, with Qwen3.5-2B as the base model. The post points to Unsloth's documentation page on training your own decision model with Unsloth. The source gives no details on the model's intended use, evaluation results, or training data.

    UnslothAI (X) · 🔥 0
  18. Unsloth releases free notebook to train decision models locally

    Unsloth published a free Colab notebook and companion guide showing how to train Qwen3.5-4B locally into a “decision model” that outputs decisions instead of text, using only 8GB of VRAM. Unsloth says its open-source repo can turn any model, such as Qwen3.8 or Gemma 4, into a decision model, and claims it raised Qwen3.5 0.8B’s aggregate accuracy from 20.7% to 74.3% across three decision benchmarks on just 4GB of VRAM (these figures are Unsloth’s own and the benchmark methodology was not disclosed). The walkthrough covers data preparation (state, questions, gold answers), training and serving, and likens the result to Jev.

    UnslothAI (X) · 🔥 0
  19. Microsoft open-sources mxc sandbox; Unsloth integrates it with <100ms overhead

    Microsoft has released an open-source repository called mxc for sandboxed code execution. Unsloth says it collaborated with Windows to add mxc OS-level sandboxing to Unsloth, adding under 100 ms of overhead, and points to its GitHub repo and a documentation guide. These claims come from Unsloth's post on X; no independent statement or further technical detail from Microsoft was seen in the source.

    UnslothAI (X) · 🔥 0
  20. Google releases EmbeddingGemma 2 for on-device multimodal retrieval

    Google released EmbeddingGemma 2, a 740M-parameter model licensed under Apache 2.0 that maps text, code, images, video, and audio into a shared 768-dimensional embedding space. Based on Gemma 4, it has a modular design with a 270M-parameter text component and optional 170M-parameter vision and 300M-parameter audio encoders. It supports an 8K-token context window and Matryoshka Representation Learning, allowing embeddings to be truncated to reduce storage. The reports say models are available through Hugging Face and LiteRT; arXiv metadata lists the accompanying paper separately.

    Google Developers Blog · 🔥 0

← Previous

Experience and discussion from the community

Share my unsloth experienceAsk about unsloth

unsloth: A desktop app for running and training models locally.

What problem it solves Running and training models locally requires tools for model execution, fine-tuning, agent integration, and deployment across different model formats and hardware. The project brings these workflows into a desktop app, a web UI, and a code-based version. Key features Run and train models — Run and train language, diffusion, decision, embedding, audio, image, and video models…

github.comhttps://github.com/unslothai/unsloth