Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

ROCm

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 8

  1. Developer proposes mixed CUDA and ROCm inference for Project MAYA

    A Reddit developer said they developed a Project MAYA fork and pull request to run GLM-5.3-Flash Maya-M GGUF prefill and decode cooperatively across an NVIDIA RTX 5090 and an AMD Strix Halo iGPU. The CUDA backend handles attention, dense or shared layers, and the expert cache on the RTX 5090, while the ROCm backend executes complementary experts on the Strix Halo during prefill and decode. The developer reported the approach could be up to 48% faster and published experiment results. Project MAYA’s creator welcomed the design and planned to test it before deciding whether to merge it.

    r/LocalLLaMA · 🔥 3
  2. Deploying Qwen3.5 4B on a Legacy RX 580 Server

    A report from 掘金 · 人工智能 documents running Qwen3.5 4B with Ollama 0.40.14 on Windows 10 using an Intel Xeon E5-2670 v2, 32 GB of memory, and an 8 GB AMD RX 580. The author exposes the model to WorkBuddy and Dify through an OpenAI-compatible API, with the Xeon serving as fallback compute at a reported prefill speed of about 54–74 tokens per second. Because the Vulkan flash-attention path produced zero-filled output on longer prompts, the deployment disables flash attention and accepts slower generation for more predictable results.

    掘金 · 人工智能 · 🔥 0
  3. vLLM Migrates CUDA Builds to the LibTorch Stable ABI

    vLLM says its CUDA build now uses PyTorch’s LibTorch Stable ABI, allowing compatible LibTorch updates to be imported without rebuilding or updating vLLM. Starting with v0.33.0, the project publishes LibTorch ABI Stable CUDA wheels and uses torch-abi-audit in CI to check for unstable symbols. The Stable ABI does not guarantee compatibility across all PyTorch releases because Python APIs, GPU and CUDA requirements, and third-party dependencies remain relevant.

    vLLM Blog · 🔥 0
  4. int8-vllm fork adds support for Nemotron 3.5 Bolt 74B

    A community maintainer said the int8-vllm fork of vLLM now supports original Nvidia Nemotron 3.5 Bolt 74B checkpoints. The update adds fused NVFP4 and IQ2_xxs dequantization, a configurable INT8 weight cache, and AMD Infinity Storage model loading. The maintainer said the latter cut Bolt's initial load time from about seven minutes to two minutes in his test; the supplied report does not independently verify the claim.

    r/LocalLLaMA · 🔥 0
  5. Local Qwen3.8-Flash-Next and Claude Review One Commit

    A Reddit user ran Qwen3.8-Flash-Next locally through Claude Code to review one commit in the ChatOKF repository, with Claude Opus serving as the comparison reviewer. The poster reported that Qwen produced eight valid findings with no false positives, including two issues Claude missed, while missing five issues found by Claude and getting two details wrong. The setup used UD-Q4_K_XL quantization, llama-server with ROCm, an AMD Strix Halo 128 GB system, a 262K context window, and an MTP draft head. Reported decode speed fell from about 21 tokens per second at 4K context to about 14 at 128K.

    r/LocalLLaMA · 🔥 0
  6. Reddit user reports LFM3-1.7B local inference results

    A Reddit user testing Liquid's LFM3-1.7B in Q8_0 GGUF with llama.cpp and ROCm reported 4.1k tokens per second prefill and about 78 tokens per second decode on a Strix Halo iGPU. The user also reported roughly 7.1k prefill and 151 decode tokens per second when tensor-splitting the model across a Radeon RX 7900 XTX and an unspecified “XT” device. They described LFM3-1.7B as much more coherent at 32k–64k-token prompt sizes than a 1.7B model.

    r/LocalLLaMA · 🔥 0
  7. vLLM v0.33.0rc2 fixes AITER GEMM issue on ROCm gfx950

    vLLM v0.33.0rc2 includes a ROCm bugfix that skips AITER tuned GEMM when M*N is at least 2^31 on gfx950. The release note says the change was cherry-picked from commit 6d01893.

    vLLM Releases · 🔥 0
  8. PyTorch adds FlyDSL backend support for AMD MI350 GPUs

    PyTorch Blog reports that AMD Instinct MI350-series GPU users can access FlyDSL kernels through existing PyTorch operator APIs. The backend covers dense and grouped GEMM, MXFP8 and MXFP4 scaled GEMM, attention, RMSNorm and TopK. GEMMs use the optional backend through torch.compile, eligible RMSNorm and TopK calls can dispatch automatically in eager mode, and unsupported inputs retain existing PyTorch implementations. FlyDSL attention kernels are included in AOTriton 0.14.2b or later. Official tests on MI355X reported a 1.10x geometric-mean speedup for the BF16 dense-GEMM suite, while end-to-end vLLM A/B tests reached whole-request speedups of 1.13x for BF16, 2.04x for MXFP8 and 3.01x for MXFP4. These results depend on the workload, model and concurrency, and were not independently verified in the supplied reports.

    PyTorch Blog · 🔥 0

Experience and discussion from the community

Share my ROCm experienceAsk about ROCm

Nobody has shared their experience with ROCm yet.