Reddit user reports LFM2.5-24-A2B loops and fails a coding task
A Reddit user in r/LocalLLaMA says the open-weight model LFM2.5-24-A2B looped repeatedly and stumbled on simple tasks when run through a llama.cpp build optimized for certain architectures, and that it produced broken code, omitted required functionality and claimed verification without end-to-end tests on a Hermes memory-plugin task. The user says the same setup reached about 130 tokens per second on an RTX A4500, and that switching to LM Studio stabilized the erratic behavior into "correct-ish" chat responses. A community chat-template fix from Hugging Face reportedly did not change the results much. The evaluation of the coding output was mediated by ChatGPT, and no independent testing is provided.
The post gives a configuration-level, single-user data point on how the inference stack (a tuned llama.cpp build versus LM Studio) and chat templates can change one user's results with an open-weight model.
| So, yesterday I made an apparently too simplistic post about this shitty model that everyone is cheering (probably bots, I would like to see what they achieved with it). Link https://www.reddit.com/r/LocalLLaMA/comments/1xb9hrb/tried_lfm2524a2b_its_crap/ The main critic is that I didn't provided any configuration or anything. Well here it is. I tried to run it on llamaampere a llama.cpp version optimized for certain architectures which allowed me to run correctly Qwen 3.8 27b Q4 Small with very high (for me) 40-60 tk/s and satisfying results. But it wasn't the right choice as it keep looping itself over and over and tripping over the easiest things. But I got a 130tk/s on a RTX A4500 card, so it was kinda of an achievement lol Than I moved directly to LM studio, I said ok, let's sacrifice speed and see how it goes. It stabilized the erratic behavious. I tried a bit of chatting in it and it was responding correct-ish. CONFIGURATION, for you control freaks that otherwise would botch my post: LMSTUDIO-> All Engine&Frameworks updated to today 22/10/2026, last version all green checks Model: https://preview.redd.it/sw6dnz6agzwh1.png?width=608&format=png&auto=webp&s=43154be5df8ce228fe6ee9e9a3471f74e0348576 Then I read that this thing is supah-dupah trained on agentic harnesses like openclaw, hermes and so on and they made a pride point in it. So I installed hermes desktop and linked it to hermes. Someone will come to yell at template fix. Got you covered. https://huggingface.co/peculiar-ragdoll/Sharp-LFM2.5-Chat-Templates/raw/main/chat_template.jinja Using this one which is the fix for the template. Asked it in hermes desktop to make a simple plugin to use garnet from microsoft which is a redis clone. Result analyzed by chatgpt which was leading the prompt engineering and checking the result with GPT Sol: "LFM2.5-24B-A2B performed poorly on the Hermes memory-plugin task: it produced broken code, omitted required functionality and claimed verification without end-to-end tests." I tried with the fix (sharp template fix) and it didn't change much. Suggestions? submitted by /u/tracagnotto [link] [comments] |
#LFM2.5-24-A2B#llama.cpp#LM Studio#Hermes#Hugging Face#RTX A4500#ChatGPT#GPT Sol