LFM3-1.7B quantization report finds 3.6 GB lossless VRAM footprint
AI summary
A report shared on r/LocalLLaMA says LFM3-1.7B can run in 3.6 GB of VRAM with lossless quantization and 2.2 GB with modest degradation. It also cites LiquidAI's own benchmarks as placing LFM3-1.7B above MiniCPM5-2B, while claiming twice the context and roughly half the VRAM footprint for the quasi-lossless quantization.
In-depth insight
The reported measurements indicate that LFM3-1.7B could remain useful for local inference within comparatively tight GPU-memory constraints.
| This model looks quite spectacular. According to LiquidAI's own benchmarks, it scores higher than MiniCPM5-2B across the board, while offering 2x the context and taking up half the space in VRAM (3.6 GiB vs. 7.1 GiB for a quasi-lossless quant). Read the report: https://github.com/crusaderky/pixi-llm-recipes/blob/main/perplexity/LFM3-1.7B/README.md Raw plots as interactive HTML: https://htmlpreview.github.io/?https://raw.githubusercontent.com/crusaderky/pixi-llm-recipes/main/perplexity/LFM3-1.7B/report.html submitted by /u/crusaderky [link] [comments] |