Reddit 用户质疑LLM写出高质量散文的能力
r/LocalLLaMA 用户 u/kompania 称,经过多年本地 QLoRA 和全量微调,生成结果仍仅略高于平庸水平。作者称自己测试了 Gemma、Qwen、Mistral、LFM、IBM 以及社区开发者从零构建的模型,并认为 EQBench 无法可靠衡量文学质量。发帖者表示,通常仍需自己写作、仅让 LLM 校对,随后询问社区是否有本地模型能达到优秀小说的水平。
这篇帖子提供了一名实践者横跨多个模型系列和微调方法后,在提升模型文学散文质量方面遭遇困难的详细自述。
I have been following the evolution of LLMs since the days of GPT-3.5, and I have been actively using local LLMs since the release of Gemma 2.
I read a great deal, including this subreddit. I have performed QLoRA fine-tuning on my local machine many times, and I have also conducted full fine-tuning (FFT) training runs on RunPod.
Hundreds of models-fine-tuned by well-known practitioners in English, Chinese, and Russian-have passed through my drive.
After four years, I arrived at a fundamental question yesterday: are LLMs even capable of writing good prose?
When training models myself, I used both my own prose datasets and those created by others. In the training process, I experimented with various approaches-ranging from 100 epochs of FFT at a learning rate of 1e-7, to aggressive LoRA/QLoRA configurations. The results, however, barely rise above mediocrity.
I have used models from all the leading providers-Gemma, Qwen, Mistral, LFM, IBM-as well as niche attempts by developers building models from scratch.
#Gemma#Qwen#Mistral#LFM#IBM#RunPod#EQBench#GPT-3.5#LoRA#QLoRA