Qwen2.5动画教程演示下一词元选择
Reddit作者/u/Rajucg发布了一段七分钟的动画教程,使用Qwen2.5-0.5B-Instruct的真实概率讲解softmax、贪心解码、采样、温度、top-k和top-p。视频追踪一个关于月球的提示词所生成的两种续写,展示模型在“The”之后为全部151,936个词元评分,以及不同设置或随机种子如何产生不同句子。作者称这些数据是在Colab中使用Transformers和CPU运行得出,并邀请学习者指出错误。
该教程以可复现的具体案例解释生成设置如何决定模型选择高概率词元,还是探索排名较低的候选项。
I made this explainer for my small channel and would like feedback from people learning this, especially if anything is wrong.
It follows one real example the whole way. I gave Qwen2.5-0.5B-Instruct "Write one sentence about the moon." twice with the same settings (temperature 1, no top-k or top-p, different seeds). Both answers start with "The", then split: "The moon, a satellite of the Earth…" vs "The lunar South Equatorial region…".
- After "The", the model scores all 151,936 vocabulary tokens. Softmax gives moon 78.04%, full 4.46%, vast 2.24%, Moon 1.51%, lunar 1.50%.
- Greedy always takes moon. Sampling is a weighted random pick, so lunar wins about 1 time in 67, and run 2 got it.
- The loop: append the token, run the model again. Most later picks were rank 1, but run 2 also picked "South" (rank 219) and "empty" (rank 1,098), and run 1 picked "green" (rank 536).
- Temperature divides the scores before softmax: moon goes from 100% at T=0.2 to 6.3% at T=2.0.
- Top-k vs top-p on the same step: top-k 3 rescales moon to 92.1%; top-p 0.9 keeps 8 tokens here but only 1 at a later step where "of" has 97.9%, which is the argument in Holtzman et al. (2019). At the model's default top-p 0.8, lunar is cut.
- Even temperature 0 can vary on a server: Thinking Machines got 80 different answers from 1,000 runs of Qwen3-235B, caused by batch-size-dependent addition order.
All numbers come from running the model on CPU in Colab with transformers. Sources are in the description. The narration is an AI voice reading my own script.
Video: https://www.youtube.com/watch?v=r0_LTPf7MSw
submitted by /u/Rajucg [link] [comments]