创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

NVIDIA发布3B与8B Nemotron-Labs-Sigma扩散模型

AI 总结

NVIDIA发布了3B和8B规模的Nemotron-Labs-Sigma模型,其中包括指令微调变体。据所提供的模型文档,8B指令模型在16维连续词元嵌入空间中对高斯噪声去噪,并通过可配置的DDPM或DDIM采样步骤并行解码词块。文档称这些模型可用于商业或非商业用途。NVIDIA声称其推理准确率可与领先的掩码离散扩散语言模型竞争,但所提供的报告并未独立验证这一比较。

Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.

This is the SFT (supervised fine-tuned) variant, fine-tuned for instruction following on top of the corresponding Base model. Use the bundled chat template to format conversations.

This model is ready for commercial or non-commercial use.

Highlights

  • Continuous-space diffusion language modeling at 8B scale with reasoning accuracy competitive with state-of-the-art masked (discrete) diffusion LMs.
  • Block-parallel decoding: whole blocks of tokens are denoised together; sampling quality holds up well down to few denoising steps per block.
  • Efficient DDPM/DDIM-based samplers with self-conditioning and optional classifier-free guidance, implemented in plain PyTorch in this repository.
  • Standard transformers loading via trust_remote_code; no custom kernels, flash-attention, or training framework required for inference.

#NVIDIA#Nemotron-Labs-Sigma-8B#Nemotron-Labs-Sigma-8B-instruct#Nemotron-Labs-Sigma-3B#Nemotron-Labs-Sigma-3B-instruct

原文约 2 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。