SenseNova unveils Loop-9B-Preview multimodal model
AI summary
SenseNova model documentation reposted by r/LocalLLaMA presents SenseNova-Loop-9B-Preview as a 9B multimodal language model for long-context and agentic workflows. It accepts text and images and generates text for document understanding, coding, and external-tool interaction, using a looped decoder, two-stream manifold-constrained hyper-connections, hybrid attention, and training recipes reaching 262,144 tokens. The model documentation says 24 decoder layers are applied twice, producing 56 layer executions from 32 distinct layers.
In-depth insight
The preview targets multimodal agentic workloads with long-context training and a weight-sharing looped architecture, but its technical details come from vendor documentation rather than independent testing.
| Overview SenseNova-Loop-9B-Preview is a multimodal language model developed for long-context and agentic workflows. It accepts text and images and generates text for applications that combine document understanding, coding, and interaction with external tools. The model combines a looped decoder with two-stream manifold-constrained hyper-connections (mHC). A contiguous block of decoder layers is executed twice with shared weights, while learned mappings manage information flow between two residual streams in the looped section. Together, these components provide a deeper execution path without duplicating the repeated block's parameters. Documented training recipes combine long sequences, multimodal inputs, and supervised task trajectories, with a maximum training sequence length of 262,144 tokens. For the best model performance, we recommend using our documented deployment environment and configuration parameters. Model highlights submitted by /u/jacek2023 [link] [comments] |