Phi-4
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- OASIS adds null routing to attention residuals
A team from Northwestern University, the Illinois Institute of Technology, Rutgers University, and other institutions proposed OASIS, whose token- and depth-level null options let Attention Residuals abstain from assigning full probability mass to real branches. According to Xinzhiyuan’s report on the paper, the method was accepted to NeurIPS 2026 and reduced average activation kurtosis by 95.90% versus Vanilla AttnResidual across LLaMA-3.2-1B, Qwen3-0.6B, and Phi-4. The reported experiments also found a 13.21-point Qwen3-0.6B W4A4 GSM8K Pass@1 advantage over Vanilla AttnResidual and recovered 77.3%–91.8% of selected RULER and LongBench performance losses at 12K-token contexts.
新智元 · 🔥 0
Experience and discussion from the community
Share my Phi-4 experienceAsk about Phi-4
Nobody has shared their experience with Phi-4 yet.