Tufa Labs’ tufazip reduced 1 GB of Wikipedia information to 105.39 MB losslessly, without hand-built compression models, according to a research post reported by Tom’s Hardware. The method uses a 163-million-parameter Transformer and reportedly achieved the smallest result among benchmark entries relying on a neural network as their only predictor. On the reported test system, compression and decompression each took roughly 7.3–7.5 days on two Nvidia B200 GPUs, excluding offline starter-model training.
Ben's Bites uses a deliberately generous hypothetical model to estimate how many people would be needed to keep large-scale AI chatbots running. Assuming around four user messages per person per day, 500-word replies, constant 200-words-per-minute typing, and ten-minute waits, the author arrives at 10.4 million typists and 77.4 million total staff. The article then explains that transformer models predict text in parallel across many users rather than having a person type every reply.
The paper HiUni: Harmonizing Understanding and Generation in Shared Visual Space presents a unified multimodal model that reuses the same language-aligned ViT and dense Transformer backbone for visual understanding and generation. It adds a Pixel Diffusion Head for pixel-space generation and a lightweight HiGate pathway for fine-grained spatial details near the output stage. The authors report an average understanding score similar to the pretrained VLM and competitive image generation and editing performance, based on paper experiments.
An arXiv preprint reports that latent feedback, which feeds each token’s top-layer state into the next token’s input, can help language models reach a standard Transformer’s loss with less compute or unique data in tested settings. The authors report maximum savings of 48% in compute and 47% in unique data, with savings increasing with unique data per parameter and repetition; they also say the benefit is reduced when unique data is scarce relative to model size. A scaling law fitted on models from 60 million to 271 million parameters predicted losses within 0.3% and model gaps within 0.05% at a held-out 778-million-parameter size.
Anazhixin said its AX100 analog computing chip completed tape-out in October, using mature processes that do not depend on advanced nodes. The company claims the chip can solve matrix equations in less than 100 nanoseconds at about 80 milliwatts per operation, while calling it the world’s first fully integrated one-step matrix-solving analog chip. Vendor tests also reported gains in AI training, embodied intelligence, and wireless communication workloads.
A Juejin AI tutorial walks through autoregressive transformer generation from prompt prefill and KV caching to token-by-token decoding, attention, normalization, residual paths, feed-forward networks and LM-head sampling. It also corrects several misconceptions, including using K rather than V for attention scoring and combining partitioned K/V results through weighted reduction rather than concatenation. FlashDecoding is presented as a decode-time attention optimization that preserves the attention output, though the report supplies no independent performance benchmark.
Reddit user /u/WAMFT shared code and results for two small neural-network architecture experiments: PHSMLP, which reuses one MLP across Transformer layers with learned head pooling, and Pilot, which combines FFT-computed input history with nonlinear recurrent matrix memory. In the reported tests, shared pooled PHSMLP used 13.4% fewer parameters, while parameter-matched Pilot reported 16.7% lower test perplexity than a four-head Transformer, though Pilot performed worse on memory probes and was about 60 times slower to train.
Stanford Hazy Research and Together AI introduced Next-Concept Modeling (NCM), a text-generation architecture that first creates a coarse representation of an entire passage and then refines it layer by layer. The two-stage system combines a multiscale residual VQ-VAE with a Transformer that predicts all codes in a layer in parallel. The team reported generation speeds 10 to 31 times faster than similarly sized diffusion language models while using four to five times fewer training tokens, and released the models and code under the MIT License.
The Muir paper introduces a linear-attention architecture that combines sensitivity-allocated gating, matched-filter readout, and a numerically stable parallel update. Its authors report lower perplexity and stronger recall than Gated DeltaNet-2 and Transformer baselines, along with 10M-token streaming from a 4K training length and near-perfect passkey retrieval up to 1M tokens. A Reddit analysis also describes a 25 MB state at 10M tokens, but these technical and benchmark claims have not been independently confirmed.
Q Labs introduced Dust, a zeroth-order optimization method that estimates update directions from random, token-level activation perturbations instead of standard backpropagation. The company reported training Transformers on FineWeb up to 243M parameters and 20M tokens, while a SOTAAZ Lab reproduction found basic reproducibility but no clear advantage over backpropagation. Dust also remained far more computationally expensive than backpropagation.
An arXiv paper proposes an attention-free sequence architecture based on Grassmann flows. Its Causal Grassmann layer linearly reduces token states, encodes local token pairs as two-dimensional subspaces on a Grassmann manifold through Plücker coordinates, and fuses the geometric features back into hidden states through gated mixing.
The authors report that pure Grassmann models with 13 million to 18 million parameters achieve WikiText-2 validation perplexities within about 10% to 15% of size-matched Transformers. On SNLI, a Grassmann-Plücker head on DistilBERT achieves reported best validation and test accuracies of 0.8550 and 0.8538, compared with 0.8545 and 0.8511 for a Transformer head. The supplied report does not include an independent reproduction of these experiments.
InfoQ 中文 AI&大模型 reported on the first ICAAS 2026 roundtable, where Michael Levitt, Jack Dongarra, mathematician Xiao Ma and Jiy an He discussed AI’s technical paths, research methods and potential next phase. Participants noted that foundation models are converging on large Transformer architectures, while arguing that research should also explore alternative routes, autonomous learning environments and verification. They disagreed over whether another AI winter could emerge, with concerns including overpromising, token costs, latency and the reliability of autonomous problem-solving.
In an interview with DeepTech published by MIT Technology Review China, MIT researcher Liang Qiyao says she is optimistic about AI and therefore pessimistic about scientific research, citing the concentration of money and talent in the field. She argues that large models show surprising capabilities that researchers still cannot explain, so current AI research retains an "alchemy"-like character in which people know which methods improve performance but not why. Trained in physics, she studies AI as a complex system using small "toy models", from compositional generalization in diffusion models to post-training of large language models with evolution strategies.
Shakeel Hashim, writing for Transformer, reports that a leaked federal AI bill draft associated with Anthropic has drawn widespread criticism from AI safety advocates, who say it is significantly weaker than the bipartisan FRONTIER Act. The report says the leaked draft bill is weaker than the FRONTIER Act and makes Anthropic's safety-first pitch ring hollow. The supplied report includes no text of the draft's provisions and no Anthropic response.
What problem it solves
vLLM users need a way to run the framework on Huawei Ascend NPU hardware through a supported integration path. The plugin decouples Ascend NPU integration from vLLM and enables supported model types to run on the platform.
Key features
Hardware-pluggable vLLM integration — The plugin provides an interface for integrating Ascend NPUs with vLLM and follows the community's hard…
What problem it solves
Large language models can be difficult for individuals to train, deploy, or understand because of their scale and the abstractions used by many frameworks. The project addresses this by providing native PyTorch implementations, compact model structures, open datasets, and staged workflows for pretraining, fine-tuning, preference optimization, tool use, and agentic training.
…
What problem it solves
Real-world translation involves more than converting text between languages: tasks can require terminology, style, personalization, delimiter preservation, and structured-data constraints. Hy-MT2 is designed to support translation among 33 languages and follow translation instructions in multiple languages for complex real-world scenarios.
Key features
Instruction-driven tas…
What problem it solves
Machine learning workflows need a shared model definition so that models can work consistently across training frameworks, inference engines, and adjacent modeling libraries. Transformers centralizes these definitions and exposes a unified interface for loading pretrained models, running inference, and adapting model internals.
Key features
Broad Model Support — The framewor…
What problem it solves
Speech-processing workflows often require separate systems for transcription, translation, spoken language identification, and voice activity detection. A single multitask model can handle these tasks while reducing the number of stages in a traditional speech-processing pipeline.
Key features
Multitask speech processing — One Transformer sequence-to-sequence model covers mu…
What problem it solves
Refusal mechanisms in transformer models can block legitimate research, creative writing, and red-teaming alongside genuinely harmful content. Researchers need transparent, reproducible ways to locate refusal representations, evaluate their relationship to other capabilities, and apply controlled interventions without retraining or fine-tuning.
Key features
Refusal direction…
What problem it solves
Machine learning models must operate within practical constraints such as memory, latency, power, reliability, and safety. Existing books and papers often cover models or individual systems separately, leaving a need for material that teaches the infrastructure of machine learning systems as one subject.
Key features
Four-volume systems curriculum — The series progresses fro…
What problem it solves
Time-series forecasting requires models that can forecast univariate and multivariate series across different context lengths. Forecasters also need to incorporate optional covariates and produce quantile forecasts without requiring per-task tuning.
Key features
Univariate and multivariate forecasting — Forecast individual time series or multiple target variates from NumPy a…
What problem it solves
Deep learning models often need to be adapted for efficient deployment across different inference frameworks. Model Optimizer addresses this by providing a unified set of optimization techniques and export workflows for quantized checkpoints.
Key features
Post-training quantization — Compress models with quantization techniques available for Hugging Face, Megatron-Bridge, Di…
What problem it solves
A personalized feed must select relevant posts, rank them, and determine whether they can be shown. The implementation combines posts from followed and unfollowed accounts with filtering, transformer-based ranking, labeling, and visibility controls.
Key features
Personalized candidate retrieval — The feed gathers candidates from accounts the viewer follows and from accounts …
What problem it solves
This project provides simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations and rendered as side-by-side formatted notes to help readers understand the algorithms better.
Key features
PyTorch Implementations — The project provides implementations of neural networks and related algorithms using PyTorch…
What problem it solves
Financial candlestick sequences are continuous, multidimensional, and high-noise, making them difficult to represent with general-purpose time-series models. Kronos addresses this by quantizing OHLCV data into hierarchical discrete tokens and pre-training an autoregressive Transformer for quantitative forecasting tasks.
Key features
Financial candlestick tokenization — The t…