JAX

Headlines · 4

1. 7d
   Researchers fix precision bugs in Softmax and cross-entropy

   A paper titled “A Fix for Softmax and Cross-Entropy” reports two floating-point defects in standard Softmax and cross-entropy implementations used by PyTorch, JAX, TensorFlow and other systems. The issues can prematurely zero the correct-class gradient in confident classifiers and leave residual gradients when a student model exactly matches a teacher during knowledge distillation. The proposed changes preserve the original mathematical definitions and avoid requiring higher floating-point precision, while experiments reported smoother training and improved stability in modular-arithmetic and distillation tasks.

   机器之心 · 🔥 0
2. Oct 23
   Mirendil-Base trained by one researcher and AI agents

   Mirendil says one researcher used AI agents to pretrain the 100B–500B-parameter, code-focused Mirendil-Base from scratch in 3.5 months for less than $10 million, including agent token usage, data processing, and research experiments. The company says agents wrote more than 99% of the code, while the researcher set direction, evaluated results, and intervened at key points; training used public data without purchased datasets, distillation, or synthetic data. Mirendil’s tests reported that Mirendil-Base ranked second behind Kimi K3 on cross-entropy across seven held-out code repositories, ahead of Nemotron-550B-Base, GLM-5.3, and DeepSeek-V4-Pro-Base.

   36氪 人工智能 · 🔥 0
3. Oct 11
   Hugging Face Transformers tops GitHub Trending

   Hugging Face's Transformers library appeared on GitHub Trending's daily list. The project positions itself as the model-definition framework for state-of-the-art text, vision, audio, video and multimodal models, for both inference and training, and its documentation says there are over 1M Transformers model checkpoints on the Hugging Face Hub. It says the library interoperates with training frameworks such as Axolotl, Unsloth, DeepSpeed, FSDP and PyTorch-Lightning, and with inference engines and modeling libraries including vLLM, SGLang, TGI, llama.cpp and mlx, and requires Python 3.10+ and PyTorch 2.6+.

   GitHub Trending（每日） · 🔥 0
4. Oct 10
   Google adds cycle-level Pallas kernel profiling to XProf on TPU v7

   Google has added a kernel performance profiling suite to XProf, its open-source profiler for TPU workloads, giving developers cycle-level detail on custom Pallas kernels. Such kernels — built with Pallas, Mosaic or Triton — skip standard XLA passes and previously appeared as a single opaque block in traces, skewing static cost-model metrics such as "optimal FLOPs" and effective throughput. The suite exposes lowered MLIR and per-cycle low-level operation (LLO) data, and samples hardware counters at a 1µs floor (sub-microsecond with external triggers). In a Google blocked matrix-multiplication example, sync_wait counters revealed memory stalls; adding triple buffering cut kernel execution from 125.5µs to 88µs, about 30%, though the article stresses this is one demo kernel, not a benchmark.

   InfoQ 中文 AI&大模型 · 🔥 0

Experience and discussion from the community

Share my JAX experienceAsk about JAX