Tarek Ziadé reported a Transformers implementation that streams quantized MoE experts from SSD into a bounded device cache, enabling the 180B-parameter Qwen3.8-Flash-Next model to run on a MacBook M5 with 32 GB of unified memory. The author measured 13.9 tokens per second after optimization and said this remained about 6% behind Strata, with per-layer synchronization identified as the remaining gap. The implementation is available through an open pull request, but its API was still under discussion with maintainers.
What problem it solves
AI and machine learning workflows require vector search, fast random access, multimodal data storage, data evolution, and hybrid search combining vectors, full-text, and SQL predicates. Lance brings these capabilities directly into the lakehouse format.
Key features
Expressive hybrid search — Combine vector similarity search, full-text search with BM25, and SQL analytics on …
What problem it solves
Running large language and vision-language models can require substantial setup and may depend on the available hardware. The project aims to make inference accessible across a wide range of local and cloud hardware with minimal setup.
Key features
Minimal-setup inference — The project provides installation options for common platforms, including scripts, Docker, pre-built b…
What problem it solves
Serving language models requires reading checkpoints, tokenizing requests, and running accelerated kernels while managing a local model server. TensorFold addresses these requirements with a native binary that can run on Apple Silicon and NVIDIA GPUs without requiring Python or MLX for serving.
Key features
Native Metal and CUDA inference — The Zig-based engine executes mode…
What problem it solves
Organizations need to structure, store, organize, and visualize technical and non-technical cyber threat information while linking it to primary sources. They also need to relate that information, infer useful connections, and exchange data with other platforms and tools.
Key features
Structured threat intelligence — Organize technical and non-technical information through a…
What problem it solves
Developers need a development environment that combines an IDE with terminal multiplexing while remaining accessible across machines. Rune also addresses language tooling, workspace organization, extensibility, and remote development through a connected application and headless mode.
Key features
Connected development workspaces — Rune instances can form an end-to-end encryp…
What problem it solves
Mosaics obscure portions of videos and images, making it difficult to view the affected content without changing the rest of the picture. Users need a restoration workflow that can identify selected mosaic regions and process them with GPU-accelerated video handling.
Key features
Targeted mosaic restoration — Detects mosaics in videos and restores them while leaving the rest…
What problem it solves
LLM and Visual Gen inference requires specialized kernels, efficient runtime orchestration, and flexible deployment across NVIDIA GPU configurations. Developers also need a modular framework that supports customization while integrating with established inference tooling.
Key features
Optimized inference kernels — Custom kernels accelerate common inference operations, includ…
What problem it solves
RAW photo editing requires a workflow that can handle camera files, detailed color and tone adjustments, masks, and local AI processing without sending image data to an account-based service. The project addresses these needs with a cross-platform editor that supports non-destructive changes, offline AI tools, and broad RAW format support.
Key features
Non-destructive RAW wo…
What problem it solves
Technical drawing, layout work, and solid modeling often require tools that can edit common CAD files and operate across desktop and browser environments. Open CAD Studio addresses these needs with native DWG and DXF handling, a shared editing core, and workflow options for both interactive and automated use.
Key features
Native DWG and DXF workflow — Open, edit, recover, an…
What problem it solves
Coding agents can misunderstand requirements, use overly verbose terminology, work without sufficient feedback loops, and accelerate codebase complexity. These skills address those failure modes with repeatable practices that preserve developer control over the engineering process.
Key features
Requirements alignment — Grilling workflows ask detailed questions about a propos…