Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

NVIDIA

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1493

  1. Honeywell Deploys Forge Plant Cognition at Dangote Lekki Refinery

    Honeywell announced that it will introduce the Forge Plant Cognition industrial AI platform at Dangote Group’s Lekki Refinery in Nigeria, according to TechNews. Built using NVIDIA’s AI platform and Dell Technologies’ AI infrastructure, the system is designed to identify production bottlenecks, optimize yield and product value, detect problems, recommend corrective actions, and scale industrial expertise. Honeywell estimates the deployment could raise operating profit by more than $0.29 per barrel and generate $60 million in annual profit at full capacity, but these figures have not been independently demonstrated.

    科技新报 · 🔥 0
  2. NVIDIA reportedly requires TSMC 3nm base dies for NVHBM

    TechNews, citing Seoul Economic, reports that NVIDIA asked Samsung Electronics, SK hynix and Micron Technology to have TSMC manufacture the base dies for customized NVHBM using its 3-nanometer process. Samsung reportedly accepted the requirement because its 4-nanometer capacity was already full, while retaining its in-house approach for standard HBM and customized HBM for customers such as AMD. TechNews also reports that Samsung's two HBM production tracks share 1c Core Die capacity, which may constrain differentiation between its in-house bare dies.

    科技新报 · 🔥 0
  3. Tenspresso Seeks $50M at a Potential $500M Valuation

    According to reports citing people familiar with the matter, Tenspresso, a three-month-old AI chip startup founded by former Google AI chip engineers, is discussing new financing that could value it at $500 million. The company plans to raise at least $50 million after recently receiving a $20 million injection. Altimeter Capital Management has reportedly contacted Tenspresso about a potential investment, while the company is also discussing the round with strategic investors. The final amount and valuation remain subject to change. Tenspresso's current plan calls for completing the design of its first chip in late 2027 or early 2028.

    Bloomberg Technology · 🔥 0
  4. NVIDIA Details AI Factory Operations Agent for GPU Clusters

    NVIDIA Technical Blog says the AI Factory Operations Agent uses specialized sub-agents to detect failures, correlate root causes across software and hardware layers, and provide recommendations through a chat interface. Built around the NVIDIA NemoClaw blueprint, the developer example runs on NVIDIA OpenShell with locally served NVIDIA Nemotron open models and communicates through Model Context Protocol. NVIDIA says the system is deployed internally for NVIDIA DGX SuperPOD operations and is available as experimental open-source software.

    NVIDIA Technical Blog · 🔥 0
  5. Goldman and Apollo Seek Buyers for SpaceX's $40 Billion AI Chip Financing

    Bloomberg reports that Goldman Sachs and Apollo Global Management are in talks with Wall Street investors to sell portions of a roughly $40 billion chip-financing arrangement for SpaceX. Some investors are being asked for commitments of up to $500 million before formal loan syndication begins. TechNews, citing related reporting, says the financing would fund Nvidia chip purchases and include an $8.3 billion senior secured loan plus multiple bond tranches, but the deal remains unconfirmed.

    Bloomberg Technology · 🔥 0
  6. Coherent reports fiscal 2027 Q1 revenue growth of 49.9%

    According to an AI-generated IT之家 report, Coherent reported fiscal 2027 first-quarter revenue of $2.37 billion, up 49.9% year over year, and attributable net income of $385 million, up 70.2%. Data center and communications revenue rose 79% and represented about 82% of total revenue. Operating cash flow was negative $92.7 million, while capital expenditure totaled $775.5 million. The company guided to second-quarter revenue of $2.55 billion to $2.75 billion and Non-GAAP EPS of $2.28 to $2.48. The figures were not independently verified in the supplied material.

    IT之家 AI · 🔥 0
  7. NVIDIA 621.14 Driver Optimized for DLSS 5 Game Integrations

    Wccftech reports that NVIDIA released GeForce Game Ready Driver 621.14, optimized for the launch of World of Warcraft Forever, DLSS 5 integration in Hogwarts Legacy and the upcoming War Thunder: Eagle's Shadow update. Hogwarts Legacy uses Neural Rendering for specific visual elements while leaving character faces untouched; the War Thunder update applies it to shadows, lighting and reflections. World of Warcraft Forever does not include DLSS technologies but supports NVIDIA Reflex for lower latency.

    Wccftech · 🔥 0
  8. Hugging Face CEO Says AI Doom Risk Is Close to Zero

    At the WIRED World Fair in Miami, Hugging Face CEO Clément Delangue said his personal P(doom)—the probability of AI causing human extinction—is “close to zero.” He attributed most AI risk to a small number of companies developing frontier models behind closed doors, arguing that open development and a broad ecosystem are the main mitigations. Delangue also said Hugging Face now has more than 20 million AI builders and has sold 50,000 Microducks robots, figures disclosed by the company’s CEO.

    WIRED AI · 🔥 0
  9. Ricursive Releases Ripple-65B Preview

    Ricursive Intelligence released Ripple-65B Preview, a 65-billion-parameter dense Transformer model under the Apache 2.0 license. The company says it was pretrained on 12 trillion tokens, supports up to 1 million tokens of context, and scored 80.52 on TerminalBench 2.1. Other reported results place it behind Inkling 2 Small on SWE-Bench Pro v2.0 Hard, DeepSWE v1.1, and MCP Atlas. Ricursive reported total development and training costs of about $32 million.

    Hacker News 首页(100+ 分) · 🔥 0
  10. CUDA Toolkit 13.5 Brings Vera Rubin to General Availability

    NVIDIA released CUDA Toolkit 13.5 with general-availability support for the Vera Rubin GPU architecture, moving it beyond early developer access for enterprise production. The release expands GPU scheduling through locality domains and MPS v3, adds multi-GPU communication features, and updates libraries and developer tools; NVIDIA also reports cuBLAS optimizations for NVFP4 and MXFP8 workloads, though the supplied reports provide no independent performance benchmarks.

    NVIDIA Technical Blog · 🔥 0
  11. Hugging Face Policy Chief Weighs Open AI Economics and Security

    TechCrunch’s Equity podcast interviewed Hugging Face policy chief Irene Solaiman about the costs, monetization, security and infrastructure trade-offs facing organizations building on open AI models. The discussion focused on startups, enterprises and governments adopting open models to reduce costs, while raising the unresolved question of how model builders capture value when models are freely available. Solaiman also considered the need for a well-funded U.S. player such as Nvidia to develop an open frontier model competitive with China’s, alongside the difficulty of securing open AI platforms without closing the ecosystem.

    TechCrunch AI · 🔥 0
  12. NVIDIA launches FOX blueprint for factory operations agents

    NVIDIA Technical Blog introduced the Factory Operations Blueprint (FOX), a reference architecture that connects a factory manager agent to existing factory data, tools, specialist agents, and equipment. At Siemens’ Erlangen electronics factory, FOX agents powered by NVIDIA Nemotron run production-planning simulations through Siemens Intelligence Center X, while a separate reference example coordinates defect inspection, root-cause analysis, rework, and robot fleets. NVIDIA claims the Siemens system can enable cost savings of up to 20%, but that figure and related time-to-market claims are not independently verified.

    NVIDIA Technical Blog · 🔥 0
  13. NVIDIA NeMo Turns Agent Failures into Tests and Harness Fixes

    An NVIDIA Technical Blog tutorial demonstrates an agent optimization cycle: use NeMo Compass to identify problems in recorded traces, create evaluations with NeMo Eval Author, update the agent’s instructions or harness, and compare results on development and held-out tasks. The NemoClaw example uses Hermes Agent in an NVIDIA OpenShell sandbox with recorded traces from a fictional company. In a separate car-door design example, NVIDIA reports that adding validation for malformed tool calls reduced such calls from 142 to zero. The same example reportedly achieved a 28% relative improvement in the workflow’s overall evaluation score and 35% fewer tool calls per run across six tested scenarios, with no regressions reported elsewhere.

    NVIDIA Technical Blog · 🔥 0
  14. Unitree H2-X Mishap Highlights Embodied-AI Validation Gap

    A commentary published by the WeChat account Zhixie Island reports that Unitree’s H2-X humanoid robot lost balance while landing a sideways flip in an official promotional video and broke a mechanical lower leg. The report says Unitree did not disclose how many of the robot’s actions were autonomous or provide real-scene test data. Wang Xingxing described aligning AI models with the physical world’s “last few millimeters” as a central embodied-AI challenge. The analysis also examines Unitree’s use of hardware revenue, IPO proceeds and research spending to fund models, computing power, data and ecosystem initiatives.

    虎嗅 AI · 🔥 0
  15. Apollo to Buy $40 Billion in Nvidia Vera Rubin Chips and Lease Them to SpaceX

    SpaceX announced last week that it would lease $40 billion worth of Nvidia Vera Rubin chips. Apollo Global Management will purchase the chips through a subsidiary and lease them to SpaceX. The arrangement follows a June deal involving Apollo, Broadcom and Blackstone, which raised $35 billion to buy Google tensor processing units and lease them to Anthropic.

    The Information · 🔥 0
  16. Emirates SkyCargo launches AI Tech cargo service

    Emirates SkyCargo launched AI Tech, a cargo service for companies building AI infrastructure. The service covers critical and sensitive hardware, including server racks, chips and GPUs, with priority loading and scheduling, specialised handling, airport-to-airport tracking and monitoring from the airline’s command centre in Dubai. Emirates SkyCargo says the service is intended to help keep customers’ deployment schedules on track. The company says it transported more than 700 tonnes of AI infrastructure across its global network in July. It plans to operate 23 dedicated freighters by March 2027, while its website lists 12 dedicated freighters currently.

    The Next Web · 🔥 0
  17. Democratic Sweep Raises Questions for AI Data Centers

    Stratechery and TechNews report that the 2026 U.S. midterm results are unlikely to halt American AI investment because both parties support maintaining U.S. leadership in AI, while a federal moratorium on data-center construction faces major legislative hurdles. TechNews identifies potential policy priorities including grid-cost sharing, permit reviews, transmission reform and China-related export controls. Grist reports that Democrats gained control of both chambers of Congress and several state governments, increasing their leverage over climate policy and oversight. At the state and local levels, environmental permitting and community opposition could still affect data-center schedules. TechNews cites an external estimate that 13.1–13.3 GW of planned Texas data centers face schedule risk.

    Techmeme · 🔥 0
  18. Supermicro to Sell SambaNova SN50-Nvidia Inference Systems

    Supermicro will build and sell AI inference systems combining SambaNova SN50 chips with Nvidia GPUs, with the initial collaboration planned around 10,000 SN50 chips. SambaNova says Nvidia GPUs will read prompts while SN50 chips generate answers, and the racks will join Supermicro’s Data Center Building Block Solutions. Customers can order SN50 RDUs alongside GPUs, with initial sales planned for neoclouds and sovereign clouds before on-premises data-center operators.

    The Next Web · 🔥 0
  19. Nvidia RTX Spark laptops run on Linux, but AC power cuts GPT-OSS-120B inference to 13.9 tok/s

    Hands-on testing found native Linux installs and runs on Nvidia RTX Spark laptops such as the MSI Prestige N16 Flip AI Plus, with GPU acceleration via Nvidia's open kernel driver, a full desktop, 4K 120Hz output and working audio and input devices after configuration. The catch is power management: when plugged into wall power the GPU is locked to roughly 780 MHz on Linux versus about 2.2 GHz on Windows, and on GPT-OSS-120B Windows reached 65.4 tokens per second, WSL 65.1, an Nvidia DGX Spark 61, while native Linux managed only 13.9. The report attributes the gap to an encrypted firmware power table and a Windows-only component called Nvidia Platform Controllers and Framework, which has no Linux equivalent, and says a fix requires Nvidia to port that power-management logic into its Linux driver. On battery, Linux clocks and throughput approach Windows levels, and WSL is presented as the practical middle ground since it keeps full GPU boost while retaining tools such as vLLM.

    Hacker News 首页(100+ 分) · 🔥 0
  20. Agent Harnesses Reportedly Drive Large ARC-AGI Score Gains

    An InfoQ 中文 AI&大模型 account of YC Paper Club's Harness Night says the same model weights can score from 30% to 95% or even 100% on ARC-AGI when the harness changes. Seth Karten described a harness as an expressive layer spanning model weights, active context with a REPL, and filesystem-like scheduling state, with compression, CRUD operations, and refinement connecting the layers. Other talks addressed hardware limits and organizational issues involving permissions, trust, and collaboration.

    InfoQ 中文 AI&大模型 · 🔥 0
  21. Drug executives challenge near-term AI disease-cure predictions

    TechNews, citing The Wall Street Journal, reports that Anthropic CEO Dario Amodei has predicted AI could cure most human diseases within ten years, but executives from Vertex Pharmaceuticals, AstraZeneca, and Regeneron dispute near-term claims of sweeping cures or sharply shortened drug-development timelines. Vertex CEO Jeff Leiden said about 50 AI companies failed to produce a usable drug when given data on thousands of its compounds, while McKinsey senior partner Alex Devereson said current models cannot reliably predict the safety problems behind roughly one-quarter of mid-to-late-stage development failures. McKinsey data cited by TechNews shows $8.4 billion in global venture-capital and private-equity investment in AI drug development during 2025, although only a small number of AI-native biotechnology companies reached late-stage trials and none of their drugs had FDA approval.

    科技新报 · 🔥 0
  22. NVIDIA demonstrates RTX Spark with 97 compatible PC games

    cnBeta reported that NVIDIA demonstrated RTX Spark on November 2 running 97 popular PC games through Arm-native versions or Windows 11 Prism. The system combines a Blackwell-generation GPU and a Grace Arm CPU co-developed with MediaTek in one SoC, with up to 128GB of memory shared between CPU and GPU. It is intended for gaming, content creation and local AI workloads. The report said launch support would reach hundreds of games, while some competitive titles remained limited by anti-cheat adaptation requirements.

    cnBeta · 🔥 0
  23. Oracle reportedly scraps $7B Tencent AI chip lease

    The Financial Times reported that Oracle scrapped an estimated $7 billion, five-year agreement to lease advanced AI-chip capacity to Tencent after US officials raised concerns. The proposed deal covered about 100,000 advanced chips across Oracle data centers in Southeast Asia, with roughly 30% of the value payable upfront; Oracle, Tencent and the White House did not answer Reuters requests for comment.

    The Next Web · 🔥 0
  24. Chinese AI firms reportedly await just 1% of RTX Pro 5500 orders

    Bloomberg, citing unnamed sources, reported that Alibaba, Tencent, ByteDance and other Chinese AI companies ordered about 6 million Nvidia RTX Pro 5500 GPUs worth roughly $70 billion. The buyers reportedly expect only tens of thousands, or about 1%, to arrive within four months amid production constraints, while earlier market estimates suggested a 5% to 10% delivery rate. Nvidia declined to comment on shipment figures but criticized outdated US export controls.

    Tom’s Hardware · 🔥 0
  25. AI chip stocks rebound after post-election rout

    AI hardware stocks rebounded on November 6 after a November 5 sell-off, with the Philadelphia Semiconductor Index rising as much as 6.57%. Nvidia gained 6.44%, AMD 13.2%, TSMC 7.55%, and Marvell 9.46%, while the Nasdaq opened higher. Reuters reported that the preceding rout reflected investor concern that a Democratic congressional victory could lead to greater scrutiny of AI data-center expansion, rather than a confirmed policy outcome.

    IT之家 AI · 🔥 0
  26. How Token Economics Connects AI Inference, Models, and Data

    品玩 reports that tokens are becoming an important pricing and measurement interface between model services, inference infrastructure, and data supply, citing an official disclosure that China’s daily token calls exceeded 500 trillion as of June 2026. The analysis says token counts alone do not measure the value of embodied AI work, which also depends on inference latency, chip power consumption, execution success, equipment investment, output quality, and end-to-end task cost. It presents cases involving DeepSeek, Google Gemini, NVIDIA, AGIBOT, and Chinese AI infrastructure providers while cautioning that some performance figures are vendor-reported or specific to test conditions.

    品玩 实时要闻 · 🔥 0
  27. Agility Robotics begins Nasdaq trading after SPAC merger

    Agility Robotics began trading on Nasdaq under AGLT after merging with Churchill Capital Corp XI, becoming the first dedicated U.S. humanoid-robotics company to go public. The transaction was valued at $2.5 billion on a pre-money equity basis and was expected to raise about $465 million before transaction fees. Filings reported 2025 revenue of $1.782 million and a net loss of about $138.09 million. Agility also said it had received more than $300 million in multiyear Digit 5 commitments, a figure not independently verified in the available reports.

    Bloomberg Technology · 🔥 0
  28. Chinese AI Developers May Receive 1% of Ordered Nvidia RTX Chips

    Chinese AI developers expect to receive about 1% of the $70 billion in advanced Nvidia RTX chips they ordered for delivery over the next several months, as Nvidia struggles to boost production amid soaring demand.

    Bloomberg Technology · 🔥 0
  29. AI Spending Payback and Cash Flow Divide Big Tech Stock Reactions

    Dingjiao One’s analysis of six Magnificent Seven companies, excluding Nvidia, found that revenue growth did not produce uniform stock reactions. Alphabet’s Google Cloud revenue rose 103% to $30.7 billion, yet the shares fell 12.64% the next day; Amazon dropped more than 7% after hours following 39.3% AWS growth, while Meta fell 7% despite 26% revenue growth. The report links steadier reactions at Microsoft, Tesla and Apple to clearer spending payback paths, positive free cash flow or reduced capital spending, but presents this as editorial analysis rather than an independently established cause.

    钛媒体 · 🔥 0
  30. Nvidia CEO backs open models and specialized enterprise AI

    At the Fireworks Forge conference in San Francisco, Nvidia CEO Jensen Huang said Nvidia benefits whether open or closed AI models prevail, provided their infrastructure runs on Nvidia chips. He also said specialized models combining domain knowledge, tools and evaluations could give enterprises proprietary systems comparable to superintelligence, while predicting agents and hardware-in-the-loop robot systems could become mainstream next year. TechNews reported that Fireworks.ai received as many as 25,000 Nvidia GPUs for its European expansion.

    科技新报 · 🔥 0

← PreviousNext →

Experience and discussion from the community

Share my NVIDIA experienceAsk about NVIDIA

Switchyard: An open-source library that helps an AI agent choose which model handles each request.

What problem it solves AI agents need to choose among models while balancing task success, cost, and latency for their workloads. Applications also need flexibility in where routing runs and who controls model clients, credentials, retries, and service operation. Key features Multiple routing algorithms — Routing can be configured with Auto, Task, Execution, or Composite strategies, including mode…

github.comhttps://github.com/NVIDIA-NeMo/Switchyard

vllm: A fast and easy-to-use library for LLM inference and serving.

What problem it solves Serving large language models requires efficient memory management, flexible model execution, and practical integration with existing models and APIs. vLLM addresses these needs with inference and serving capabilities spanning batching, caching, quantization, distributed execution, and multiple hardware platforms. Key features Efficient memory and serving — vLLM manages atte…

github.comhttps://github.com/vllm-project/vllm

skills: Portable instruction sets that teach AI agents how to use NVIDIA software for end-to-end workflows.

What problem it solves AI agents need product-specific instructions to use NVIDIA software reliably across physical AI, robotics, simulation, CUDA-X libraries, RAG, and platform workflows. The catalog centralizes these skills while keeping them maintained in their respective product repositories and synchronized through an automated pipeline. Key features Agent-discoverable skill catalog — The NVI…

github.comhttps://github.com/NVIDIA/skills

SkillEvaluator: An open-source tool that checks AI agent skills and measures their effect on task results.

What problem it solves AI agent skills need to be checked for quality, security, duplication, and their actual effect on agent behavior. Static checks alone cannot establish whether guidance improves task results, so evaluation must also support live comparison with and without a skill. Key features Three-tier evaluation pipeline — Independent tiers validate skill safety and structure, detect over…

github.comhttps://github.com/NVIDIA/SkillEvaluator

llama.cpp: Plain C/C++ implementation for local and cloud LLM and VLM inference.

What problem it solves Running large language and vision-language models can require substantial setup and may depend on the available hardware. The project aims to make inference accessible across a wide range of local and cloud hardware with minimal setup. Key features Minimal-setup inference — The project provides installation options for common platforms, including scripts, Docker, pre-built b…

github.comhttps://github.com/ggml-org/llama.cpp

TensorFold: A native language-model inference engine for serving checkpoints on Apple Silicon and NVIDIA GPUs.

What problem it solves Serving language models requires reading checkpoints, tokenizing requests, and running accelerated kernels while managing a local model server. TensorFold addresses these requirements with a native binary that can run on Apple Silicon and NVIDIA GPUs without requiring Python or MLX for serving. Key features Native Metal and CUDA inference — The Zig-based engine executes mode…

github.comhttps://github.com/ashhart/TensorFold

sglang-omni: A multi-stage serving runtime for omni, speech, and TTS models.

What problem it solves Interactive multimodal and speech systems often require generation to span heterogeneous stages with different compute patterns, dependency structures, and resource needs. Serving these pipelines requires coordinated stage lifecycles, inter-stage transport, model integrations, and a unified API surface. Key features Coordinated multi-stage runtime — Models generation as coor…

github.comhttps://github.com/sgl-project/sglang-omni

polars: An analytical query engine written in Rust for DataFrame and SQL queries.

What problem it solves Data analysis workflows often require expressive query composition, flexible execution models, and interoperability with columnar data. Polars addresses these needs by allowing the DataFrame API and SQL to share the same optimizer and engine, with support for large workloads and custom extensions. Key features Unified DataFrame and SQL Queries — Queries can be written with e…

github.comhttps://github.com/pola-rs/polars

FreeToken: An edge-native MoE serving engine for running frontier-scale open-weight models on personal and consumer hardware.

What problem it solves Running frontier-scale open-weight models locally requires an inference platform that can make efficient use of the heterogeneous hardware available on a personal device. The project addresses this by treating GPUs, CPUs, host memory, and interconnects as a unified, elastic inference platform. Key features Edge-native MoE runtime — The runtime provides MoE serving with bandw…

github.comhttps://github.com/FlashML-org/FreeToken

OpenShell: A safe, private runtime for fleets of autonomous AI agents.

What problem it solves Autonomous AI agents often need to read files, install packages, call APIs, and use credentials, but unrestricted access can expose host data, secrets, and networks. OpenShell provides these capabilities within an isolated runtime where declared policies constrain agent access. Key features Kernel-level runtime enforcement — Agents run in isolated sandboxes where kernel cont…

github.comhttps://github.com/NVIDIA/OpenShell

SkillSpector: Security scanner for AI agent skills.

What problem it solves AI agent skills execute with implicit trust and often receive minimal vetting before installation. SkillSpector helps users assess whether a skill contains vulnerabilities, malicious patterns, or other security risks before installing it. Key features Multi-format scanning — Scan agent skills supplied as Git repositories, URLs, zip files, directories, or individual files. So…

github.comhttps://github.com/NVIDIA/SkillSpector

jasna: Restores mosaics in videos and images through a simple app and command line that run on your GPU.

What problem it solves Mosaics obscure portions of videos and images, making it difficult to view the affected content without changing the rest of the picture. Users need a restoration workflow that can identify selected mosaic regions and process them with GPU-accelerated video handling. Key features Targeted mosaic restoration — Detects mosaics in videos and restores them while leaving the rest…

github.comhttps://github.com/Kruk2/jasna

Model-Optimizer: A library for composing model optimization techniques and exporting optimized quantized checkpoints.

What problem it solves Deep learning models often need to be adapted for efficient deployment across different inference frameworks. Model Optimizer addresses this by providing a unified set of optimization techniques and export workflows for quantized checkpoints. Key features Post-training quantization — Compress models with quantization techniques available for Hugging Face, Megatron-Bridge, Di…

github.comhttps://github.com/NVIDIA/Model-Optimizer

nvke: A self-hosting management plane for creating and operating Kubernetes clusters on NVIDIA GPU infrastructure.

What problem it solves Platform teams need to create and operate isolated Kubernetes clusters for customers on infrastructure they supply and control. They need a management plane that can provision GPU machines, manage cluster lifecycle operations, and establish node identity before issuing join credentials. Key features Independent workload clusters — Create workload clusters through the NVKE AP…

github.comhttps://github.com/dsx-ai-factory/nvke

LocalAI: An open-source AI engine for running models locally across modalities and hardware.

What problem it solves Running AI models locally can require integrating multiple inference engines, model formats, hardware accelerators, and API surfaces. LocalAI addresses this with a small core, on-demand backends, and compatible APIs that work across supported hardware without requiring a GPU. Key features Composable inference backends — Backends are packaged separately and pulled only when a…

github.comhttps://github.com/mudler/LocalAI

TensorRT-LLM: An open-source library for optimizing LLM and Visual Gen inference on NVIDIA GPUs.

What problem it solves LLM and Visual Gen inference requires specialized kernels, efficient runtime orchestration, and flexible deployment across NVIDIA GPU configurations. Developers also need a modular framework that supports customization while integrating with established inference tooling. Key features Optimized inference kernels — Custom kernels accelerate common inference operations, includ…

github.comhttps://github.com/NVIDIA/TensorRT-LLM

cuda-rust: NVIDIA's CUDA platform for Rust, providing host-side runtime crates and GPU kernel programming models in idiomatic Rust.

What problem it solves Rust developers need host-side access to the CUDA runtime and tools for expressing GPU kernels while preserving Rust's ownership and borrowing rules. The platform provides separate programming models for workloads that require direct thread-level control and workloads naturally expressed through tensor operations. Key features Shared CUDA runtime — Host-side crates provide d…

github.comhttps://github.com/NVIDIA/cuda-rust

auto-ontology: An open library that builds a context layer from enterprise metadata and exposes it to agent harnesses.

What problem it solves Agents can retrieve documents and query databases, but unreliable results undermine agentic analytics. Teams need a context layer that records where data lives, what its definitions mean, whether it is usable, how it should be queried, its provenance, and related concepts. Key features Enterprise ontology generation — Generate semantic ontologies from database metadata, infe…

github.comhttps://github.com/NVIDIA/auto-ontology