Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

OpenAI-compatible API

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 4

  1. JitArchives opens an AI-assisted electronic archive system

    The JitArchives project team announced an open-source electronic-archive system for enterprises and public institutions, built on the JitWord collaborative AI document foundation. It combines archive lifecycle workflows, an evidence-bound read-only archive Agent, and a knowledge-graph feature implemented with SQLite FTS5 and TF-IDF without separate vector or graph databases. The team says this lightweight design reduces private-deployment overhead, while noting limitations such as weak semantic generalization and no text extraction from scanned PDFs.

    掘金 · 人工智能 · 🔥 0
  2. Contract-Summary Service Uses Staged Rollout to Migrate from M1 to M2

    The contract-summary service team reported migrating its main model from M1 to M2 through 10%, 50%, and 100% traffic stages over nine days. During the rollout, M2 produced a 24% downstream parsing failure rate because of a changed date format, and its peak p99 latency exceeded seven seconds when concurrent-request limits caused queuing; the team fixed both issues, increased concurrency capacity, and added automatic fallback from M2 to M1 after HTTP 429 responses. The team said monthly model costs fell from 3,300 yuan to 2,000 yuan after the migration, while the 41 affected contracts were automatically retried.

    掘金 · 人工智能 · 🔥 0
  3. BusyBox client adds LLM fallback to Xiaomi AI speaker

    The project author released an MIT-licensed BusyBox shell client for the Xiaomi AI Speaker MDZ-25-DA, also identified as S12A. It keeps native XiaoAI handling for requests such as lighting, volume, alarms and weather, while routing unsupported questions or explicit DeepSeek requests to a configurable large-model API. Answers are synthesized with EdgeTTS and played sentence by sentence, without requiring a relay server or an always-on computer. The author reports testing only the MDZ-25-DA/S12A. Installation requires opening the device, connecting a serial interface and possibly modifying system partitions, creating risks of bricking, warranty problems and instability. Voice interruption during large-model playback is not supported, and all-day stability remains under validation.

    掘金 · 人工智能 · 🔥 0
  4. Open Typora 0.22.0 adds AI-assisted Markdown workflows

    Open Typora developer anichikage announced version 0.22.0 with AI-assisted selection rewriting, reviewable cross-document changes, local retrieval and directory question-answering. The independent open-source Markdown editor also preserves recovery points, protects selected content by default, and supports configurable OpenAI-compatible, OpenRouter, LM Studio, and Ollama model services. The announcement notes that AI features require the desktop app and that compatibility is still being refined.

    V2EX · OpenAI 节点 · 🔥 0

Experience and discussion from the community

Share my OpenAI-compatible API experienceAsk about OpenAI-compatible API

chatgpt2api: A Go-based, OpenAI-compatible API and self-hosted workspace for ChatGPT image generation, image editing, text, search, and audio capabilities.

What problem it solves Developers need a unified way to integrate ChatGPT website capabilities into applications that expect OpenAI-compatible endpoints. They also need account-pool management, proxy configuration, import workflows, and a self-hosted interface for creating and reviewing media. Key features OpenAI-compatible endpoints — Provides compatible routes for image generation, image editing…

github.comhttps://github.com/basketikun/chatgpt2api

unsloth: A desktop app for running and training models locally.

What problem it solves Running and training models locally requires tools for model execution, fine-tuning, agent integration, and deployment across different model formats and hardware. The project brings these workflows into a desktop app, a web UI, and a code-based version. Key features Run and train models — Run and train language, diffusion, decision, embedding, audio, image, and video models…

github.comhttps://github.com/unslothai/unsloth

vllm: A fast and easy-to-use library for LLM inference and serving.

What problem it solves Serving large language models requires efficient memory management, flexible model execution, and practical integration with existing models and APIs. vLLM addresses these needs with inference and serving capabilities spanning batching, caching, quantization, distributed execution, and multiple hardware platforms. Key features Efficient memory and serving — vLLM manages atte…

github.comhttps://github.com/vllm-project/vllm

PyRIT: An open-source toolkit for finding security and safety weaknesses in generative AI systems through automated scans, adaptive multi-turn attacks, and human-led testing.

What problem it solves Generative AI systems and applications can contain security and safety weaknesses that require structured testing across models, endpoints, and complete applications. PyRIT addresses this with repeatable assessments, interactive exploration, and extensible workflows for red teaming these systems. Key features Repeatable command-line assessments — The scanner runs repeatable …

github.comhttps://github.com/microsoft/PyRIT

VieNeu-TTS: An on-device Vietnamese text-to-speech system with instant voice cloning.

What problem it solves Developers need Vietnamese speech synthesis that can run locally, produce natural reading styles, and support voice cloning without sending audio to a hosted service. They also need integration paths for SDK use, streaming applications, and an OpenAI-compatible API. Key features Instant voice cloning — Clone a voice from a short reference recording. The clipping pipeline aut…

github.comhttps://github.com/pnnbao97/VieNeu-TTS

llama.cpp: Plain C/C++ implementation for local and cloud LLM and VLM inference.

What problem it solves Running large language and vision-language models can require substantial setup and may depend on the available hardware. The project aims to make inference accessible across a wide range of local and cloud hardware with minimal setup. Key features Minimal-setup inference — The project provides installation options for common platforms, including scripts, Docker, pre-built b…

github.comhttps://github.com/ggml-org/llama.cpp

experiential: An open source gateway and router for agent workflows.

What problem it solves Agent workflows need to use hosted, BYOK, and local models while applying access, usage, and spending controls. They also need a way to turn production traffic into routing decisions or a specialized model tailored to observed workloads. Key features Unified model access — Route hosted, BYOK, and local models through one OpenAI-compatible API, with support for connecting pro…

github.comhttps://github.com/experientiallabs/experiential

sglang-omni: A multi-stage serving runtime for omni, speech, and TTS models.

What problem it solves Interactive multimodal and speech systems often require generation to span heterogeneous stages with different compute patterns, dependency structures, and resource needs. Serving these pipelines requires coordinated stage lifecycles, inter-stage transport, model integrations, and a unified API surface. Key features Coordinated multi-stage runtime — Models generation as coor…

github.comhttps://github.com/sgl-project/sglang-omni

infinite-canvas: An open-source infinite-canvas workbench for image creation and iterative visual workflows.

What problem it solves Image creators need to organize generation, editing, conversational assistance, prompts, and assets while repeatedly exploring visual ideas. The project brings these activities into one interface so that related work can be arranged, connected, and iterated on the same canvas. Key features Infinite canvas workflow — Create and manage multiple canvas projects with draggable a…

github.comhttps://github.com/basketikun/infinite-canvas

maigret: Collect a dossier on a person by username from thousands of websites.

What problem it solves Username-based investigations require checking many websites and collecting the information exposed on matching profile pages. This process can be fragmented across site-specific searches, while recursive discovery, filtering, and report generation add further complexity. Key features Broad Username Search — Search thousands of sites from the command line, Python API, or bui…

github.comhttps://github.com/soxoj/maigret

open-duet: An open-source workspace for creating named AI agents with memory, routines, and approval before they act.

What problem it solves Teams that rely on AI agents need a way to give each agent a role, workspace, and instructions, keep context across conversations, and control what agents are allowed to do. Without that, agents run blind: actions happen without review, memory does not persist, and repeating scheduled work means redoing setup by hand. Key features Create agents and teams — Give an agent a na…

github.comhttps://github.com/Anil-matcha/open-duet

FreeToken: An edge-native MoE serving engine for running frontier-scale open-weight models on personal and consumer hardware.

What problem it solves Running frontier-scale open-weight models locally requires an inference platform that can make efficient use of the heterogeneous hardware available on a personal device. The project addresses this by treating GPUs, CPUs, host memory, and interconnects as a unified, elastic inference platform. Key features Edge-native MoE runtime — The runtime provides MoE serving with bandw…

github.comhttps://github.com/FlashML-org/FreeToken

agentgateway: An open source proxy for AI-native connectivity across agents, LLMs, tools, and external data sources.

What problem it solves Agentic applications need to communicate securely with LLMs, tools, external data sources, and other agents across different frameworks and environments. Agentgateway provides a central proxy that adds security, observability, and governance to agent-to-LLM, agent-to-tool, and agent-to-agent communication. Key features Unified LLM Gateway — Route traffic to major LLM provide…

github.comhttps://github.com/agentgateway/agentgateway

jan: An open-source ChatGPT replacement for running LLMs locally with full control and privacy.

What problem it solves Jan provides a desktop application for downloading and running LLMs with local control and privacy. It also connects to supported cloud models and exposes an OpenAI-compatible local API for use by other applications. Key features Local AI Models — Download and run supported LLMs from HuggingFace on the local machine. The README lists Llama, Gemma, Qwen, and GPT-oss as exampl…

github.comhttps://github.com/janhq/jan

freellmapi: An OpenAI-compatible gateway for aggregating free LLM providers and custom endpoints.

What problem it solves Combining free-tier AI providers manually requires handling separate SDKs, rate limits, and failure modes. Provider catalogs and quotas also change frequently, making it difficult to keep a multi-provider setup current and within usage limits. Key features Unified API surfaces — Expose chat, responses, completions, media generation, speech, transcriptions, embeddings, and mo…

github.comhttps://github.com/tashfeenahmed/freellmapi

mesh-llm: Mesh LLM pools GPUs and memory across machines and exposes the result as one OpenAI-compatible API.

What problem it solves Running large language models can require resources that do not fit on a single machine. Mesh LLM addresses this by pooling GPUs and memory across nodes, routing requests to suitable peers, and splitting large models across machines. Key features OpenAI-compatible serving — Every node exposes the same `/v1` API, while requests are routed by the `model` field to a peer that c…

github.comhttps://github.com/Mesh-LLM/mesh-llm