Endpoint Candidates

Headlines · 1

1. 5d
   Modal launches Endpoint Candidates for evaluating inference endpoints

   Modal announced Endpoint Candidates, now generally available for all Modal workspaces. Users can create a candidate deployment for an existing Dedicated Endpoint, choosing an open model from Modal's catalog or bringing their own weights from a Hugging Face repo or a Modal Volume, then configure a traffic sample so the candidate mirrors a portion of production traffic. Production keeps handling every original request and returning the response users see, while the dashboard shows the candidate's latency and throughput metrics and supports load benchmarks; each candidate also has its own URL for running existing evaluations, and it can be promoted to production from the dashboard. Modal says mirroring is best-effort by design, with a separate bounded buffer per candidate so slow or failing mirrors are cancelled without holding up production, and candidates use standard Modal compute pricing including mirrored requests.

   Modal Blog · 🔥 0

Experience and discussion from the community

Share my Endpoint Candidates experienceAsk about Endpoint Candidates