Modal launches Endpoint Candidates for evaluating inference endpoints
Modal announced Endpoint Candidates, now generally available for all Modal workspaces. Users can create a candidate deployment for an existing Dedicated Endpoint, choosing an open model from Modal's catalog or bringing their own weights from a Hugging Face repo or a Modal Volume, then configure a traffic sample so the candidate mirrors a portion of production traffic. Production keeps handling every original request and returning the response users see, while the dashboard shows the candidate's latency and throughput metrics and supports load benchmarks; each candidate also has its own URL for running existing evaluations, and it can be promoted to production from the dashboard. Modal says mirroring is best-effort by design, with a separate bounded buffer per candidate so slow or failing mirrors are cancelled without holding up production, and candidates use standard Modal compute pricing including mirrored requests.
It puts A/B-style model and endpoint evaluation inside the inference platform, letting teams compare quality, latency, throughput and cost against real production traffic without building their own traffic-mirroring infrastructure.