At Modal, our serverless compute powers real-time inference, training, and batch jobs, scaling automatically with demand. Sandboxes offer a different model, containers whose lifecycles you explicitly control. A growing number of workloads require something in-between; autoscaling serverless compute that maintains continuity between requests. Consider a speech-to-text model with decoder state for an audio stream, a voice agent that holds a conversation, or a multiplayer game. For these workloads, users must come back to the same container on every reconnect for the entirety of their session.

We recently launched Sticky Sessions, built on top of our low-latency Modal Servers. Every session’s requests land on the same container. To use Sticky Sessions, just create a Modal Server with the @modal.sessioned() decorator and start the sessions with the SDK or the HTTP API:

@app.server()
@modal.sessioned()
class Server:
    @modal.enter()
    def start(self):
	      subprocess.Popen(["python", "-m", "http.server", "8000"])

@app.local_entrypoint()
def main():
    session = Server.sessions.start(idle_timeout=600)
    headers = {"Modal-Authorization": f"Bearer {session.token}"}
    print(requests.get(Server.get_url(), headers=headers).text)