AWS and UC Berkeley have released Infinity, an open-source runtime that suspends AI agents while they wait on tools instead of holding a process and memory open, under the Apache 2.0 licence as part of the Hydro project. In a benchmark, about 75,000 idle agents fitted into 8 gigabytes of memory, the memory of a Raspberry Pi 5, because a suspended agent keeps only its conversation history. The runtime also runs on serverless platforms, where a function can end while an agent waits and resume when a tool result arrives. AWS describes Infinity as an early research release and is seeking feedback on the ideas and the implementation.
Suspend, do not wait
According to AWS, most agent harnesses run the agent loop as blocking steps: the harness calls the model, invokes a tool, waits for the result and repeats. While the agent waits, it holds a process and its memory, and it cannot respond to events that arrive in the meantime. Infinity treats each tool call as a suspension point. When an agent calls a tool, the runtime dispatches the call, saves the conversation and suspends the agent. When the tool result arrives, the runtime resumes the agent from its saved state. Each run is a bounded slice: the runtime loads the agent’s history from a store, runs one model completion, persists the output and dispatches at most one tool call, then the slice ends. A slice never blocks for a tool, a timer or any other external operation.
Built for serverless platforms
Because a suspended agent holds no running process, Infinity can run on serverless platforms, where a function is invoked for each unit of work and may exit as soon as it finishes. AWS says its reference deployment uses an SQS FIFO queue as the input queue, with one message group per thread. Each delivery invokes an AWS Lambda function that runs one step and exits. Conversation histories are stored in Aurora DSQL and deduplication state in DynamoDB, while sleep tools use SQS delivery delays and EventBridge Scheduler for timers. An agent waiting on a three-day task therefore uses no compute until its result arrives. The same agent code runs in a local in-memory mode or in step mode on the cloud, since tools, model providers and tests written against the core library work in either mode without changes.
Events arrive as synthetic tool calls
Model APIs give no way to represent an asynchronous event, since each tool call must be answered by exactly one result. Infinity solves this with synthetic tool calls: when an event arrives, the runtime adds an assistant tool call the model did not actually make, followed by a tool result carrying the event, under the name receive_event__injected. The same mechanism carries messages between parent and child agents. Each agent has a built-in spawn_thread tool that creates a child thread, and a child reports back with report_to_parent, with its final report arriving when it calls close_thread. By default each event is handled in a separate child thread, so the parent’s history stays unchanged unless the child reports. A tool can instead mark an event as associative, adding the synthetic call directly to the subscribing thread’s history. AWS lists open questions it wants feedback on, including how interleaved events affect model decisions, scheduling policies, compaction in trees of agents and event authenticity.
Why It Matters
Infinity matters because long-running agents spend most of their lives waiting, and waiting is expensive when every idle agent holds a process open. A runtime that parks an agent as stored history changes the economics of running thousands of agents at once, and it makes serverless deployment practical for work that stretches over hours or days. The benchmark behind the headline figure deserves care: AWS ran 80,000 agents for 3.2 million completions on a single thread of an AMD EPYC 9R14 server processor with in-memory stores and a simulated model, finding memory grew by about 100 kilobytes per idle agent. That is a laboratory measurement, not a deployment on a Raspberry Pi, but the direction is clear. Neighbouring approaches take a different route, such as Google’s Agent Compute Engine, which gives every Gemini agent its own isolated microVM. Infinity instead bets that suspension, not isolation, is the way to scale agent fleets.
Background
Infinity comes from the Hydro project, a collaboration between AWS and UC Berkeley that re-imagines agent runtimes using the suspension techniques programming languages use for asynchronous functions. The problem it addresses is familiar to anyone building agents that do work rather than only chat: coding agents wait on builds, triage agents wait on pull requests and assistants wait on approvals. AWS has been expanding its agent tooling alongside, including SageMaker training sessions for custom model work, and Infinity is available now on GitHub with a quickstart for building a first agent in minutes.
Source: AWS Open Source Blog
0 Comments