What is the OpenAI Flex Program?

What Flex processing is

OpenAI Flex processing is a lower-cost service tier for ordinary Responses or Chat Completions requests. You request the same eligible model and API behavior, but allow OpenAI to serve the request from lower-priority capacity. In return, the request may wait longer, time out, or receive a resource-unavailable error when Flex capacity is scarce.

The key trade is capacity, not model intelligence. Flex does not save money by sending the prompt to a smaller model or changing the task. It saves money because the workload agrees to be patient and interruptible. OpenAI describes Flex as a beta offering with limited model availability, so eligible models and prices should be checked before a rollout.

For direct OpenAI calls, the request opts in with service_tier: "flex". OpenAI recommends longer client timeouts and explicit retry behavior because Flex requests can be substantially slower than standard processing.

How Flex differs from the Batch API

Flex and Batch can have the same token economics, but they solve different workflow problems. OpenAI prices Flex tokens at Batch API rates. Batch, however, is a separate asynchronous job system with file submission, status polling, result files, a separate rate-limit pool, and a 24-hour completion window.

Flex keeps a request-shaped interface. Your application sends one request and waits on that connection for its response, which makes Flex usable in sequential workflows where step two depends on step one. Batch is usually better for large independent work queues because it offers much more throughput and does not require a client connection to remain open.

• Standard: synchronous, regular price, intended for normal availability and latency.
• Flex: synchronous, Batch-rate pricing, variable latency, and best-effort capacity.
• Batch: asynchronous bulk submission, Batch-rate pricing, separate rate limits, and completion within the batch window.

What Flex means for model costs

At list price, Flex turns service tier into another cost dimension alongside model choice, input tokens, output tokens, and prompt caching. If a model's standard input and output prices are S, Flex uses that model's Batch rates, which are commonly 50% below its standard rates. Prompt-cache discounts can reduce eligible input-token charges further.

Do not model the discount as a guaranteed 50% reduction to the whole AI bill. A practical estimate is: eligible token spend × Flex share × Flex rate, plus standard-tier spend, retries, and any fallback requests. Include engineering costs for queues, longer timeouts, idempotency, retry limits, monitoring, and duplicate-work prevention.

Fallback policy changes the realized savings. Retrying Flex until capacity appears preserves the lower unit price but increases completion time. Falling back immediately to Standard improves completion probability but reduces the discount. If a failed attempt triggered paid tools or an external side effect before the model call was retried, the effective cost can rise even when the replacement inference is cheap.

Why Flex matters for model routing

Most routing discussions focus on choosing between models or providers. Flex adds a separate decision: which service tier should run an already-selected model? A router can therefore reduce cost without accepting the quality uncertainty that comes with switching to a different model.

A useful routing order is to first decide whether the request is latency-sensitive, then select a model, then choose its service tier. Background work can be sent to Flex while interactive work stays on Standard. If Flex fails, the router can retry later, move to another Flex-capable provider for an equivalent model, or promote the request to Standard under a cost ceiling.

This works especially well when the router has workload metadata that the model cannot infer reliably: user-facing versus background, deadline, retry budget, maximum acceptable price, idempotency, and whether partial work has external side effects. Those fields make Flex selection an explicit policy rather than a guess based on prompt text.

• Route by deadline, not just prompt complexity
• Keep model selection and service-tier selection as separate decisions
• Record the requested tier, actual tier, latency, errors, retries, fallback tier, and final cost
• Set a maximum retry delay and a clear rule for Standard-tier promotion

When Flex is the right thing to try

Flex is a strong experiment when requests are valuable enough to run but not urgent enough to reserve standard capacity. Start with a small, observable slice of traffic and compare completion rate, tail latency, retry count, and effective cost against Standard and Batch.

Good candidates include offline evaluations, data enrichment, document analysis, classification, translation, background research, non-urgent agent steps, content indexing, and internal jobs with a queue already in place. Sequential workloads are especially interesting because they cannot always be packed conveniently into one batch job, yet each step can still tolerate minutes of delay.

Flex is also useful when model quality must remain fixed. If policy or evaluation results require a particular model, service-tier routing can create savings without substituting a cheaper model.

When Flex should be avoided

Avoid Flex on request paths with tight response-time objectives or high availability requirements. User-facing chat, voice, autocomplete, synchronous checkout, incident response, and latency-sensitive tool loops are usually poor default candidates. A sudden capacity error is more expensive than the token discount when it breaks the user journey.

It is also a poor fit when the caller cannot hold a long connection, retry safely, or distinguish a resource-capacity error from a permanent request error. Non-idempotent tools deserve special care: a retry must not send the same email, place the same order, or mutate the same record twice.

For large independent queues that already tolerate a 24-hour result window, use Batch first. Batch generally offers better throughput and a purpose-built job lifecycle. Finally, do not promise Flex savings in a fixed budget until production traffic proves the eligible-model mix, cache behavior, failure rate, and Standard fallback rate.

Which vendors support Flex routing?

The word Flex is not standardized. OpenAI and Google use it for discounted, lower-priority capacity. Groq uses the same label for higher rate limits at the regular on-demand price, with fast failure when capacity is unavailable. Routing platforms can normalize some of these differences, but your policy still needs to know whether Flex means lower cost, more throughput, or both.

The table below includes directly documented Flex tiers and routing products that can select them. It deliberately excludes Batch-only discounts. For example, Anthropic offers asynchronous Message Batches at 50% of standard prices, but its direct API does not document a synchronous Flex tier.

A safe rollout pattern

Start with queued, idempotent work that has a generous deadline. Tag every request with a workload class and record Standard as the control group. Use bounded exponential backoff for capacity errors, then promote to Standard only when the remaining deadline and business value justify the higher price.

Measure the entire attempt chain, not just the successful response. The core scorecard should include Flex eligibility rate, Flex acceptance rate, p50 and p95 completion time, retries per completed request, fallback-to-Standard rate, failed jobs, and effective dollars per completed task. A router should expand Flex only when the completed-task economics improve without violating the workload's deadline.