Security and privacy considerations for model routers

A router becomes part of the application's trust boundary

A hosted model router is not only a traffic director. It terminates the connection from the client, reads the application-layer request, and creates a separate connection to the upstream model provider. The service can therefore have plaintext access to prompts, responses, request metadata, model choices, tool definitions, tool calls, and any secrets included in those payloads.

Ordinary TLS protects each network connection from outside observers, but it does not hide the payload from the router that the client intentionally selected. It also does not prove that the response returned by the router preserves the response produced upstream. Selecting a router is therefore a security and supply-chain decision as well as a cost, reliability, or developer-experience decision.

The right level of scrutiny depends on the workload. Public summarization and synthetic test traffic have a different risk profile from source code, customer records, health information, financial data, privileged tool calls, or agents that can change external systems.

Documented risks include manipulation and secret exfiltration

The paper Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain studies routers in the application-layer man-in-the-middle position. It describes payload-injection attacks that rewrite returned tool calls and secret-exfiltration attacks that collect sensitive material visible in routed traffic.

The study reports malicious behavior in a sample of paid and free routers, including code injection, adaptive delivery, interaction with researcher-controlled cloud credentials, and theft from a researcher-controlled cryptocurrency key. The sample should not be read as an estimate for the entire commercial market, but it demonstrates that the attack path is practical rather than hypothetical.

Agent workloads deserve particular attention because a modified response can become an executable tool call. A small change to a shell command, package source, destination URL, or tool argument can turn an apparently normal model response into a supply-chain or credential attack.

Evaluate the provider, deployment, and full data path

Begin with the legal entity and operating history behind the service. Confirm how incidents are reported, how security contacts are handled, what contractual commitments apply, and whether the provider publishes architecture, subprocessors, retention rules, and changes to those policies.

Map the complete path rather than reviewing only the upstream model endpoint. Include router logs, caches, analytics exports, support access, backups, regional replicas, fallback providers, observability tools, and any feature that stores prompt or response content. A regional model endpoint does not establish regional handling for every intermediary.

Self-hosting changes the allocation of responsibility but does not remove it. Teams operating their own router still need dependency controls, secret management, hardened deployment, access review, patching, logging, and a way to validate the software and configuration running in production.

• Identify every party that can receive request or response content
• Confirm retention, training, support-access, and deletion policies
• Review authentication, authorization, audit, and key-management controls
• Test whether fallbacks preserve residency and data-policy requirements
• Require a documented incident and notification process

Minimize the sensitive material a router can observe

Do not send secrets merely because a model or tool may need them later. Keep credentials in the execution environment, pass references instead of values where possible, and let narrowly scoped tools retrieve only the data required for an approved action.

Classify workloads before routing. Sensitive requests can be limited to approved deployments, providers with suitable contractual and technical controls, or direct provider connections. Redact unnecessary personal or proprietary data before the routing boundary and avoid content logging unless it serves a defined operational purpose.

Use separate credentials and budgets for router traffic. Short-lived, scoped credentials and per-environment keys make suspicious activity easier to contain and attribute than one broad credential shared across development, testing, and production.

Client-side controls reduce exposure but do not prove provenance

The research evaluates three controls that can be deployed without model-provider changes: fail-closed policy gates for high-risk tools, response-side anomaly screening, and append-only transparency logging. Each addresses a different part of the problem.

A policy gate can block obvious high-risk commands or unapproved destinations before execution. Anomaly screening can flag unusual tool names, arguments, schemas, or secret-like strings. Transparency logs preserve request and response evidence needed to investigate a suspicious router or session.

None of these controls can prove that the intermediary preserved the upstream response. Allowlist-only gates can be bypassed when an attacker uses an approved domain or a pre-positioned local component, anomaly detection trades coverage against false positives, and logs help after an event rather than preventing it. Use layered controls and keep high-impact tools behind explicit authorization and sandbox boundaries.

Plan for rapid containment and provider switching

A router incident can affect confidentiality, response integrity, credentials, and every downstream action taken by an agent. The response plan should be able to disable the route, revoke associated credentials, stop high-risk tools, preserve logs, identify affected sessions, and move traffic to a known alternative.

Record the selected router, upstream provider, model, policy version, tool calls, and credential scope for each request chain. Store only the content necessary for the stated security and operational purpose, and protect the incident evidence from modification.

Practice the switch before an incident. A compatibility claim is not enough when provider-specific models, tool schemas, streaming behavior, or error handling differ. Maintain a tested direct-provider or secondary-router path for workloads whose recovery time matters.

Use a workload-specific security review

There is no universally safe router or single checklist that replaces system-specific analysis. Evaluate the provider against the data, tools, privileges, availability requirements, and failure consequences of the workload being routed.

Repeat the assessment when the provider, deployment, policy, upstream model, agent permissions, or data classification changes. Products in this category evolve quickly, and a review reflects only the sources and observations available at its publication date.

• Define what the router can read, change, store, and forward
• List the highest-impact tool actions reachable from model responses
• Set hard eligibility rules for sensitive workloads
• Limit credentials, logging, retention, and outbound destinations
• Test fail-closed behavior and the emergency traffic-switching path
• Review current provider documentation and conduct your own assessment