Helping enterprise developer teams move faster with GenAI — a secure, managed routing service built on Envoy AI Gateway
SAN FRANCISCO, July 16, 2025 /PRNewswire/ — Tetrate, the company enabling safe, fast and profitable AI transformation, today announced the launch of the Tetrate Agent Router Service, a managed solution to improve reliability and reduce costs of Large Language Models (LLMs) at scale for developers building generative AI (GenAI) applications.
Tetrate Agent Router Service allows developers to route AI queries dynamically to the most appropriate model based on optimization factors such as inferencing cost, query complexity, model performance and/or task specificity. This helps avoid lock-in, work around model unreliability, and mitigate cost overruns. When deployed alongside Tetrate Agent Operations Director, Tetrate Agent Router Service enables centralized control of GenAI developer traffic — unlocking fast developer adoption while maintaining data governance and compliance standards.
“Enterprises are under pressure to adopt AI to improve customer experiences and operational agility, yet developers working in these industries face serious challenges in balancing innovation with risk and cost control,” said David Wang, head of product management at Tetrate. “Tetrate Agent Router Service embodies our commitment to helping these developers safely navigate the fast-growing GenAI landscape. By providing a trusted, flexible way to choose the right models in real time, we are helping customers avoid taking on the complexities of building scalable AI architectures.”
Built for Developers
Tetrate Agent Router Service is a managed service that reduces infrastructure overhead for developers and supports isolated tenancy and/or on-premises deployment. Developers can access models with their own API keys or use those provided by Tetrate. Additional features include automatic fallback to more reliable or cheaper models, an interactive prompt playground to test and refine GenAI applications quickly, and A/B testing to help developers evaluate which models perform better.
Built on Envoy AI Gateway and operated by its core maintainers, Tetrate Agent Router Service supports the most common GenAI use cases:
- For chatbots, it routes conversations to the most responsive, cost-effective model — ensuring low latency and continuity during high traffic or outages.
- For code generation, it enables dynamic model selection based on programming language, context, or compliance policy — helping developers avoid expensive misfires and hallucinated code.
- For AI agents, it coordinates API calls across multiple LLMs and tasks, delivering cost-aware execution — without introducing operational friction.
Integrated Governance from the Experts
The Tetrate Agent Router Service builds on the company’s recent membership in the Fintech Open Source Foundation (FINOS), where Tetrate is aligning AI governance with leading standards such as those from the National Institute of Standards and Technology (NIST). To meet demands for security, Tetrate Agent Router Service works seamlessly with Tetrate Agent Operations Director, which provides centralized visibility and policy enforcement across teams, clouds and models. Informed by frameworks Tetrate helped develop through FINOS and NIST, these products work in tandem to enable enterprises to maintain rigorous governance standards fit for regulated industries without compromising developer adoption speed.
The Next Step in AI Adoption
As per Gartner®, “GenAI model routers optimize costs and maximize response accuracy by intelligently directing requests to appropriate models. They ensure that each query is handled by models optimized for a specific need (e.g., creative writing, coding and image generation), which enhances output quality. GenAI model routers help achieve performance-cost trade-offs in harnessing model enhancements and innovations while limiting costs.” Gartner also notes that “GenAI Model Routers is an emerging area with only 1% to 5% market penetration, implying significant upside for growth.”*
The Agent Router Service complements Tetrate’s existing product portfolio, including the Agent Operations Director, and extends the company’s leadership in service mesh-based security and governance. By offering dedicated management planes and on-premises options, Tetrate addresses the needs of organizations requiring strict data control and security posture alignment.
Learn More About Tetrate Agent Router Service
Read the Blog >
Learn More >
Sign-Up For Details >
*Gartner, Hype Cycle for AI in Software Engineering, 2025, Adrian Leow, Haritha Khandabattu, 17 June 2025
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
About Tetrate
Tetrate enables safe, fast, and profitable AI transformation by providing trusted connectivity in AI operations. As the driving force behind Envoy and Envoy AI Gateway, Tetrate builds on proven open-source foundations to deliver solutions fit for regulated industries and government organizations. Our products, including Agent Operations Director and Agent Router Service, optimize ROI while establishing trust in AI initiatives.
MEDIA CONTACT
Cristin Connelly
Cathey.co for Tetrate
cristin@cathey.co
SOURCE Tetrate