Kong Inc.the cloud connectivity company, has announced the overall availability of Kong AI Gatewaythe corporate provides an AI-native API gateway to administer and secure generative AI workloads in any cloud environment. The product, Initial release Released in beta in February, it has already seen significant adoption as corporations rush to place the generative AI technology into operation.
The Kong AI Gateway provides a variety of infrastructure features tailored for AI, including support for multiple large language models (LLMs), semantic caching, semantic routing, semantic firewalls, and model lifecycle management. Crucially, it integrates seamlessly with Kong's existing API platform, allowing enterprises to administer their AI alongside traditional APIs.
“Companies and developers are developing use cases for brand spanking new generation AI to create higher user and customer experiences. But like several technology on this planet, within the early days, if you start something recent with cutting-edge technology, it's very hard to scale it in production without having the suitable infrastructure in place,” explained Marco Palladino, CTO and co-founder of Kong, in an exclusive interview with VentureBeat.
According to Palladino, the Kong AI Gateway is “probably probably the most powerful AI infrastructure technology on this planet” since it builds on Kong's existing gateway capabilities while also providing deep AI-specific features. He drew parallels to the early days of APIs, noting that APIs only really exploded when the suitable infrastructure emerged to support developers.
Introspecting AI traffic and providing a unified API
A key differentiator is Kong AI Gateway's ability to research AI traffic and supply a unified API for consuming a number of AI providers. “With every other API management platform, the LLM, the API that provides you access to the LLM, is viewed as an API like several other API,” Palladino said. “That's not enough. Kong goes to the subsequent level.”
The gateway provides fast security, compliance, governance, templating and a lifecycle around AI prompts. It also offers “L7 AI remark metrics” to supply insight into provider performance, token usage and value of overall AI traffic – capabilities that Palladino says are unmatched because the competition “stops on the API layer and doesn’t go deeper into the AI ​​layer.”
Monetizing fine-tuned AI models alongside APIs
Kong's unified control plane, Connectwill allow corporations to monetize their fine-tuned AI models alongside traditional APIs. “We consider the subsequent frontier to API monetization is monetizing the models themselves,” Palladino told VentureBeat. “Because when the models are fine-tuned with specific company information that only your organization has, there’s value in leveraging that information and monetizing it.”
The start takes place in the center rapidly increasing interest in generative AI following the viral success of OpenAI's ChatGPT. However, many corporations are still grappling with the right way to effectively use and control the technology. Kong goals to simplify this with its purpose-built AI gateway.
“Every enterprise organization on this planet was attempting to construct the identical thing we were constructing,” Palladino said, noting that the product was made generally available more quickly attributable to overwhelming demand. “Once we announced there was a beta feature, everyone desired to learn more about it. We have already got enterprise customers using this feature in production, a few of them have even already began the beta version.”
The company was founded in 2009 under the name Mashape and spun off as Kong in 2017. $170 million in enterprise capital thus far. The company enables trillions of API transactions for over 500 organizations worldwide. With its forward-thinking bet on AI-native infrastructure, Kong is uniquely positioned to enable the subsequent wave of generative AI adoption within the enterprise.
“To unleash GenAI at scale, it’s critical that we deploy the suitable AI infrastructure,” Palladino argued. “Kong AI Gateway is strictly that.”
