As the demand for generative AI continues to grow, concerns about its protected and reliable deployment are more necessary than ever. Companies wish to be certain that Large Language Model (LLM) applications developed for internal or external use deliver the best quality results without venturing into unknown territory.
Given these concerns Microsoft today announced the launch of recent Azure AI tools that may help developers address not only the issue of automatic hallucinations (a quite common problem related to Gen AI), but additionally vulnerabilities corresponding to model prompt injection will create personal or harmful content – ​​just like the Taylor Swift deepfakes generated by Microsoft's own AI image creator.
The offerings are currently being previewed and are expected to be generally available in the approaching months. However, Microsoft has not yet announced a selected schedule.
With the appearance of LLMs, prompt injection attacks have turn into increasingly necessary. Essentially, an attacker can modify the model's prompt to bypass the model's normal operation, including security controls, and manipulate it to disclose personal or malicious content, thereby compromising security or privacy. These attacks may be carried out in two ways: directly, by the attacker interacting directly with the LLM, or not directly, by utilizing a third-party data source corresponding to a malicious website.
To address these two types of prompt injection, Microsoft is adding Azure AI Prompt Shields, a comprehensive feature that leverages advanced machine learning (ML) algorithms and natural language processing to routinely goal third-party prompts and data for malicious intent analyze and block them reaching the model.
It will likely be integrated into three AI offerings from Microsoft: Azure OpenAI service, Azure AI content security and that Azure AI Studio.
But there may be more.
In addition to working to dam security-threatening prompt injection attacks, Microsoft has also introduced tools that deal with the reliability of Gen AI apps. This includes pre-built templates for safety-oriented system messages and a brand new feature called “Groundedness Detection”.
The former, as Microsoft explains, allows developers to create system messages that steer the model's behavior towards protected, responsible and data-driven output. The latter uses a fine-tuned custom language model to detect hallucinations or inaccurate material within the text output produced by the model. Both come to Azure AI Studio and the Azure OpenAI Service.
In particular, the metric for detecting down-to-earthness can be accompanied by automated evaluations to be able to subject the genetic AI app to a stress test for risk and safety. These metrics measure the potential of the app being jailbroken and producing inappropriate content of any kind. The reviews also include natural language explanations to assist developers develop appropriate workarounds for the problems.
“Today, many corporations lack the resources to emphasize test their generative AI applications so that they can safely move from prototype to production. First, it will probably be difficult to create a high-quality test data set that reflects a spread of recent and emerging risks corresponding to jailbreak attacks. Even with high-quality data, evaluations could be a complex and manual process, and development teams may struggle to interpret the outcomes to take effective remedial motion,” noted Sarah Bird, chief product officer of Responsible AI at Microsoft, in a blog post
Improved monitoring in production
When the app is eventually in production, Microsoft will provide real-time monitoring so developers can keep an in depth eye on which inputs and outputs trigger safety features like Prompt Shields. The feature, which will likely be available for Azure OpenAI Service and AI Studio, will create detailed visualizations highlighting the quantity and ratio of blocked user inputs/model outputs, in addition to a breakdown by severity/category.
This visibility allows developers to grasp malicious request trends over time and adjust their content filter configurations, controls, and broader application design to extend security.
Microsoft has been expanding its AI offering for a while. The company began with OpenAI's models but has recently expanded to other offerings, including Mistral's. Recently, the corporate even hired Mustafa Suleyman and the Inflection AI team, which appears to be an approach to reducing dependence on the research lab led by Sam Altman.
The addition of those latest security and reliability tools now builds on the corporate's work and provides developers with a greater and safer strategy to construct Gen AI applications based on the models offered. Not to say, the deal with security and reliability also underscores the corporate's commitment to constructing trustworthy AI – something that’s critical for businesses and can ultimately help attract more customers.
