The National Institute of Standards and Technology (NIST), the U.S. Department of Commerce agency that develops and tests technologies for the U.S. government, businesses, and most people, has re-released a testbed designed to measure how malicious attacks—particularly attacks that “poison” AI model training data—can impact the performance of an AI system.
Called Diopters (after classical astronomical and surveying instrument), the modular, web-based open source tool, Approved in 2022, is designed to assist corporations that train AI models – and the individuals who use those models – assess, analyze, and track AI risks. Dioptra will be used to benchmark and research models, in response to NIST, and provides a typical platform to show models to simulated threats in a “red teaming” environment.
“Testing the impact of adversarial attacks on machine learning models is certainly one of the goals of Dioptra,” NIST wrote in a press release. “The open source software, comparable to Generating Child, is on the market free of charge download and will help the community, including government agencies and small and medium-sized businesses, conduct evaluations to evaluate AI developers' claims in regards to the performance of their systems.”
Dioptra was presented along with documents from NIST and the recently prepared NIST Institute for AI Security show the probabilities of mitigating among the dangers of AI, comparable to its misuse to generate non-consensual pornography. It follows the launch of the UK's AI Safety Institute's Inspect, a toolset that also goals to evaluate model capabilities and overall model safety. The US and UK have an ongoing partnership to jointly develop advanced AI model testing, announced on the UK's AI Safety Summit at Bletchley Park in November last 12 months.
Dioptra can also be the product of President Joe Biden's Executive Order (EO) on AI, which (amongst other things) requires NIST to assist test AI systems. The EO also sets standards for the security and security of AI, including requiring corporations that develop models (e.g. Apple) to notify the federal government and share the outcomes of all security tests before they’re made available to the general public.
As we've written before, AI benchmarks are difficult—not least because probably the most sophisticated AI models today are black boxes, with their infrastructure, training data, and other key details kept secret by the businesses that develop them. A report published this month by the Ada Lovelace Institute, a UK-based nonprofit research institute that studies AI, found that assessments alone usually are not enough to find out the real-world security of an AI model. This is partially because current policies allow AI vendors to be selective about which assessments they conduct.
NIST doesn’t claim that Dioptra can completely disarm models. However, the agency suggests that Dioptra can provide insight into what sorts of attacks could impact an AI system's performance and quantify that impact on performance.
A significant limitation, nonetheless, is that Dioptra only works with models that will be downloaded and used locally, comparable to Meta's growing Llama family. Models hidden behind an API, comparable to OpenAI's GPT-4o, are – at the least for now – a no-go.
