NIST Releases New Guidelines for Securing AI Systems Against Adversarial Attacks
Beyond Traditional Security: Protecting Your AI Assets
The 2025 update to NIST's adversarial machine learning guidelines lays out how AI systems get attacked - and the defenses that hold up.
NIST's 2025 adversarial machine learning report gives organizations a shared vocabulary for how AI systems are attacked - and a practical framework for defending them.
The document, AI 100-2e2025, builds on NIST's earlier 2024 work with an expanded section on generative-AI attacks, a searchable index of attacks and mitigations, and new contributors from the U.K. and U.S. AI Safety Institutes. As AI moves into everyday business operations, understanding these attack types stops being an academic exercise - it becomes part of ordinary risk management.
The Main Ways Attackers Target AI
NIST groups the threats by where in the AI's life they strike.
Attacks hit AI at three points: during training (poisoning), during deployment (evasion), and through queries that extract the model's secrets (privacy attacks).
- Evasion attacks. During deployment, attackers craft "adversarial examples" that trick a model into misclassifying - for instance, subtle stickers on a stop sign that make an autonomous vehicle read it as a speed-limit sign.
- Poisoning attacks. During training, an adversary who controls even a small fraction of a public dataset can insert malicious data that quietly corrupts the model's behavior.
- Privacy attacks. Through queries, attackers reconstruct training data, determine whether specific data was used in training (membership inference), or steal the model's architecture and parameters (model extraction).
What Is New: GenAI-Specific Threats
Generative AI inherits every predictive-AI threat and adds its own.
Generative systems face all the classic attacks plus abuse and prompt injection - manipulating inputs to slip past the model's safety guardrails.
| Threat category | Predictive AI | Generative AI |
|---|---|---|
| Availability | Degrade model performance | Degrade model performance |
| Integrity | Force incorrect predictions | Force incorrect or manipulated output |
| Privacy | Extract training data or the model | Extract training data or the model |
| Abuse | — | Repurpose the system to generate harmful content |
| Prompt injection | — | Craft inputs that bypass safety guardrails |
Two GenAI attacks deserve special attention. Prompt injection manipulates inputs to bypass a model's safety rules, and indirect prompt injection hides those instructions in outside resources - like a web page - that the AI later reads and acts on. Add supply-chain attacks on model files and training pipelines, and it is clear that securing a generative system means watching far more than the prompt box.
Putting AI Into Your Business?
Every AI tool you adopt adds a new attack surface most security programs never accounted for. A free assessment maps where yours is exposed.
Get Your Free Assessment →NIST's Recommended Mitigations
No single fix is enough - defense in depth is the whole point.
NIST is blunt that machine learning lacks the strong security guarantees cryptography enjoys, so protection comes from stacking several imperfect defenses together.
- Adversarial training. Deliberately adding adversarial examples to the training data so the model learns to resist them.
- Randomized smoothing. Transforming a classifier so it is provably resistant to certain kinds of manipulation.
- Training-data sanitization. Cleaning the training set to remove potentially poisoned samples before they shape the model.
- Supply-chain assurance. Verifying model artifacts and confirming the integrity of the data sources that feed training.
- Red teaming. Testing an AI system against these attack types before it goes live, not after something breaks.
The report is realistic about the tradeoffs: security improvements often cost accuracy, performance, or compute, and multimodal models are not automatically more resistant. The only durable answer is to build security into the entire AI lifecycle, from design through deployment, and combine defenses rather than trusting any one of them.
The unsettling part of AI security is that the attack often is not on your network at all - it is on the data the model learned from, or the prompt someone feeds it. NIST's taxonomy matters because you cannot defend against a threat you do not have a name for.
Security for the AI You Actually Use
CinchOps helps Houston-area businesses assess and secure the AI tools they are adopting - against the exact threat categories NIST outlines - as part of everyday managed IT and cybersecurity.
Explore CinchOps cybersecurity →How CinchOps Helps Secure Your Business
CinchOps is a Katy, Texas managed IT services provider serving businesses across the Houston metro, helping organizations adopt AI without opening themselves to adversarial attacks.
- AI security assessments. Evaluating your AI systems against the threat vectors in the NIST guidelines.
- Supply-chain verification. Processes to confirm the integrity of model files and training-data sources.
- Continuous monitoring. Watching for signs of attacks against your AI infrastructure.
- Mitigation guidance. Practical help implementing the defense strategies NIST recommends.
The NIST guidelines give a solid framework for understanding adversarial machine learning; putting it to work is where a partner helps. Contact CinchOps to secure the AI your business depends on.
Frequently Asked Questions
What is adversarial machine learning?
Adversarial machine learning is the study of attacks that target AI and machine-learning systems - and the defenses against them. Rather than breaking into a server, these attacks manipulate the data a model learns from, the inputs it receives, or the queries used against it to make the model behave incorrectly or leak information.
What are the main types of attacks on AI systems?
NIST groups them into evasion attacks (fooling a deployed model with adversarial inputs), poisoning attacks (corrupting training data), and privacy attacks (extracting training data or the model itself). Generative AI adds abuse attacks and prompt injection, which manipulate inputs to bypass safety guardrails.
What is prompt injection?
Prompt injection is a generative-AI attack that crafts inputs to bypass a model's safety rules. Indirect prompt injection hides malicious instructions inside outside resources - like a web page - that the AI later processes, so the harmful instruction reaches the model without the user typing it.
How can organizations defend their AI systems?
NIST recommends combining several defenses: adversarial training, randomized smoothing, training-data sanitization, supply-chain assurance, and red teaming - applied across the whole AI lifecycle. Because no single mitigation is sufficient, defense in depth is essential.
What is NIST AI 100-2e2025?
It is NIST's March 2025 report, "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations." It expands the 2024 version with a dedicated generative-AI section, a searchable index of attacks and mitigations, and contributions from the U.K. and U.S. AI Safety Institutes.
Discover More
Sources
