Version 2025-09-22 By Spencer Brawner

LLM Security: Patterns and Pitfalls

LLM security Instruction isolation Prompt injection Tool security Output validation

TL;DR

LLM applications fail when instructions are not isolated, context is unsanitized, tools are over-privileged, or outputs are trusted blindly. Use instruction isolation, input/output filters, retrieval hardening, tool allow-lists with least privilege, and human-in-the-loop for sensitive actions. Test continuously with reproducible attacks.

Published September 22, 2025

Section 01 // Key Facts

Key Facts

5 facts documented

LLMs follow instructions and can be induced to override guardrails.
[1]
Instruction isolation and strict tool scopes reduce impact.
[1]
Retrieval must sanitize and constrain cross-domain content.
[1]
Output validation prevents unsafe actions and data leakage.
[1]
Regression testing is required after model/config changes.
[2]

Section 02 // Implementation

Implementation Steps

5 steps

01
Isolate system prompts → versioned prompt repo.
02
Sanitize retrieval → allow-list, strip directives.
03
Gate tools → scoped keys, approvals.
04
Validate outputs → regex/semantic checks.
05
Regressions → test suite results.

Section 03 // Glossary

Glossary

6 terms

Instruction isolation

Separation of system instructions from user inputs to prevent override

Semantic check

Validation of output meaning and intent, not just format

Allow-list

Predefined list of permitted inputs, tools, or actions

Least privilege

Principle of granting minimum necessary permissions or capabilities

Regression suite

Collection of tests to detect security or functionality degradation

Directive stripping

Removal of instructions or commands from retrieved content

Section 04 // References

References

2 sources

[1]
NIST AI Risk Management Framework https://www.nist.gov/itl/ai-risk-management-framework
[2]
ISO 42001 AI Management Systems Standard https://www.iso.org/standard/78380.html

Section 05 // Facts

Machine-Readable Facts

3 claims

[
  {
    "id": "f-override",
    "claim": "LLMs can be induced to override intended instructions without isolation.",
    "source": "https://www.nist.gov/itl/ai-risk-management-framework"
  },
  {
    "id": "f-scope",
    "claim": "Tool scopes and least privilege reduce blast radius in LLM apps.",
    "source": "https://www.nist.gov/itl/ai-risk-management-framework"
  },
  {
    "id": "f-regress",
    "claim": "Security regressions occur after model or prompt changes; re-testing is required.",
    "source": "https://www.iso.org/standard/78380.html"
  }
]