Security leaders need to build security controls for AI-powered products quickly, and in a way that doesn’t stifle innovation. Traditionally, an extensive review process has been used for AI features. An overly long review process for AI features can lead developers to use AI tools that are outside of the approved process. An AI Security Checklist offers a more practical approach. It helps security leaders quickly identify the most pressing threats and determine the minimum safeguards and security controls. The OWASP GenAI LLM Top 10 offers a good starting point. The 2026 Edition identifies the 10 most pressing risks in LLM applications. This guide helps CISOs quickly introduce controls based on the threats outlined in the OWASP GenAI LLM Top 10, while allowing product teams to be as agile and iterative as they want.

Why CISOs Need an AI Security Checklist

A customer service chatbot can expose customer data through manipulated prompts. An AI assistant could perform unauthorized actions using integrated tools. Similarly, malicious data in an RAG pipeline can influence how responses are generated without compromising the application. These threats necessitate that security teams amend their risk management processes.

AI Security Checklist helps CISOs implement consistent security requirements for AI projects. Rather than assessing every AI project manually, organizations can implement risk controls based on the sensitivity, exposure, and impact of the project. The goal is to integrate security into product development as opposed to a separate process.

CISO’s AI Security Checklist: 10 OWASP LLM Top 10 Controls

The following controls align the OWASP GenAI LLM Top 10 (2026) with practical security controls to perform effective AI pentesting. They cover application design, data protection, model integrity, and continuous review.

AI Security Checklist: 10 OWASP LLM Top 10 Controls

1. Address Prompt Injection

A prompt injection attack occurs when malicious instructions are provided to an LLM, causing it to perform unintended actions. These prompts can be provided by end users or generated by AI itself and provided to end users through documents, websites, or other content.

Controls:

  • Treat end user prompts and documents as insecure input.
  • Where feasible, separate system instructions from user input.
  • Validate and filter user input based on context.
  • Restrict the actions an LLM can perform.
  • Test for prompt injection as part of security controls.

It is inadvisable to rely on prompts or guardrails to prevent injection. These can only reduce exposure and do not always ensure that a model will follow prescribed instructions.

2. Avoid Sensitive Information Disclosure

AI processes and generates documentation that contains personal information, business and financial records, and other sensitive data. If not protected, this can be disclosed to unauthorized individuals or included in response templates.

Things to do:

  • Categorize data before it is processed by AI.
  • Use role-based and least-privilege access controls.
  • If fully unrestricted data is not necessary, remove or mask it.
  • Prevent cross-user sessions and/or cross-tenant data access.
  • Monitor prompts and responses for confidential information.

In RAG, access permissions should be enforced at the retrieval layer. This means a user can only retrieve documents they are authorized to access.

3. Limit Agency in AI Systems

Because AI can interact with external applications and services, and be tasked to perform business process management actions, control and permissiveness become a concern. It is a security risk for an AI agent to receive more permissions than required.

Things to do:

  • Limit the actions permitted for an AI agent.
  • Restrict access to tools, APIs, and connected systems with least-privilege access.
  • Require human approval for high-impact, potentially irreversible actions.
  • Limit transactions and establish boundaries for automation.
  • Maintain a log of the decisions made by agents and actions taken.

Agents should not be allowed to perform some activities, such as the preparation of a payment request, but approving and executing the payment should require human approval.

“Securing Organizations Since 2013”

Rated 4.9/5 on Gartner Peer Insights, 4/5 on G2 and 4.5/5 on Trustpilot

Book Your Free Cybersecurity Consultation Today!

*By clicking submit, you agree to our T&C, consent to our privacy policy.

4. Secure the AI Supply Chain

A trustworthy AI application can still be compromised by a malicious or vulnerable third party in its supply chain.

Controls to implement:

  • Identify all models, datasets, and third-party components.
  • Evaluate vendor security, data protection, and incident response processes.
  • Vet dependencies for trust.
  • Evaluate model versions and ensure they have not been tampered with.
  • Review the permissions granted for plug-ins and external integrations.
  • Maintain visibility into model changes because behavioral changes impact application security.

5. Protect Training Data and Models from Poisoning

Data and model poisoning are ways to manipulate training or fine-tuning data. In both cases, an attacker could modify data in such a way that it introduces bias or other undesirable qualities in the resulting AI model.

What to do:

  • Verify the integrity of training and embedding datasets.
  • Trace the ownership, lineage, and history of training and embedding datasets.
  • Restrict the ability to modify training and embedding datasets.
  • Investigate the root cause of unexpected changes in model behaviors.
  • Create and store trusted copies of dataset versions.

Preserving data integrity is equally important for RAG systems. Unauthorized changes to a knowledge base, for example, may potentially lead to unexpected model responses even if the foundation model remains unchanged.

6. Unbounded Consumption

LLM applications can stress computing resources, increase operational costs, and create denial-of-service conditions by consuming too many resources in an unmanaged fashion. Factors that can lead to unmanaged consumption include repeated or long prompts, excessively long or voluminous output, and long-running, resource-intensive agent workflows.

Controls to take:

  • Set limits on requests, tokens, and output.
  • Throttle requests.
  • Set quotas.
  • Identify and investigate atypical request patterns.
  • Establish timeouts for long-running operations.
  • Set cost controls and usage limits.

7. Reduce Misinformation and Unsafe AI Decisions

LLMs are prone to generating fabricated or misleading information. This creates a security and business risk when a company utilizes AI-generated content for making financial or business decisions, interactions with customers, or other crucial activities.

Potential controls:

  • Important AI-generated information should be validated against trusted sources.
  • Wherever appropriate, use retrieval-based models.
  • Set appropriate confidence thresholds, and define rules for escalation.
  • Implement human control for important decisions.
  • Track and prevent generation of incorrect or harmful information.
  • Define clear boundaries for the decisions AI can make on its own.

8. Protect Hidden Context and System Instructions

Sometimes, a system may expose its internal operations when an instruction is interpreted by the model and included in its output. This may contain sensitive information or instructions on how to operate the model.

Potential controls:

  • Sensitive information and secrets (e.g., passwords, keys, tokens) should always be stored outside prompts.
  • Secrets management systems should be used for secrets.
  • Limit the context provided to the model.
  • Monitor outputs for sensitive information.
  • Restrict access to instructions and configurations.

Prompts should not be used for defining the security boundaries of the system. Authorizations should be performed by the application and its downstream services.

9. Vector Databases and Embedding Pipelines

RAG applications utilize vector databases and embeddings to retrieve relevant information. Inadequate access controls, improper indexing, and untrusted data can lead to the disclosure of confidential information or manipulation of the context of the information retrieved.

Controls to implement:

  • Ensure authorization before documents are retrieved.
  • Isolate tenant data and sensitive knowledge repositories.
  • Validate documents prior to indexing.
  • Encrypt and secure access to vector databases.
  • Monitor and investigate patterns of abnormal document retrievals and unauthorized access.

Security teams should test whether malicious documents manipulate the outcomes of a retrieval.

 10. Protect and Verify LLM Outputs

LLM outputs cannot be trusted. Applications should not simply pass LLM output to downstream systems, APIs, or services. Attackers will certainly leverage these trust issues.

Controls to implement:

  • Evaluate, interpret, and sanitize LLM outputs before further processing or presentation.
  • Use parameterized queries for database commands rather than generated text.
  • Restrict code execution and validate tool-call parameters.
  • Assess the validity of structured or schema responses from LLMs.

An AI-generated SQL command should not be executed against a database without authorization and validation. 

Conclusion

Combining AI into security operations provides enormous opportunities for automated detection and response. AI can be seamlessly incorporated into CISOs workflows to identify and remediate prompt injection and other application-related risks to manage cyber risk. Automate security controls, define risk tolerances, and continuously assess AI systems. Create clear ownership for controls to implement at scale while protecting the organization’s applications, data, and business processes.

Cyber Security Squad – Newsletter Signup

FAQs

  1. What is an AI Security Checklist?

    An AI Security Checklist identifies common controls to address security risks posed by AI, including prompt injection, data exposure and access control, as well as supply chain and other related threats.

  2. What is the OWASP LLM Top 10?

    The OWASP LLM Top 10 identifies the major security risks associated with LLM applications and guides developers on how to remediate the risks.

  3. Why should CISOs follow the OWASP LLM Top 10?

    It allows CISOs to identify AI threats, formulate security requirements, decide on threat remediation prioritization, and integrate security into the product development process.

  4. What steps can be taken to avoid prompt injection attacks?

    Organizations must assume that all external content is untrustworthy, separate instructions from user inputs, and restrict access permissions for machine learning models.

  5. How can organizations secure AI agents?

    Least privilege and restricted tool permissions should be applied, and actions should be logged with validation and approval by a human before sensitive tasks are performed.

  6. What is the advantage of AI risk management?

    With AI risk management, organizations can identify, prioritize, and manage security risks in the AI development and deployment process.

  7. What steps can be taken to integrate AI security in SDLC?

    Use automated security tests and integrates security components in the CI/CD pipeline to identify security risks at an early stage of development.

  8. How frequently should AI security controls be assessed?

    AI applications should be monitored for security controls, and access privileges for models should be adjusted based on new integrations, changes to datasets, and permission sets.