AI Security | Cyrus’s Cloud Corner

Secure the whole AI application: its identities, data, retrieval, tools, outputs, and operating environment. Model-level safety controls are one layer, not a replacement for application security.

AI application security illustrated with Identity, Context, Tools, Outputs, Operations — AI application security — an illustrated overview of the topics on this page.

Content reviewed: October 25, 2026 · Practical guide

Map the trust boundaries

An AI system can receive instructions from its developer, requests from users, and content from documents, websites, or tools. Those inputs do not have the same authority. Prompt injection exploits the confusion between data the system should examine and instructions it should obey.

Draw the boundary around every connection: user to application, application to retrieval, model to tool, and tool to business system. For each boundary, identify the calling identity, permitted action, data sensitivity, logging, and failure behavior.

Threats worth designing around

Risk	Practical defensive direction
Prompt injection	Separate untrusted content from instructions; constrain tool behavior and test adversarial documents.
Sensitive information disclosure	Minimize supplied data, enforce document access outside the model, and examine outputs for inappropriate disclosure.
Excessive agency	Give tools narrow functions and permissions; require approval for consequential actions.
Unsafe output handling	Validate generated values and encode output for its destination. Do not execute generated commands or queries automatically.
Supply-chain risk	Review models, packages, connectors, data sources, and update provenance.

Worked example: a hostile document

Illustrative design: a search result contains a paragraph telling an assistant to send confidential files to an external address. The retrieval service may return that passage as document content, but the application must not treat it as authorization. A well-written system prompt can help guide behavior; it is not a hard security boundary.

Constrain the assistant to approved read operations, enforce access checks at the retrieval service, and prevent arbitrary outbound destinations. If sending a message is a legitimate feature, use a separate, scoped tool with explicit recipients and an approval step appropriate to the action.

AI Security topic map: Map the trust boundaries, Threats worth designing around, Worked example: a hostile document, Layer controls across the lifecycle — AI Security — topic overview

Layer controls across the lifecycle

Protect the model endpoint with ordinary cloud identity, network, and secret-management controls. Restrict what goes into context. Validate tool arguments in application code. Log the decision path while minimizing sensitive prompt retention. Use rate limits, timeouts, and spend limits to contain misuse.

Guardrails and content filters can reduce particular risks, but their coverage depends on configuration, supported inputs, and the application. Test the complete system rather than assuming that enabling a filter makes every connected action safe.

A practical security test set

Ask for another user’s documents and check for denial before retrieval.
Include malicious instructions in retrieved content and tool results.
Request an unsupported destination, action, or tool argument.
Try to extract credentials, hidden configuration, or sensitive context.
Check that failures are logged, bounded, and recoverable.

Sources & further reading

Primary references reviewed for this guide. Worked examples and delivery checklists are illustrative design guidance.