Step-by-step security checklists for the most common AI security tasks. Each playbook distils field experience into the minimum viable set of controls — what to do, in what order, and what to watch for.
Six analytic rules and a dedicated workbook for Copilot telemetry, contributed to the official Microsoft Sentinel GitHub repository by Samik Roy (May 2026). Deploy as a single solution from Sentinel Content Hub → Microsoft Copilot solution. Requires CopilotActivity table ingestion via the Copilot Data Connector.
Step 1: Ensure CopilotActivity logs are ingested
→ Sentinel Content Hub → Copilot Data Connector (Public Preview Feb 2026)
→ Requires Global/Security Administrator role
Step 2: Deploy the Microsoft Copilot solution
→ Microsoft Sentinel → Content Hub → search "Microsoft Copilot" → Install
→ Includes: workbook + 6 analytic rules + hunting queries in one solution
GitHub source:
Analytic rules: Azure-Sentinel/Solutions/Microsoft Copilot/Analytic Rules
Hunting queries: Azure-Sentinel/Solutions/Microsoft Copilot/Hunting Queries
Workbook: Azure-Sentinel/Solutions/Microsoft Copilot/Workbooks/MicrosoftCopilotActivityMonitoring.json
Step 3: Open workbook and validate data against your environment
Step 4: Enable and tune analytic rules to your policy and risk appetiteMicrosoft has shipped an Agent 365 data connector for Sentinel that ingests agent telemetry into the Sentinel data lake from three sources at once: Agent 365, Microsoft Foundry, and Copilot. This is the unified connector you want for cross-platform agent investigation — it replaces the need to wire three separate sources.
Step 1: In Sentinel → Content Hub → search "Agent 365"
Step 2: Install the Agent 365 data connector
Step 3: Configure data ingestion to the Sentinel data lake
Step 4: Use the Sentinel MCP server interface to query in natural language
or write KQL directly against the new tables
⚠ If this connector is enabled and then deactivated, hunting/graph/MCP
workflows that depended on its data will stop functioning.When you enable "Continuously detect managed devices" in Agent 365 Shadow AI, Intune automatically creates a Properties catalog profile called A365 - Monitor OpenClaw. It uses the new Local AI Agent Settings Catalog node, runs via the Intune Management Extension (IME), and refreshes every 24 hours. Read-only — safe to deploy. Source: Derk van der Woude, May 2026.
Agent Name — canonical identifier for the agent typeAgent Version — version string of the installed agentHost Process — parent process executing the agentInstall Location — filesystem path of the installationInstall Scope — per-user vs per-machineInstall Scope Platform User ID — Windows SID of installing userInstall Scope User ID — Entra ID user identifier (UPN)Local AI Agent Execution Context — user / elevated / SYSTEM ⚠️ SYSTEM = high risk// Requires Intune device inventory data in Log Analytics
// IntuneDeviceCompliancePolicies or custom inventory table from IME
// Filter for elevated/SYSTEM execution context
IntuneDevices
| where Properties has "Local AI Agent Execution Context"
| extend AgentName = extract("AgentName:([^,]+)", 1, Properties)
| extend ExecContext = extract("ExecutionContext:([^,]+)", 1, Properties)
| extend InstallUser = extract("InstallScopeUserID:([^,]+)", 1, Properties)
| where ExecContext in ("elevated","SYSTEM")
| project DeviceName, AgentName, ExecContext, InstallUser, LastSync
| order by ExecContext desc, LastSync descThe CloudAppEvents table captures Copilot and agent activity from the M365 Unified Audit Log via Defender for Cloud Apps. Requires: Settings → Cloud Apps → App connectors → M365 activities enabled. Metadata only — no prompt content.
CloudAppEvents
| where Timestamp > ago(7d)
| where ActionType startswith "CopilotAgent" or ActionType startswith "UpdateCopilot"
| project Timestamp, ActionType, AccountDisplayName,
AgentName = tostring(RawEventData.CopilotAgentName),
ChangeDetail = tostring(RawEventData)
| order by Timestamp descCloudAppEvents
| where Timestamp > ago(7d)
| where ActionType startswith "UpdateCopilotAgent"
| project CopilotActionTime = Timestamp,
AdminAccount = AccountDisplayName,
AgentName = tostring(RawEventData.CopilotAgentName),
Action = ActionType
| join kind=inner (
EmailEvents
| where Timestamp > ago(7d)
| where Subject has_any ("confidential","restricted","sensitive")
) on $left.AdminAccount == $right.SenderFromAddress
| project CopilotActionTime, AdminAccount, AgentName, Action,
EmailTime = Timestamp1, EmailSubject = SubjectCloudAppEvents
| where Timestamp > ago(30d)
| where ActionType == "CopilotForSecurityTrigger"
| summarize TriggerCount = count(), UniqueUsers = dcount(AccountObjectId)
by ActionType, Application, bin(Timestamp, 1d)
| order by Timestamp descIdentifies which AI model each Copilot Studio agent is using by extracting modelNameHint from RawAgentInfo. Flags EU Data Boundary (EUDB) compliance status per agent — critical for EU organisations where Anthropic-hosted models process data outside the EUDB. Source: Blue161616/Agent-Identity.
AIAgentsInfo
| summarize arg_max(Timestamp, *) by AIAgentId
| where Platform == "Copilot Studio" and AgentStatus != "Deleted"
| extend ModelNameHint = extract(@"modelNameHint:\s*([A-Za-z0-9_\-\.]+)", 1, RawAgentInfo)
| extend HintLower = tolower(ModelNameHint)
| extend Provider = case(
HintLower startswith "sonnet" or HintLower startswith "haiku" or HintLower startswith "opus", "Anthropic",
HintLower startswith "gpt" or HintLower startswith "o1" or HintLower startswith "o3", "OpenAI (Microsoft-hosted)",
isempty(ModelNameHint), "Environment default",
"Other"
)
| extend EUDB_Status = case(
Provider == "Anthropic", "OUT OF EUDB — cross-geo processing",
Provider == "OpenAI (Microsoft-hosted)", "In EUDB (if environment is in EU)",
Provider == "Environment default", "Depends on tenant default — verify",
"Verify"
)
| project AIAgentName, Provider, ModelNameHint, EUDB_Status,
EnvironmentId, CreatorAccountUpn, OwnerAccountUpns, LastModifiedTime
| sort by Provider asc, AIAgentName ascRegistrySource column distinguishing agent origin: "A365" (Agent 365 registered) vs "PowerPlatform" (Copilot Studio via Power Platform connector). Use these filters to target the right agent population. The four queries below cover the highest-risk patterns for A365-registered agents.RegistrySource == "PowerPlatform" in the same queries. The no-auth query from Step 1 remains the most important for Copilot Studio — Query 8a–8d are specifically designed for A365-registered agents.pip install pyrit # Required env vars (for Azure OpenAI target + scorer) export AZURE_OPENAI_API_KEY="your-key" export AZURE_OPENAI_ENDPOINT="https://your-instance.openai.azure.com/" export AZURE_OPENAI_DEPLOYMENT="gpt-4o"
Config-driven — switch strategies, datasets, or thresholds with no code changes. Point target at your agent entry point.
# scan.yaml
scan:
name: "my-agent-security-scan"
target:
module: "my_agent.main" # Your agent module
function: "handle_prompt" # Your agent entry function
strategies:
- name: "prompt_sending" # Single-turn baseline
- name: "crescendo" # Gradual escalation
max_turns: 5
- name: "tap" # Tree of Attacks with Pruning
converters: # Stack — applied in order
- name: "Base64Converter"
- name: "LeetspeakConverter"
- name: "TranslationConverter"
datasets:
- "airt_illegal" # Microsoft AI Red Team dataset
- "harmbench" # Harm evaluation benchmark
- "advbench" # Adversarial benchmark
- path: "datasets/custom.yaml" # Your domain-specific prompts
gate:
fail_on:
critical_findings: 0 # Zero tolerance for critical
high_findings: 3
max_score: 0.3 # Defense rate must exceed 70%Run in sequence — if plain prompts pass, layer evasion. Each phase builds on the previous.
Phase 1: Plain prompts → Baseline — establishes what passes without evasion → Catches basic prompt injection and policy violations Phase 2: Encoded prompts → Base64, ROT13, Leetspeak, Unicode confusables → Tests whether your agent/guardrails decode before evaluating Phase 3: Semantic attacks → LLM-powered rephrasing, translation, multimodal injection → Converters stack: translate → Base64 → embed in image Phase 4: Multi-turn dialogue attacks → CrescendoAttack: gradual escalation over 5–10 turns → TreeOfAttacksWithPruning (TAP): branching attack trees → Tests whether context accumulation bypasses initial guardrails
Map findings to OWASP LLM Top 10 (2025) for structured risk reporting. Turns PyRIT output into a risk register your security team understands.
LLM01 Prompt Injection → PromptSendingAttack + injection datasets (airt_illegal) LLM02 Sensitive Info → Data exfiltration datasets + PII scorers LLM06 Excessive Agency → Tool-calling attack datasets (advbench) LLM07 System Prompt Leakage → System prompt extraction datasets LLM10 Unbounded Consumption → High-volume automated attack patterns
Integrate into your pipeline. Exit code 0 = pass (deploy), exit code 1 = fail (block). No custom actions needed.
# GitHub Actions example
jobs:
security-scan:
steps:
- name: Run AI security scan
run: |
pip install pyrit
python scanner.py --config scan.yaml --output reports/
- name: Evaluate release gate
run: |
python gate.py --report reports/scan_results.json
# Exit 1 blocks deployment automatically
# When to run:
# Every merge to main: Quick scan only (phases 1–2, ~10 min)
# Pre-release branch: Full scan (all 4 phases, architect approval)
# Weekly scheduled: Full scan across full agent estate