Before Lab 5.6 — Anomaly Detection
Before Lab 5.6 — Anomaly Detection
In Labs 5.1–5.5, you worked with known violations — things you could define rules for. Now you will look at unknown threats — things that don't fit any rule and require observation to detect.
What is anomaly detection?
Anomaly detection finds unusual patterns — things that don't fit any rule. Where Azure Policy checks against known violations, anomaly detection looks for deviations from the normal.
For a full overview of rule-based vs statistical anomaly detection and baselining, see the Anomaly Detection theory page. For alert fatigue mitigation, see the Alarm Fatigue and Sentinel Tuning theory page.
Quick reference — Rule-based vs anomaly-based detection
| Approach | Pros | Cons |
|---|---|---|
| Rule-based (Policies, alerts) | Precise, low false positives | Only catches known patterns |
| Anomaly-based (ML, baselines) | Catches unknown patterns | Higher false positives, needs training |
How anomaly detection works in Azure
Azure offers several anomaly detection capabilities:
Defender for Cloud — built-in anomaly detection
Defender for Cloud uses machine learning to detect unusual activity:
- Unusual network connections
- Suspicious sign-in patterns
- Anomalous data access
- Unexpected resource creation
KQL — custom anomaly queries
KQL (Kusto Query Language) lets you write queries that detect anomalies using statistical thresholds (e.g., alert when count > Mean + 3σ).
Log Analytics — baseline comparison
Log Analytics can compare current activity against historical baselines and alert when things deviate.
What you will do in this lab
You will write KQL queries to detect anomalies in your Day 5 environment. Before you start, make sure your data sources are populated:
Important: Enable Azure Activity Log diagnostic settings at two levels — your resource group and each App Service individually. Go to the resource group → Diagnostic settings → Add diagnostic setting → select
AllLogs→ send to your Log Analytics workspace. Then do the same for each App Service (App Service → Diagnostic settings → Add diagnostic setting → AllLogs). Resource group settings capture subscription-level events (policy changes, resource creation), while App Service settings capture resource-level events (deployments, start/stop). Deploy the apps (day-5/deploy-apps.ps1 -StudentPrefix <your_prefix>) and generate demo traffic. Also ensure Key Vault diagnostic settings are enabled (Lab 5.1) so you have audit logs to query. Wait at least 5–10 minutes after generating traffic before querying — Application Insights needs time to ingest the initial requests.
You will write KQL queries for these anomaly patterns:
- Error spike detection — find unusual error rates (a common indicator of an attack or misconfiguration)
- Unusual endpoint access — find endpoints being accessed unusually often (a sign of scanning or enumeration)
- Cross-plane correlation — correlate Azure Activity Log events (management plane) with application errors (data plane)
- Key Vault audit analysis — detect failed access attempts (a sign of unauthorized access)
- Unusual sign-in patterns — find anomalous sign-in patterns such as repeated failed logins or logins from unexpected locations using administrative tools (a sign of potential credential compromise)
For context on what data sources are available, review the following tables in your Log Analytics workspace:
AppExceptions— application error logsAppRequests— HTTP request logs from Application InsightsAzureActivity— Azure management plane activity (requires diagnostic settings at both resource group AND App Service levels for full coverage)SigninLogs— Azure AD sign-in events (requires diagnostic settings on Key Vault)
What to look for in Lab 5.6
Lab 5.6 checklist
Lab focus
In Lab 5.6, you are looking for things that don't fit any rule. The goal is to observe patterns and find deviations — not to enforce specific policies.
Check before starting
You should be able to answer these questions before starting the lab:
- Have I enabled Azure Activity Log diagnostic settings at both the resource group AND App Service levels?
- Have I deployed the apps and verified they are running?
- What is the difference between rule-based and anomaly-based detection?
- Why do anomaly-based detections have more false positives?
- How does Defender for Cloud use machine learning for anomaly detection?