Day 4 Challenge — Monitoring Readiness Review

During Day 4, you enabled monitoring across the application, infrastructure, cloud platform, and deployment pipeline.

You configured application telemetry with OpenTelemetry and Application Insights. You enabled diagnostic settings on App Service and Key Vault. You connected Azure Activity Logs to Microsoft Sentinel. You created analytics rules for selected security scenarios. You also configured Azure DevOps audit streaming to monitor pipeline activity.

The monitoring stack is now operational. Data is flowing into Log Analytics and Sentinel.

However, operational does not mean production-ready.

In this challenge, you will assess whether the current monitoring setup provides enough visibility to support real operational and security work. You will build a monitoring workbook, review telemetry coverage, design a detection rule based on observed behaviour, validate the alert flow, and identify remaining blind spots.

The focus is not on creating the “perfect” dashboard. The focus is on making defensible monitoring decisions:

• What needs to be visible during normal operations?
• Which events are worth alerting on?
• How do you know the monitoring stack itself is working?
• What gaps would prevent effective incident response?
• What would need to change before this setup could support a production environment?

---

Scenario

You are part of a cloud/security team preparing this environment for a production readiness review.

The application consists of multiple services deployed on Azure. Some components are public-facing, while others should only be accessed indirectly. Sensitive configuration is stored in Key Vault. Deployment activity is handled through Azure DevOps.

Your task is to evaluate the monitoring setup from an operational and security perspective.

At the end of this challenge, you should be able to explain:

1. What the current monitoring stack can detect.
2. Which data sources are active and which are missing.
3. Which dashboard views would help an operations or security team.
4. Which alert you would promote to production and why.
5. Which monitoring gaps still create risk.

---

Deliverables

By the end of the challenge, produce the following:

1. A Sentinel workbook called "Day 4 Monitoring Readiness Dashboard"
2. A coverage check showing which telemetry sources are active
3. One custom analytics rule based on observed behaviour
4. Evidence that the alert flow works end-to-end
5. A short monitoring readiness assessment

Your final assessment should answer:

Would you consider this monitoring setup production-ready?
If not, what are the most important improvements?

---

Challenge A — Build an Operational Monitoring Workbook

Create a custom workbook in Microsoft Sentinel called:

Day 4 Monitoring Readiness Dashboard

This workbook should provide a practical operational view for someone responsible for keeping the environment healthy and secure.

Before creating tiles, define the questions your dashboard should answer.

For example:

• Is the application currently healthy?
• Which services are receiving traffic?
• Are error rates or response times increasing?
• Are internal services being accessed directly?
• Are sensitive resources such as Key Vault being used as expected?
• Have any incidents been created?
• Is telemetry still arriving from the expected sources?

Required dashboard sections

Create at least four workbook sections or tiles.

Each section should use a KQL query and should support a real operational or security decision.

---

Tile 1 — Application Health Overview

Create a tile that gives a quick overview of application health.

Include one or more of the following:

• Request volume by service
• HTTP status code distribution
• 4xx and 5xx error trends
• Average or percentile response time
• Failed requests over time

Example questions this tile should answer:

• Is the application responding successfully?
• Are errors concentrated in one service?
• Is performance degrading?
• Is the current behaviour different from what you expect?

After creating the tile, document:

• Which table(s) you used
• What time range you selected
• What would indicate an unhealthy state
• What follow-up action an engineer should take if the tile looks abnormal

---

Tile 2 — Public vs Internal Access Patterns

Create a tile that helps distinguish expected public access from suspicious or unwanted access.

Include one or more of the following:

• Requests to public endpoints
• Requests to internal endpoints
• Top client IP addresses
• Direct access attempts to services that should not be public
• Unexpected traffic patterns

Example questions this tile should answer:

• Are internal services exposed or being accessed directly?
• Which IP addresses are generating the most traffic?
• Are there requests that bypass the intended application flow?
• Is the network segmentation working as expected?

After creating the tile, document:

• Which access patterns are expected
• Which access patterns would be suspicious
• Whether additional filtering is needed to reduce noise

---

Tile 3 — Sensitive Resource Activity

Create a tile focused on sensitive resource usage, especially Key Vault.

Include one or more of the following:

• Key Vault operations over time
• Failed Key Vault operations
• Caller identity, where available
• Unusual operation types
• Access outside expected test activity

Example questions this tile should answer:

• Is Key Vault being accessed?
• Are there failed or unexpected access attempts?
• Can you identify which identity performed an operation?
• Would this data be useful during an incident investigation?

After creating the tile, document:

• Which Key Vault events are normal in this lab environment
• Which events would be suspicious in production
• What context is still missing from the logs

---

Tile 4 — Sentinel Incident and Alert Overview

Create a tile that gives an overview of Sentinel incidents and alerts.

Include one or more of the following:

• Incidents by severity
• Incidents over time
• Open vs closed incidents
• Most frequently triggered analytics rules
• Incidents mapped to MITRE ATT&CK tactics

Example questions this tile should answer:

• Are alerts being generated?
• Which rules are creating incidents?
• Are incidents mostly informational, medium, or high severity?
• Is the environment noisy?

After creating the tile, document:

• Which alerts are useful
• Which alerts may cause fatigue
• Which alerts would need tuning before production use

---

Optional Tile 5 — Monitoring Coverage Status

If useful, add a dashboard tile showing whether expected data sources have reported data recently.

This tile should help answer:

• Is monitoring still working?
• Which sources have gone silent?
• Is “no data” expected, or does it indicate a configuration issue?

This tile overlaps with Challenge B, but it is valuable in a professional dashboard because monitoring systems also need to be monitored.

---

Workbook Review

After building the workbook, review it as if you were handing it over to an operations or SOC team.

Answer:

1. Which operational questions does the workbook answer well?
2. Which security questions does it answer well?
3. Which questions remain unanswered?
4. Which tiles would be useful during a real incident?
5. Which tiles are mostly educational and would need to be redesigned for production?

---

Challenge B — Monitoring Coverage Review

A monitoring setup is only useful if the expected data sources are actually sending data.

In this challenge, you will verify whether the configured data sources have produced logs in the last hour.

This is a monitoring-of-monitoring check.

Expected data sources

Review the expected telemetry sources for the environment.

Area	Expected Table	What It Shows	Possible Issue if Missing
Application telemetry	AppRequests	SDK-generated request telemetry	SDK not initialised or app not receiving traffic
App Service HTTP logs	AzureDiagnostics	Platform HTTP logs	Diagnostic setting not enabled
Key Vault audit events	AzureDiagnostics	Secret access and Key Vault operations	Diagnostic setting missing or no recent Key Vault activity
Azure Activity Log	AzureActivity	Management-plane operations	Sentinel connector not enabled
Azure DevOps audit logs	AzureDevOpsAuditing	Pipeline and organisation audit events	Audit streaming not configured
Sentinel alerts/incidents	Sentinel tables	Detection and incident activity	Analytics rules not enabled or not triggered

Adjust the list if your environment contains additional sources.

---

Create a coverage check query

Start from the following structure and extend it for all expected sources:

let sdk_requests = AppRequests
| where TimeGenerated > ago(1h)
| count;

let appservice_http = AzureDiagnostics
| where TimeGenerated > ago(1h)
| where Category == "AppServiceHTTPLogs"
| count;

let keyvault_audit = AzureDiagnostics
| where TimeGenerated > ago(1h)
| where Category == "AuditEvent"
| count;

let azure_activity = AzureActivity
| where TimeGenerated > ago(1h)
| count;

let devops_audit = AzureDevOpsAuditing
| where TimeGenerated > ago(1h)
| count;

print
    SDKRequests = toscalar(sdk_requests),
    AppServiceHTTPLogs = toscalar(appservice_http),
    KeyVaultAuditEvents = toscalar(keyvault_audit),
    AzureActivity = toscalar(azure_activity),
    AzureDevOpsAudit = toscalar(devops_audit)