After Lab 4.2 — Recap
After Lab 4.2 — Recap
You verified that the data pipeline works by querying platform-level logs in Log Analytics. Diagnostic settings were deployed automatically via Terraform in Lab 4.1 — in this lab you confirmed they are working correctly and explored the available log tables.
What You Should Understand
In the lab, you enabled both telemetry paths on the same set of resources:
- OpenTelemetry-based SDK (
@azure/monitor-opentelemetry, embedded in your code) captured application-level telemetry — custom log messages, dependency call details, and exception stack traces. - Diagnostic Settings (configured on the resource) captured platform-level telemetry — HTTP requests before they reached your code, console output, and audit events from Azure resources.
The SDK tells you what your code did. Diagnostic settings tell you what Azure did with your request. An investigation often needs both — for example, if a request causes an error in your code, the SDK may produce no useful log, but diagnostic settings still captured the HTTP request at the platform level.
For the detailed breakdown of SDK vs Diagnostic Settings, see Application Insights Fundamentals.
What We Observed in the Lab
| Resource | Log Category | Table You Queried | What You Saw |
|---|---|---|---|
| Public backend App Service | AppServiceHTTPLogs | AzureDiagnostics (Category == "AppServiceHTTPLogs") | HTTP request entries with httpStatusCode, urlPath, clientIp |
| Key Vault | AuditEvent | AzureDiagnostics (Category == "AuditEvent") | Audit entries showing who accessed which secret/key and when |
| Storage account | StorageRead/Write/Transaction | AzureDiagnostics (Category has_any(...)) | Table/Blob/Queue operations and utilization metrics |
| OpenTelemetry-based SDK | Automatic (no config needed) | AppRequests, AppDependencies, AppTraces | SDK-instrumented telemetry from your Node.js app |
Why You Need Both SDK and Platform Logs
For more detail, see Application Insights Fundamentals.
Reflection Questions
If the OpenTelemetry-based SDK already captures HTTP requests, why do we also need App Service HTTP logs from diagnostic settings?
The SDK captures requests after your code processes them. Diagnostic settings capture the HTTP request before it reaches your code, including clientIp and UserAgent even when the application crashes.
Security angle: An attacker may send a request that causes your application to crash. The SDK produces no log. Diagnostic settings still show the attack at the platform level.
The diagnostic settings data takes longer to appear in Log Analytics than the SDK data. Why is this?
Architecture difference:
- SDK: The OpenTelemetry-based SDK sends telemetry directly from your application process to the Azure Monitor endpoint. It is an in-process operation with minimal overhead.
- Diagnostic Settings: Azure's platform service collects the log, buffers it, and then ships it to Log Analytics. This involves Azure's internal log routing infrastructure, which adds latency.
What you observed in the lab: Even though diagnostic settings were deployed via Terraform in Lab 4.1, logs still took time to appear after generating traffic. You needed to wait and retry with queries like AzureDiagnostics \| top 10 by TimeGenerated desc.
Practical impact: During incident investigation, SDK data appears first. Diagnostic settings data arrives later. Plan your queries with this timing difference in mind.
Why does the daily cap matter, and what happens when it is reached?
The daily cap protects your subscription from runaway costs. When the cap is reached:
- New log entries are rejected (ingestion stops)
- Log Analytics stops accepting new data
- Existing data (already stored) is not affected
- Sentinel analytics rules cannot query rejected data
This is why the cap must be set BEFORE enabling monitoring — once logs start flowing without a cap, costs can accumulate faster than you realize. In the lab, 1 GB/day is sufficient. In production, you would set the cap based on your expected data volume plus a safety margin.
If you need to detect Key Vault secret access, which table do you query and why can't the OpenTelemetry-based SDK help?
Query the AzureDiagnostics table with Category == "AuditEvent".
The OpenTelemetry-based SDK cannot see Key Vault access because:
- The SDK runs inside your application — it only sees what your application logs
- Key Vault access happens at the infrastructure level, not at the application level
- The SDK has no built-in awareness of Key Vault operations
Only diagnostic settings (or Azure Activity Log) capture Key Vault audit events. This is a fundamental limitation of application-level telemetry.
Common Issues
| Issue | Symptom | Cause | Fix |
|---|---|---|---|
No logs appear in AzureDiagnostics | Query returns 0 rows | Diagnostic settings take 5-15 minutes to start routing, or traffic hasn't been generated recently | Wait and retry; generate traffic to the App Service to trigger log entries |
| Wrong category name in KQL query | Query returns 0 rows | Category name mismatch (e.g., KeyVaultAuditEvents vs AuditEvent) | Verify the exact category name in the diagnostic settings configuration |
| Daily cap reached too quickly | Data stops flowing, queries return incomplete results | Verbose logging or high traffic exceeds 1 GB cap | Review logging level, reduce debug output, increase cap if needed |
| Key Vault audit events not appearing | AzureDiagnostics (Category == "AuditEvent") has no entries | Diagnostic settings enabled but no Key Vault operations performed | Verify the diagnostic setting is enabled and perform a Key Vault operation (e.g., read a secret) |
| SDK data and platform data have different timestamps | Same request appears at different times in different tables | Different collection paths and latencies | Account for timing difference when correlating data; use correlation IDs |
Bridge to Lab 4.3
You have verified that logs flow from your application to Log Analytics through both SDK and platform-level paths. In the next lab, you will trace individual requests through the full stack — from the frontend, through the public backend, to the internal backend — using operation_Id in KQL queries. This is the foundation for security investigations.