azurebackup: Fix KQL classification order and update bug tracker in telemetry skill (#2659)

* azurebackup: Fix KQL classification order and update bug tracker in telemetry skill

The KQL case() expression in kql-queries.md checked StatusCode 400-499
before exception type, which masked MCP tool bugs (ArgumentNullException,
ArgumentException, FormatException) as Customer errors when HandleException
mapped them to HTTP 400. This caused the weekly telemetry report to
undercount MCP bugs by ~33x (1 vs 34 in the May 11-17 report).

Changes:
- kql-queries.md: Reorder case() in queries 1, 2, and 4 to check MCP bug
  exception types (FormatException, ArgumentNullException, ArgumentException,
  InvalidOperationException) BEFORE StatusCode 400-499
- SKILL.md: Update decision tree to match the classification table (check
  exception type before status code), add explanatory note about
  HandleException mapping ArgumentNullException to HTTP 400
- SKILL.md: Update known bug table with current state:
  - BUG-1: Mark #2518 as partial fix, add #2621 as defense-in-depth
  - BUG-3: Mark #2518 as partial, add #2621 relaxed regex
  - Add NEW-2 (job_get FormatException in DPP SDK, azure-sdk-for-net#59306)
  - Update notes about incomplete #2518 fixes confirmed on beta.8/beta.9

* Address PR review: fix decision tree nesting and use 'pending' for unreleased versions

- Add missing NO branch under StatusCode check in decision tree for
  proper nesting (Copilot review comment 1)
- Replace 'beta.11 (pending)' with 'pending' in bug tracker Release
  column to avoid presuming version number (Copilot review comment 2)

Author: shrja-ms

tools/Azure.Mcp.Tools.AzureBackup/skills/azurebackup-telemetry-report/SKILL.md

@@ -152,20 +152,26 @@ For the Outlook version, apply these rules:
 
 ## Error Classification Decision Tree
 
+> **Important:** MCP bug exception types must be checked **before** StatusCode.
+> The `HandleException` base class maps `ArgumentNullException` (inherits `ArgumentException`)
+> to HTTP 400, which would incorrectly classify MCP bugs as "Customer" errors if StatusCode
+> is checked first.
+
 ```
 Is success == true?
   └─ YES → "Success"
   └─ NO →
-      Is StatusCode 400/403/404?
-        └─ YES → "Customer (4xx)"
+      Is ExceptionType FormatException / ArgumentNullException / ArgumentException / InvalidOperationException?
+        └─ YES → "MCP Tool Bug"
         └─ NO →
-            Is ExceptionType Azure.RequestFailedException with 5xx?
-              └─ YES → "Azure Service"
-            Is ExceptionType System.AggregateException?
-              └─ YES → "Azure Service"
-            Is ExceptionType FormatException / ArgumentNullException / ArgumentException / InvalidOperationException?
-              └─ YES → "MCP Tool Bug"
-            Otherwise → "Unknown" (investigate)
+            Is StatusCode 400/403/404?
+              └─ YES → "Customer (4xx)"
+              └─ NO →
+                  Is ExceptionType Azure.RequestFailedException with 5xx?
+                    └─ YES → "Azure Service"
+                  Is ExceptionType System.AggregateException?
+                    └─ YES → "Azure Service"
+                  Otherwise → "Unknown" (investigate)
 ```
 
 ## Known Bug IDs
@@ -174,18 +180,21 @@ These are the bugs triaged from the original telemetry analysis (April 2026):
 
 | Bug | Tool | Description | Fix PR | Release |
 |-----|------|-------------|--------|---------|
-| BUG-1 | backup_status | ArgumentNullException on null ResourceType | #2518 (code included, not in PR title) | beta.7 |
+| BUG-1 | backup_status, protecteditem_get | ArgumentNullException on null ResourceType in MapArmResourceTypeToBackupDataSourceType | #2518 (partial), #2621 (defense-in-depth) | beta.7 (partial), pending |
 | BUG-2 | protecteditem_protect | VM pre-discovery loop timeout | #2470 | beta.6 |
-| BUG-3 | protectableitem_list | Workload normalization ArgumentException | #2518 | beta.7 |
+| BUG-3 | protectableitem_list, find-unprotected | Workload normalization ArgumentException in ValidateAndParseResourceTypeFilter | #2518 (partial), #2621 (relaxed regex) | beta.7 (partial), pending |
 | BUG-4 | soft-delete | Deprecated BackupResourceVaultConfig API | #2518 | beta.7 |
 | BUG-5 | enable-crr | Deprecated BackupResourceConfig API | #2518 | beta.7 |
 | BUG-6 | vault_get | FormatException on subscription name (non-GUID) | #2518 | beta.7 |
 | BUG-7 | protecteditem_protect | DPP vault missing managed identity | #2470 | beta.6 |
 | BUG-8 | recoverypoint_get | Null container in resolution | #2518 | beta.7 |
 | NEW-1 | vault_get | AggregateException — pre-existing auth race (Azure Service, not MCP bug) | Not filed | — |
+| NEW-2 | job_get | FormatException in DPP SDK — XmlConvert.ToTimeSpan fails during DataProtectionBackupJobProperties.Deserialize (Azure SDK bug, not MCP code) | #2621 (workaround catch), [azure-sdk#59306](https://github.com/Azure/azure-sdk-for-net/issues/59306) (upstream) | pending |
 
 > **Note:** PR #2518 title says "Fix 5 telemetry-triaged bugs (BUG-3,4,5,6,8)" but the
-> actual merged code also includes the BUG-1 fix. Always verify the diff, not just the title.
+> actual merged code also includes the BUG-1 fix. However, the #2518 fixes for BUG-1 and
+> BUG-3 were **incomplete** — confirmed by telemetry on beta.8/beta.9. PR #2621 adds
+> defense-in-depth fixes for both. Always verify the diff, not just the title.
 
 When new errors appear, assign the next sequential ID (NEW-2, NEW-3, etc.) and add to this table.
 

tools/Azure.Mcp.Tools.AzureBackup/skills/azurebackup-telemetry-report/references/kql-queries.md

@@ -24,11 +24,11 @@ getAzureMcpEvents_ToolCalls(ago(7d), now())
 | extend StatusCode = toint(extract(@"StatusCode.*?(\d+)", 1, ExMsg))
 | extend ErrorCategory = case(
     success == true, "Success",
+    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     StatusCode >= 400 and StatusCode < 500, "Customer",
     ExType == "System.Collections.Generic.KeyNotFoundException", "Customer",
     ExType == "Azure.RequestFailedException" and StatusCode >= 500, "AzureService",
     ExType == "System.AggregateException", "AzureService",
-    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     "Unknown")
 | summarize
     Total = count(),
@@ -62,11 +62,11 @@ getAzureMcpEvents_ToolCalls(TwoWeeksAgo, now())
 | extend StatusCode = toint(extract(@"StatusCode.*?(\d+)", 1, ExMsg))
 | extend ErrorCategory = case(
     success == true, "Success",
+    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     StatusCode >= 400 and StatusCode < 500, "Customer",
     ExType == "System.Collections.Generic.KeyNotFoundException", "Customer",
     ExType == "Azure.RequestFailedException" and StatusCode >= 500, "AzureService",
     ExType == "System.AggregateException", "AzureService",
-    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     "Unknown")
 | summarize
     Total = count(),
@@ -111,11 +111,11 @@ getAzureMcpEvents_ToolCalls(ago(7d), now())
     ExType = tostring(customDimensions["exception.type"])
 | extend StatusCode = toint(extract(@"StatusCode.*?(\d+)", 1, ExMsg))
 | extend ErrorCategory = case(
+    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     StatusCode >= 400 and StatusCode < 500, "Customer",
     ExType == "System.Collections.Generic.KeyNotFoundException", "Customer",
     ExType == "Azure.RequestFailedException" and StatusCode >= 500, "AzureService",
     ExType == "System.AggregateException", "AzureService",
-    ExType in ("System.FormatException", "System.ArgumentNullException", "System.ArgumentException", "System.InvalidOperationException"), "McpToolBug",
     "Unknown")
 | summarize Count = count()
     by ToolName, StatusCode, ErrorCategory, ExType