Simplify KQL Queries with an Intelligent Telemetry Agent for D365FO apps Insights

1. Introduction
2. Architecture Overview
3. Step‑by‑Step Guide: Telemetry Agent
   1. Step 1 — Create the Agent
   2. Step 2 – Select the Model and Disable Web Search
   3. Step 3 – Add Azure Monitor Logs Query Tools
   4. Step 4 – Generate Your Knowledge Source File
   5. Step 5 – Add Your Knowledge Source to the Agent
   6. Step 6 – Test Your Agent
4. Why This is Valuable

Introduction

Working with D365 Finance telemetry in Application Insights typically requires writing complex Kusto Query Language (KQL) statements. For many functional analysts, support teams, and operations engineers, this creates friction: they need answers quickly but don’t always master KQL.

To overcome this limitation, you can build an AI-powered agent able to generate, customize, and execute KQL queries in real time, using simple natural‑language instructions.
This agent leverages a knowledge-source JSON file containing reusable KQL templates and dynamically constructs the final query at runtime.

The result:

• no KQL required
• faster root‑cause analysis
• democratized access to telemetry
• improved support efficiency

Architecture Overview

At a high level, the solution uses:

• Copilot Studio to host the natural‑language agent
• Azure Monitor Logs API to execute queries
• KQL template catalog (JSON) to guarantee accuracy and consistency
• Application Insights / Log Analytics Workspace as the telemetry backend

Step‑by‑Step Guide: Telemetry Agent

The solution uses Copilot Studio, Azure Monitor Logs API, KQL template catalog, and Application Insights as telemetry backend.

Step 1 — Create the Agent

Go to copilotstudio.com and create a new agent.

Use an initial prompt such as:

“Create an agent to generate and execute Kusto queries at runtime using Azure Monitor Logs. The agent will use a knowledge source file containing Kusto query templates to build runtime queries.”

Add the detailed instructions shown below.

Full Instructions:

# Purpose

The purpose of this agent is to dynamically generate and execute Kusto queries using Azure Monitor Logs  leveraging predefined Kusto query templates stored in a JSON file as a knowledge source

# General Guidelines

– Maintain a professional and precise tone.

– Ensure queries are syntactically correct and optimized for performance.

– Validate user input before incorporating it into queries.

– Do not expose sensitive information in responses.

# Skills

– Understanding of Kusto Query Language (KQL).

– Ability to retrieve and adapt query templates from the knowledge source.

– Capability to execute queries using Azure Monitor Logs API.

# Step-by-Step Instructions

## 1. Identify User Intent

– Goal: Understand what the user wants to query (e.g., performance metrics, error logs).

– Action: Ask clarifying questions if the request is ambiguous.

– Transition: Once intent is clear, proceed to template selection.

## 2. Select Appropriate Query Template

– Goal: Find a relevant Kusto query template from the knowledge source.

– Action: Search the knowledge source for templates matching the user’s intent.

– Transition: If a template is found, move to customization; if not, inform the user and suggest alternatives.

## 3. Customize Query Template

– Goal: Adapt the template with runtime parameters (e.g., time range, resource name).

– Action: Replace placeholders in the template with user-provided or default values.

– Transition: Validate the final query syntax before execution.

## 4. Execute Query

– Goal: Run the customized query against Azure Monitor Logs.

– Action: Use the Azure Monitor Logs API to execute the query.

– Transition: Once results are retrieved, format them for user readability.

## 5. Present Results

– Goal: Provide clear and actionable results to the user.

– Action: Summarize key findings and offer to export or visualize data if needed.

# Error Handling and Limitations

– If the query fails, provide a clear error message and suggest corrective actions.

– If no data is returned, inform the user and suggest adjusting filters or time ranges.

# Feedback and Iteration

– Ask the user if the results meet their needs.

– Offer to refine the query or run additional queries based on feedback.

# Interaction Example

User: “Show me CPU usage for the last 24 hours.”

Agent: “I will use the performance metrics template for CPU usage. Do you want to filter by a specific resource group?”

# Follow-up and Closing

– After delivering results, ask if the user needs further analysis or wants to save the query for future use.

Step 2 – Select the Model and Disable Web Search

To ensure predictable KQL generation:

• Model: select Claude Sonnet 4.5 (recommended)

Web search: Disable it
→ This ensures the agent uses only your KQL templates and avoids hallucinated KQL from online sources.

Step 3 – Add Azure Monitor Logs Query Tools

In Copilot Studio → Tools, add:

1. Run Query and List Results
2. Run Query and Visualize Results (optional)

Step 4 – Generate Your Knowledge Source File

Use Microsoft Copilot to convert sample KQL dashboard files into a clean JSON template catalog.

You can use Microsoft’s public repository: https://github.com/microsoft/Dynamics-365-FastTrack-FSCM-Telemetry-Samples/tree/main/Dashboards/AzureDataExplorer

Examples include:

• Batch telemetry
• Long-running forms
• SQL issues
• Error lookups
• Session analysis
• AOS performance

Use a prompt like:

“I built an agent to generate and execute Kusto queries at runtime using Azure Monitor Logs. I need a JSON knowledge-source file with KQL templates based on the attached scenario files.”

Copilot will generate a structured JSON file that your agent can leverage. You can then add the other dashboards present in the directory to generate a more global template file.

You can download the template from Copilot to use it as a knowledge source (LLM) for your agent.

Step 5 – Add Your Knowledge Source to the Agent

In Knowledge Sources → upload the generated JSON file containing your KQL model scenarios (batch telemetry, long-term forms, SQL problems, error searches, etc.)

The agent will now be able to:

• identify the right scenario,
• select the best KQL template,
• dynamically construct a production‑ready query.

The agent will use these templates to build queries at run time.

Step 6 – Test Your Agent

Try prompts such as:

• “Show top 5 slow-running forms”
• “Find batch failures in the last 24 hours”
• “Show SQL timeout errors today”
• “List long-running batch jobs”
• “Show errors for sales posting”
• What are the scenarios you can cover for me ?
• How is the batch jobs load on AOS ?
• Show me top 10 running batch jobs ?
• How is my overall system performance in last 2 days ?
• how me overall system health ? 

The agent will:

1. Understand your intent
2. Select the right KQL template
3. Build the final query
4. Execute it
5. Return clear, actionable insights

Why This is Valuable

• No KQL skills needed
• Faster troubleshooting and RCA
• Greatly improves efficiency of support & functional teams
• Makes telemetry accessible to everyone
• Reduces dependency on senior engineers
• Turns Application Insights into a natural-language analytics engine for D365FO