The-league

*It's not about any one of us, Tom. It's bigger than that.

From Theory to Practice: Using Copilot Agents for IT Incident Resolution 

Let’s talk about Copilot agents. We’ll skip the theory and go straight to practice. 

I strongly believe in the potential of artificial intelligence, but like many organizations, we have struggled to fully unlock its value. In this post, I’d like to share some of our practical findings and lessons learned. 

 

The Use Case 

We deliberately started with a straightforward and tangible use case: 
improving the quality and speed of IT incident resolution. 

Within our current ITSM toolset (Jira Service Desk), we lack effective capabilities to search historical incidents in a way that helps identify recurring issues or potential solutions. Although we do have an external knowledge base based on phpMyFAQ, there is no integration with Jira Service Desk, which limits its practical usability during incident handling. 

At the same time, we only have a limited number of Microsoft 365 Copilot licenses, as we are still in a proof‑of‑concept phase. To work around this constraint, we decided to create a Copilot agent based on SharePoint data sources. 

 

Data Sources 

We prepared the following source files for the agent: 

  • An export of closed incidents per group, covering a limited period (one year) 
  • An export of OneNote procedures 
  • An export of the complete content of the knowledge base portal 

When doing this, make sure to carefully review the content limitations described in the Microsoft documentation: 
https://learn.microsoft.com/en-gb/microsoft-365/copilot/extensibility/agent-builder-add-knowledge#file-types-and-size-limits 

This approach allows us to make the agent available to a broader audience without requiring an individual M365 Copilot license for each user. 

 

Licensing and Costs 

Although Microsoft’s documentation states that this specific setup should not require additional licensing (as described in https://learn.microsoft.com/en-gb/microsoft-365/copilot/extensibility/prerequisites#agent-capabilities-and-licensing-models), we are still observing costs appearing in our usage‑based billing. This is currently being investigated with Microsoft Support. 

Key Findings and Lessons Learned 

Based on our experiments, here are some practical takeaways: 

  • Experiment with source formats, especially file extensions. In our experience, PDF works best — specifically the Jira “Printable” export in PDF format. 
  • Start small with a limited group of users and create tight feedback loops. Prompting is not a one‑off task; it requires iteration and refinement. 
  • Add specific files first using Copilot Studio instead of immediately pointing the agent to a full SharePoint folder. This helps visualize and control the available source content. 
  • Ask the agent about its sources to validate that the correct information is being used. 
  • Focus on user adoption. Clearly communicate the added value for end users. Initially, engagement was low until we actively emphasized how the agent could support daily work. 

 

What’s Next 

The next phase is to expand this setup to additional teams and to establish a standard framework for future Copilot use cases. 

We are currently exploring several available frameworks and initiatives, including: 

Enjoy — and feel free to share your own experiences. 

gino.dhoker@the-league.be

I am a passionate result driven Workplace consultant. Always looking for a workplace related project with added business value.

,
,