Microsoft 365 Copilot Pilot
- October 1st, 2024
- in Software News
The first tool under review was Microsoft 365 Copilot, which integrates generative AI into the Microsoft 365 suite. This AI leverages Microsoft Graph technology to tailor outputs based on each user’s data access permissions, adding an extra layer of security. All prompts are processed by a private instance of Microsoft’s large language model (LLM), providing an added assurance of data protection.
For this pilot, OIT engaged 35 staff members across various roles, including content creators, designers, and management, who used the tool for two months. Throughout the first month, short training sessions were offered several times a week to familiarize users with Copilot’s features. Surveys were also conducted at different stages of the pilot to capture feedback on user experience.
The results were mixed. While Copilot offered some helpful functionalities, such as email coaching for sentence restructuring and image discovery, the tool still had limitations. For instance, it often altered email tone in a way that was not always helpful, and written communications generated by Copilot required significant edits. Many users indicated that more training on how to prompt the AI was necessary to maximize its effectiveness.
From a security perspective, Copilot was approved for use without restrictions, making it a safe option for the university. However, the cost—$30 per employee per month—would need to be borne by individual departments. While OIT found some useful features, the overall return on investment (ROI) was hard to quantify, leaving departments to weigh the costs and benefits.