Microsoft Announces 'Computer Use' Automation in Copilot Studio

Microsoft has introduced a new AI-powered feature called "Computer Use" for its Copilot Studio platform that allows agents to directly interact with websites and desktop applications using simulated mouse clicks, menu selections, and text inputs.

According to Charles Lamanna, corporate vice president of business and industry for Microsoft Copilot, the feature marks a major leap forward in Microsoft's vision for intelligent automation by allowing Copilot Studio agents to treat graphical user interfaces similarly to how a human user would.

"Computer Use adapts to changes in apps and websites automatically," said Lamanna. "It adjusts in real time using built-in reasoning to fix issues on its own, so work continues without interruption. It is also built on Copilot Studio's robust security measures and governance frameworks, to help ensure compliance with organizational and industry standards."

Key use cases for the new feature include:

  • Automated data entry across systems that lack direct integrations;
  • Market research automation by collecting and organizing information from online sources; and
  • Invoice processing, streamlining finance operations by extracting and transferring data into accounting systems.

Computer Use is hosted on Microsoft infrastructure. The company said this eliminates the need for organizations to provision or manage dedicated RPA environments, reducing both overhead and maintenance requirements.

Microsoft says the new feature also modernizes traditional robotic process automation (RPA) by making it more resilient and accessible. Rather than requiring extensive coding or scripting, users can describe automation tasks using natural language and refine workflows with side-by-side video previews showing how the agent interprets and executes UI actions.

Additional enhancements include full transparency, with the ability to audit every action the agent performs, including screenshots and reasoning steps.

Microsoft plans to showcase Computer Use in greater detail next month at its Build conference.

Microsoft Begins Rolling Out Controversial Recall Feature

In other Microsoft Copilot news, the company recently announced it has begun rolling out a preview of its Recall feature in Windows in the latest Windows 11 preview build.

Microsoft's Recall feature for Copilot+ PCs offers an AI-driven way to search and retrieve past activity across apps, websites, documents, and images by describing what you remember. The feature works by taking opt-in snapshots of your screen throughout the day and requires Windows Hello authentication to ensure only the device owner can access them.

The feature was initially supposed to start rolling out in preview last June. However, after security concerns from the community arose, the feature was yanked. It was then scheduled for release in October, but was yet again pulled when the feature was found to be taking screengrabs of users' credit card information.

Microsoft has made it clear that Recall is an optional feature and can be turned off at any time. It also said the new version now rolling out adheres to the company's commitment to AI and privacy.

"Recall does not share snapshots or associated data with Microsoft or third parties, nor is it shared between different Windows users on the same device. Windows will ask for your permission before saving snapshots. You are always in control, and you can delete snapshots, pause or turn them off at any time. Any future options for the user to share data will require fully informed explicit action by the user," said Microsoft.

About the Author

Chris Paoli (@ChrisPaoli5) is the associate editor for Converge360.

Featured

  • cloud and circuit patterns with AI stamp

    Cloud Management Startup Launches Infrastructure Intelligence Tool

    A new AI-powered infrastructure intelligence tool from cloud management startup env0 aims to turn the fog of sprawling, enterprise-scale deployments into crisp, queryable insight, minus the spreadsheets, scripts, and late-night Slack threads.

  • From Fire TV to Signage Stick: University of Utah's Digital Signage Evolution

    Jake Sorensen, who oversees sponsorship and advertising and Student Media in Auxiliary Business Development at the University of Utah, has navigated the digital signage landscape for nearly 15 years. He was managing hundreds of devices on campus that were incompatible with digital signage requirements and needed a solution that was reliable and lowered labor costs. The Amazon Signage Stick, specifically engineered for digital signage applications, gave him the stability and design functionality the University of Utah needed, along with the assurance of long-term support.

  • geometric pattern of interconnected triangles and hexagons

    Gravyty Merges with AI-Powered Student Engagement Companies Ivy.ai and Ocelot

    Gravyty, a provider of alumni and donor engagement and fundraising solutions, has announced a merger with AI-powered student enrollment and engagement companies Ivy.ai and Ocelot. The combined company will operate under the Gravyty brand.

  • person typing on a touch screen schedule plan calendar

    2025 Tech Tactics in Education Conference Agenda Announced

    Registration is free for this fully virtual May 7 event, focused on "Thriving in the Age of AI" in K-12 and higher education.