Microsoft has unveiled a big upgrade to its Copilot Studio platform with the introduction of “computer use” functionality in an early access research preview.
This new capability enables Copilot Studio agents to interact directly with graphical user interfaces across websites and desktop applications, opening new possibilities for enterprise automation.
Breaking Traditional Automation Barriers
The computer use feature has the potential to change how businesses can approach automation. Unlike traditional methods that require API connections, this new capability allows agents to operate any system with a graphical interface by mimicking human interactions – clicking buttons, navigating menus, and entering data into fields.
Charles Lamanna, Corporate Vice President of Business & Industry Copilot at Microsoft explained:
Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen. This allows agents to handle tasks even when there is no API available to connect to the system directly. If a person can use the app, the agent can too.
A key advantage is the feature’s adaptability to interface changes. The system uses built-in reasoning to adjust in real-time when buttons or screens change, preventing workflow disruptions that plague traditional automation solutions.
Enterprise Applications and Use Cases
The potential enterprise applications span numerous departments and industries:
- Automated Data Entry: Organizations can deploy agents to input large volumes of data from diverse sources into centralized systems, reducing manual effort and minimizing human error.
- Market Research: Marketing teams can automate collection of market data from various online sources, gathering valuable insights without manual intervention.
- Invoice Processing: Finance departments can streamline operations by automatically extracting data from invoices and inputting it into accounting systems, eliminating repetitive tasks and reducing processing errors.
Reimagining Robotic Process Automation
Computer use agents are transforming the RPA landscape by addressing traditional limitations. Unlike conventional RPA tools that can break when interfaces change, these agents can handle dynamic interfaces through intelligent adaptation.
“It responds to changes in real time: When buttons or screens change, the tool keeps working without breaking your flow,” notes Lamanna when describing how the technology overcomes common automation challenges.
Enhanced Accessibility and User Experience
The platform prioritizes ease of use with natural language instructions, eliminating the need for specialized coding skills. Users can describe desired actions conversationally, then test and refine prompts with real-time feedback. This democratizes automation capabilities across organizations, allowing non-technical staff to create functional automations for their specific needs.
The system provides complete visibility with comprehensive activity logging. Makers can review historical computer use actions, including captured screenshots and reasoning steps, providing transparency and auditability throughout the process.
This development comes shortly after Microsoft introduced deep reasoning capabilities for agents, support for model context protocol (MCP), and made agent flows generally available in Copilot Studio. Computer use functionality integrates seamlessly with these earlier innovations, creating a more comprehensive enterprise solution.
The tool supports cross-browser compatibility, working with Edge, Chrome, and Firefox, expanding implementation options for diverse enterprise environments. This flexibility allows businesses to leverage existing technology investments while adding new automation capabilities.
Security and Infrastructure Considerations
The feature builds on Copilot Studio’s existing security framework to help ensure organizational compliance. It runs on Microsoft-hosted infrastructure, meaning organizations don’t need to manage their own servers. Enterprise data remains within Microsoft Cloud boundaries and is not used to train frontier models, potentially reducing deployment time and lowering infrastructure costs.
Microsoft plans to share more about this capability at the upcoming Microsoft Build event in May 2025. Organizations interested in exploring the new computer use feature can fill out a form to participate in the early access preview.
