Episode Details
Back to Episodes
Control my Power App with Copilot Studio: how “Computer Use” turns agents into real desktop automation
Season 1
Published 5 months, 2 weeks ago
Description
(00:00:00) Introducing Copilot Studio's New "Computer Use" Feature
(00:01:22) The Power of Direct Computer Interaction
(00:03:19) Setting Up Computer Use: A Step-by-Step Guide
(00:06:16) Watching the AI Learn: A Fascinating but Flawed Process
(00:09:49) The Governance Catch: Balancing Autonomy and Control
(00:15:02) Building a Responsible AI Workforce
(00:20:16) Upcoming Deep Dives and Subscription Call
In this episode of M365.fm, Mirko Peters explores the new Computer Use feature in Copilot Studio and shows how Copilot can now control a real Windows desktop to open your Power App, click buttons, type into fields, and submit forms—like a suspiciously obedient digital intern instead of just a chat assistant. He explains how this vision‑driven desktopautomation works: the agent sees your screen, reasons about menus and controls in real time, and decides where to move the mouse and what to type, turning legacy Power Apps, intranet portals, and non‑API‑enabled tools into targets for true uiautomation.
Mirko walks through the full setup without breaking anything: Windows 10/11 Pro, installing Power Automate Desktop with machineruntime, registering the machine in Power Automate, and enabling it for Computer Use so Copilot Studio can securely reach the device. You will learn which accounts to use, how environment binding works, why Windows Home is excluded, and what to check in the Monitor → Machines view before you ever let an agent touch production desktops. He then shows what happens when the agent actually runs: live screen streaming, reasoning logs, mis‑clicks on date pickers, and how the model recovers by changing strategies—from clicking calendars to simply typing the correct date—demonstrating real reasoningai instead of brittle scripts.
The episode also covers governance and guardrails once agents can move the cursor for you. Mirko explains how to keep this power on dedicated machines, segment experiments from production, and combine Computer Use with standard powerautomate flows so you handle back‑end APIs where possible and only fall back to UI control where necessary. You will hear practical rules for corporate environments: treat Computer Use like RPA with an LLM brain, log what agents do, and avoid giving them more permissions than a junior operator would ever get
WHAT YOU WILL LEARN
(00:01:22) The Power of Direct Computer Interaction
(00:03:19) Setting Up Computer Use: A Step-by-Step Guide
(00:06:16) Watching the AI Learn: A Fascinating but Flawed Process
(00:09:49) The Governance Catch: Balancing Autonomy and Control
(00:15:02) Building a Responsible AI Workforce
(00:20:16) Upcoming Deep Dives and Subscription Call
In this episode of M365.fm, Mirko Peters explores the new Computer Use feature in Copilot Studio and shows how Copilot can now control a real Windows desktop to open your Power App, click buttons, type into fields, and submit forms—like a suspiciously obedient digital intern instead of just a chat assistant. He explains how this vision‑driven desktopautomation works: the agent sees your screen, reasons about menus and controls in real time, and decides where to move the mouse and what to type, turning legacy Power Apps, intranet portals, and non‑API‑enabled tools into targets for true uiautomation.
Mirko walks through the full setup without breaking anything: Windows 10/11 Pro, installing Power Automate Desktop with machineruntime, registering the machine in Power Automate, and enabling it for Computer Use so Copilot Studio can securely reach the device. You will learn which accounts to use, how environment binding works, why Windows Home is excluded, and what to check in the Monitor → Machines view before you ever let an agent touch production desktops. He then shows what happens when the agent actually runs: live screen streaming, reasoning logs, mis‑clicks on date pickers, and how the model recovers by changing strategies—from clicking calendars to simply typing the correct date—demonstrating real reasoningai instead of brittle scripts.
The episode also covers governance and guardrails once agents can move the cursor for you. Mirko explains how to keep this power on dedicated machines, segment experiments from production, and combine Computer Use with standard powerautomate flows so you handle back‑end APIs where possible and only fall back to UI control where necessary. You will hear practical rules for corporate environments: treat Computer Use like RPA with an LLM brain, log what agents do, and avoid giving them more permissions than a junior operator would ever get
WHAT YOU WILL LEARN
- What Copilot Studio’s Computer Use really is and how it differs from classic RPA and flows.
- How to set up Windows, Power Automate Desktop, and machineruntime so your device appears as a Computer Use target.
- How agents actually behave on screen—clicking, typing, recovering from errors—with real uiautomation examples.
- How to combine Computer Use with standard powerautomate logic for safer, hybrid desktop/cloud automation.
- Which governance and security practices you need so autonomous desktopautomation stays controlled and auditable.