Newzly.net

technology·Oregon

Google Adds 'Computer Use' Agent to Gemini Desktop App on macOS

Google is rolling out an agentic computer use feature for its macOS Gemini app, allowing the artificial intelligence assistant to autonomously handle multi-step desktop tasks across local documents, native software, and web browsers.

2 min readvia The Verge
A MacBook running the Google Gemini desktop application executing automated workflows in a background window.
A MacBook running the Google Gemini desktop application executing automated workflows in a background window.

In this piece

Google has launched an agentic computer use capability for its Gemini desktop application on macOS, allowing the AI to execute multi-step workflows across local…

Google has introduced an agentic "computer use" feature to its native Gemini desktop application for macOS, granting the artificial intelligence assistant the ability to interact directly with local files, third-party software, and web browsers. According to company announcements, the tool is designed to carry out complex, multi-step productivity workflows on behalf of users rather than merely generating text responses or step-by-step guidance.

Under the new capability, the assistant can click buttons, type input, navigate application menus, and complete online forms autonomously. Instead of seizing direct control of the user's primary pointer or active screen, Gemini executes these tasks inside an isolated background environment. Users can observe the operation in real time through a live visual feed nested directly within the Gemini chat panel, leaving their own cursor free for simultaneous work.

Local system integration plays a central role in Google's vision for the desktop agent. Users can grant the software explicit permission to examine designated local directories and extract structured details from stored documents. For example, the system can parse product specifications, shipping weights, and dimensions from saved PDF files and automatically transfer that data into web-based logistics portals or inventory spreadsheets.

Google also showcased creative and administrative use cases intended to streamline routine desktop chores. The agent can batch process visual assets in design applications, stage photography for online sales platforms by organizing files from local folders, or assemble presentation slides using data scraped from live web analytics dashboards. By chaining together native applications with web services, the feature seeks to eliminate repetitive manual data entry.

Because granting software direct control over an operating system carries inherent security considerations, Google has embedded several operational guardrails into the software. The assistant only operates within folders and applications explicitly approved by the user during setup. Crucial actions, such as submitting finalized web forms, executing financial transactions, or sharing sensitive files, require mandatory human confirmation before proceeding.

Google explicitly acknowledged that autonomous desktop agents remain experimental and can occasionally make mistakes when interpreting visual interfaces or complex file structures. To mitigate the risk of accidental data modification or deletion, the application maintains temporary file backups during active tasks. These recovery caches are automatically purged after twenty-four hours or whenever a user initiates a new task assignment.

The deployment arrives with specific hardware and account prerequisites. The feature requires an Apple Mac computer equipped with Apple Silicon processors running macOS Sequoia version 15.0 or later. Access is also restricted to paying subscribers enrolled in Google's premium tiers, including Google AI Plus, Pro, and Ultra subscription plans.

The release underscores an accelerating industry shift from conversational chatbots toward proactive digital agents capable of operating software independently. Rival technology firms, including Anthropic with its own desktop computer use protocols and Microsoft with native Windows automation tools, have similarly targeted autonomous workplace workflows. Integrating such capabilities directly into desktop operating systems represents the next major competitive battleground for generative AI providers.

The initial rollout is currently limited to English-language users in the United States, with Google planning to expand availability across additional regions in subsequent software updates. As developers and enterprise customers evaluate how reliably the tool navigates unpredictable desktop software, user feedback will likely determine how quickly autonomous agents transition from experimental utilities into standard desktop fixtures.

Primary source: The Verge. Newzly rewrites and structures source reporting for clarity; verify critical details at the original.