Cubicles
Cubicles

What is a computer-use agent? How AI learned to use a desktop

A chatbot tells you how to do something. A computer-use agent opens the app, clicks the button and saves the file. Here's how that works, and what it means for the work you hand off.

Blog ·

For most of their short history, AI assistants lived in a chat box. You asked, they answered, and you did the work. In 2025 and 2026 that changed. A new kind of AI, the computer-use agent, started doing the work itself: browsing sites, filling forms, editing spreadsheets and writing code on a real computer.

What is a computer-use agent?

A computer-use agent is an AI that operates a computer to complete a task. It works through the same things a person uses: the screen, the mouse and keyboard, the browser and the terminal. You give it a goal, like “find five suppliers and put their prices in a spreadsheet”, and it works out the steps and carries them out.

Popular examples include OpenClaw and Hermes Agent, which act as personal assistants, and coding agents like Claude Code, Codex and OpenCode. Model makers including Anthropic and OpenAI also train models specifically to read screens and control a computer.

How it works: look, think, act, check

Under the hood, almost every computer-use agent runs the same loop:

  1. Look. It takes in the current state: a screenshot of the screen, the text of a web page, or the output of the last command.
  2. Think. The model decides on the single next step that moves it towards the goal.
  3. Act. It clicks, types, scrolls, runs a command or calls a tool.
  4. Check. It looks again to see whether that worked, then repeats until the task is done or it needs your help.

A simple task might take ten turns of this loop. A research job across thirty websites might take hundreds. Each turn is a call to the model, which is why speed and cost depend so much on how the agent sees the computer.

Three ways an agent can use a computer

MethodHow it sees and actsGood atWeak at
Screen and mouseScreenshots in, clicks and keystrokes outAny app, even ones with no APISlower, and images cost more tokens
Browser controlReads the page's structure and drives the browser directlyWebsites, forms, web appsDesktop apps
Terminal and APIsRuns commands and calls servicesCode, files, data, anything with a command lineApps without a command line

Good agents mix all three: the terminal when there's a command for the job, the browser for websites, and the screen for everything else. That's why an agent works best on a full computer with a desktop, a browser and a terminal, rather than in a chat box or on a bare server.

What computer-use agents are good at

  • Repetitive web chores. Updating prices in a store admin, downloading monthly invoices, copying data between two tools that don't integrate.
  • Research across many sites. Comparing suppliers, tracking competitors' prices, collecting sources for a report. See AI agents for web research.
  • Inbox and calendar work. Sorting, labelling and drafting replies for you to approve. See how to let an AI agent manage your inbox.
  • Documents and spreadsheets. Turning scattered findings into a table, a report or a slide outline.
  • Software work. Writing code, running tests, fixing what breaks and opening a pull request.
  • Testing web apps. Clicking through a sign-up flow in a real browser and reporting what broke.

For a list aimed at running a business, see 10 jobs a small business can hand to an AI agent.

Where they still struggle

  • Bot checks and CAPTCHAs. Some sites challenge automated traffic. A real browser helps, but the agent may still need you to step in.
  • Two-factor codes. It can't read the code on your phone. Plan to be reachable for the first sign-in.
  • Speed. Every step is a model call. A task that takes you two minutes might take an agent ten, though it doesn't need you there.
  • Long, vague tasks. Agents drift on open-ended goals. Clear tasks with a defined result work far better.
  • Being tricked. Text on a page can contain instructions aimed at the agent. Read our security checklist before giving one your accounts.

Headless browser or real desktop?

Many automation tools run a headless browser, one with no visible screen, on a server. It's cheap and fast, and fine for scraping simple pages or running a fixed script (see AI agents vs Playwright, Selenium and RPA for when a script is the better tool). For an agent doing real work, it falls short in three ways:

  • Sites notice. Headless browsers look different from the browsers people use, and many sites treat them as bots.
  • No apps. A headless server can't open a spreadsheet app, a code editor or anything else with a window.
  • Nothing to watch. When something goes wrong, there's no screen to look at and no easy way to step in.

An agent on a real desktop uses a normal browser, can open any app you install, and gives you a screen to watch. Data-centre traffic can still be challenged by some sites, so expect the odd check either way.

Where should a computer-use agent run?

Not on your own laptop, for long. An agent there uses your mouse and keyboard while you're trying to work, stops when the lid closes, and has access to all your files and accounts. It needs a computer of its own that stays on: a spare machine, a Mac mini or a cloud computer. Our comparison of laptop vs Mac mini vs VPS vs a cloud computer walks through the options.

Try one in five minutes

Cubicles gives an agent its own Ubuntu desktop in the cloud, with Chrome, Firefox, VS Code, a terminal and LibreOffice, and OpenClaw, Hermes, Claude Code, Codex and the ChatGPT desktop app installed. Open it in your browser, start an agent, give it a small task and watch it work. Your first computer is free, with no card needed.

A good first task: “Compare five suppliers for recycled packaging, with minimum orders and lead times, and save the comparison as a spreadsheet with source links.” Our research setup guide walks through it.

Frequently asked questions

What's the difference between a chatbot and a computer-use agent?

A chatbot answers in text and you act on it. A computer-use agent acts itself: it opens apps, browses sites, types, clicks and runs commands on a computer until the task is done.

Is a computer-use agent the same as RPA?

No. Robotic process automation follows fixed, pre-recorded steps and breaks when a page changes. A computer-use agent decides each step as it goes, so it can handle pages and tasks it hasn't seen before.

Do computer-use agents need a GPU?

Not usually. The model runs at your AI provider. The agent's computer only needs to run the browser and apps, which an ordinary computer handles fine.

Which agents can use a computer?

OpenClaw and Hermes Agent for personal-assistant work, Claude Code, Codex and OpenCode for coding, and tools like Browser Use for the web. All of them run on Linux and come installed or run on a Cubicles computer.

Are computer-use agents safe?

They're as safe as the access you give them. Run one on its own computer, sign it in only to the accounts its job needs, and keep payments and irreversible actions behind your approval.

Give your agent a computer