Computer Use Agents

Work

Testing your own software

The safest task here by a distance: your environment, your disposable data, cheap false positives.

What happens

  1. Point the agent at a staging environment
  2. Give it a goal — complete signup, buy something, change a setting
  3. Let it find its own path rather than scripting each step
  4. Collect what broke, with screenshots at the point of failure

Underrated, and the one case where non-determinism is close to a feature — an agent that finds its own route through a flow exercises paths your written tests do not. It runs against your own staging environment, the data is disposable, and a false positive costs a few minutes rather than money.

Where it breaks

Non-determinism cuts both ways. A test that passes four runs in five is worse than no test, because it teaches the team to ignore failures. Mix model-driven exploration with deterministic assertions rather than replacing one with the other.

Tools that suit this

Stagehand TypeScript framework layering natural-language actions over Playwright, so deterministic code and model calls can be mixed. Browser Use Python library that hands a browser to any LLM. One of the most widely adopted open-source options. Skyvern Vision-driven browser automation aimed at workflows that break conventional selector-based scripts.

More work tasks

Work Invoice processing The canonical business case. Every organisation has a version of it, and the target system almost never has an API. Work Data entry into a system with no API Where computer use most clearly beats the alternative, because the alternative is a person doing it by hand.