Tester Army's e2e

Tester Army's e2e (github.com/tester-army/e2e) is an open-source end-to-end testing framework for web and mobile apps. You write a goal in natural language, such as "upgrade the workspace to the Pro plan", and an AI agent drives the real app. You verify the result with ordinary locators and assertions. It is Apache-2.0 licensed, still pre-1.0, and gained about 6,400 stars on GitHub's weekly trending list in the second week of October 2026, with roughly 9,000 stars in total.

Because "e2e" is also a generic term, this article writes "Tester Army's e2e" throughout.

- Natural-language steps: write a goal with agent.act('…') and the agent looks at the screen and operates the app. agent.assert, agent.waitFor and agent.extract judge or read the screen
- Mixed with normal tests: locators such as screen.getByRole(...) and expect live in the same test. A test with no agent steps uses no model at all
- Record and replay: actions from an agent.act step that a later check verified are recorded, then replayed on later runs with no model calls. The agent returns only when the app changes
- Web: the engine is Playwright (Chromium, Firefox, WebKit)
- Mobile: drives iOS simulators and Android emulators through agent-device
- Bring your own model: a subscription, an API key or a local model. Claude subscriptions are not supported, so Claude works only through an API provider
- Add-ons: a GitHub PR comment reporter (@e2e-dev/github), hosted browsers (such as Kernel) and hosted simulators (EAS Simulators)

The key design choice in e2e is to use the AI once and record the result, instead of running it on every execution. The diagram shows the flow.

Run this in your app's directory. The wizard asks for an engine (web or mobile) and a model (subscription, provider or none), writes a config and an example test, and adds dependencies. It keeps existing files.

The docs require Node.js 24.8 or newer (22.22.3 or newer on Node.js 22). On Windows, run it inside WSL.

init generates e2e.config.ts and tests/example.e2e.ts. The config looks like this (the web example from the official quickstart).

The generated example test makes no model calls, so it runs without an API key. Start your dev server at http://localhost:3000 and run it. A test that uses the agent follows the form shown in the README.

Agent steps need a model in the config; there is no default. You can choose a gateway (Vercel AI Gateway, OpenRouter), a provider API key, or a local or self-hosted server, and the docs advise a model that supports tool calls and images. For a subscription, sign in with npx e2e login <provider> (for example openai or github-copilot).

Any CI service that can run Node works. The official GitHub Actions example runs on pull requests and pushes to main, installs Chromium, runs npx e2e run with the list and junit reporters, and uploads reports and artifacts even on failure. The model key (for example AI_GATEWAY_API_KEY) comes from repository secrets. @e2e-dev/github posts one pull request comment per run and updates it on reruns. Fork pull requests get a read-only token, so no comment is posted there.

For mobile, the docs show a two-job setup: iOS on a macOS runner (build with xcodebuild, run in a simulator) and Android on Ubuntu (KVM enabled, run in an emulator).

The biggest worry with AI-driven testing is spending tokens on every run. e2e records the actions of each agent.act step that a later check verified as JSON files in .e2e/cache/, then replays them next time. A replayed step makes no model call and costs no tokens. In the official example, the first run used 3 model calls and the second used none.

- Only verified agent.act steps are replayed. With the built-in agent, agent.assert, agent.waitFor and agent.extract call the model every run
- What causes a cache miss: a change to the test name, target name, instruction, params, agent config or app identity. Values that change each run (timestamps, fresh emails) miss unless wrapped in unique(). Changing the model does not miss
- If replay fails, the agent takes over, and an entry whose verification fails is deleted
- Handling recordings: e2e init adds .e2e/cache/ to .gitignore. You can commit it to share recordings, but typed values are stored as typed, so review them. Secrets are stored by name only
- Use --strict-cache in CI: a stale recording otherwise makes the agent take over silently and bill every run; the docs recommend failing stale steps with REPLAY_STALE

The mobile engine (@e2e-dev/mobile) drives iOS simulators and Android emulators. You need Xcode with a simulator runtime for iOS, or the Android SDK with an emulator for Android. npx agent-device doctor checks the toolchain. Each target names one platform and points at a build artifact (.app or .apk).

The official example projects cover Vite, Next.js, Astro, Expo, SwiftUI, Jetpack Compose, Kotlin Multiplatform and Flutter. For the wider picture of mobile QA, see Mobile app testing and quality assurance. Note that simulator-only controls, such as switching appearance or network, do not work on physical phones.

e2e is less a replacement for Playwright than a Playwright-based runner that lets an agent write only the fragile interactions. The table compares the main tools. The Playwright and e2e rows follow the official docs. The Cypress, Maestro, Stagehand and Browser Use rows are general characterizations, so check each project's current docs before deciding.

In practice, running both beats a full migration. The official migration guide also recommends running e2e side by side with Playwright in one project and moving one spec at a time. @e2e-dev/web ships its own pinned playwright-core, so your @playwright/test version is untouched, and e2e picks up only tests/**/*.e2e.ts by default.

- Move the flows that break often first: wizards and checkouts where selectors break after UI changes. Stable tests can stay in Playwright
- Behavior differences: string names match the whole text and are case-sensitive, multiple matches raise LOCATOR_AMBIGUOUS, and the default test timeout is 120 seconds (Playwright: 30)
- Missing config keys: some, such as globalSetup, fullyParallel and devices, are not available yet
- It can stay deterministic: replace each page.getBy* with screen.getBy* and you get a fully deterministic test with no agent

For ideas on testing and evaluating MCP servers, see the MCPJam Inspector guide. Reading it alongside the Claude Code Templates article helps place AI-agent developer tools in context.

- Pre-1.0: the README says APIs and config can change between minor releases. Pin the version and keep Playwright for your most critical tests
- Telemetry: anonymous usage data (commands, engines, where runs fail) is sent. No test content, app content or credentials are sent. Disable it with npx e2e telemetry disable or E2E_TELEMETRY_DISABLED=1
- LLM non-determinism: the first run depends on model judgment. Recording improves repeatability, but assert and similar steps call the model every time, so judgments can still vary
- Secrets in prompts: the runner redacts registered secrets from model input, but values you write directly into an instruction or param are stored in the recording as typed. Never put production credentials there
- Committed caches: review changed recordings in the pull request. Use --strict-cache in CI to avoid silent cost growth
- Replay scope: unverified steps are not recorded, so always put a check after an agent.act

Tester Army's e2e aims to combine the ease of writing in natural language with the speed and cost of a normal test, by letting the AI act once, then recording and replaying. It handles web (Playwright) and mobile (simulators and emulators) with one style, and you choose the model. Keep in mind that it is pre-1.0 with APIs that may change, that telemetry is on by default, and that assertions call the model every run. A practical start is to run npx e2e init on one UI-fragile flow, run it next to Playwright, and measure what record and replay saves.