Technical Documentation

1. Overview

TraceKit is an autonomous browser testing agent designed to eliminate selector maintenance and fragile test scripts. Rather than relying on rigid XPath or CSS selector paths that break on DOM updates, TraceKit translates high-level natural language test goals into resilient, multi-step browser actions powered by multimodal LLM reasoning and verified by deterministic Chromium assertions.

2. How TraceKit Works

Each test run executes inside an isolated Patchright/Chromium browser context using a 5-stage continuous execution cycle:

3. Agent Actions

The agent communicates decisions via a strict discriminated Pydantic action schema supported by the backend:

4. Locator Priority Ladder

TraceKit generally prefers accessible locators (such as role/name, text, placeholder, and label) to avoid brittle test scripts. However, when an explicit selector is provided, selector resolution takes precedence in execution:

1. CSS Selector / data-testid: (e.g. selector="[data-test='login-button']") — takes precedence whenever explicitly provided.
2. Role + Accessible Name: (e.g. role="button", name="Add to cart") — preferred accessibility locator.
3. Exact or Partial Text: (e.g. text="Checkout") — matches visible rendered strings.
4. Placeholder Attribute: (e.g. placeholder="Enter your email") — ideal for search and text inputs.
5. Associated Label: (e.g. label="Password") — resolves inputs via associated form <label> tags.
6. Disambiguation Index: (e.g. index=0) — 0-based integer to disambiguate and select an exact element when multiple matching elements are found.

5. Deterministic Assertions

Unlike subjective LLM judgments, TraceKit verifies test conditions directly against Chromium DOM APIs using Playwright's expect assertion library:

6. LLM Reasoning Providers

TraceKit supports multiple reasoning providers configured through environment variables or Test Studio advanced options:

7. Session & Storage State

TraceKit supports testing post-login flows without repeating the login sequence on every test execution. Users can supply Playwright-compatible storage_state JSON in the Advanced Settings drawer:

{
  "cookies": [
    {
      "name": "session_token",
      "value": "xyz...",
      "domain": ".example.com",
      "path": "/",
      "httpOnly": true,
      "secure": true
    }
  ],
  "origins": [
    {
      "origin": "https://example.com",
      "localStorage": [{ "name": "user_id", "value": "1001" }]
    }
  ]
}

When provided, the browser context initializes with these injected cookies and local storage tokens, immediately loading authenticated dashboards or checkout flows.

8. Failure Diagnosis Engine

When a test run finishes with failure, the backend diagnosis engine (diagnosis.py) categorizes the failure into two major classes and pinpoint causes:

9. Reports & Artifacts

Every test run produces complete, inspectable evidence packages persisted to the backend artifacts directory:

10. Example Workflow

Here is a realistic end-to-end execution trace demonstrating autonomous e-commerce checkout verification on Saucedemo: