From Gherkin to Playwright: AI-Generated Tests for Supabase Applications

As AI accelerates software development, maintaining automated tests is becoming one of the biggest challenges. While applications can evolve in minutes, keeping test suites synchronized often remains a manual and time-consuming process.

Gherkin provides a structured, implementation-independent way to describe application behavior. Its clear syntax makes it understandable by both humans and Large Language Models, making it an excellent source of truth for AI-generated tests.

In this article, we'll demonstrate how Gherkin https://cucumber.io/docs  feature files can be used to generate Playwright end-to-end tests for Supabase applications, allowing teams to maintain business behavior rather than implementation-specific test code.

Gherkin as the Source of Truth

Before writing a single Playwright test, we need to answer a more fundamental question:

What should we actually maintain?

Instead of maintaining executable tests directly, we maintain Gherkin feature files a concise, implementation-independent specification of how the application should behave https://cucumber.io/docs/gherkin/reference. The Gherkin scenarios become the source of truth, while Playwright tests become generated artifacts.

Throughout this article, we'll use a locally running SonarQube instance as our example application. The following feature describes a simple user authentication scenario.

Feature: User Authentication
  Scenario: Successful login
    Given a registered user exists
    And the SonarQube application is running
    When the user logs in with valid credentials
    Then the user should be authenticated
    And the dashboard should be displayed

Notice what is intentionally missing. The specification does not mention Playwright, CSS selectors, HTML elements, URLs, JavaScript APIs, or implementation details. It simply describes the expected business behavior from the user's perspective.

This distinction is important because business behavior usually changes much more slowly than software implementation. User interfaces evolve, APIs change, frameworks are replaced, and applications are refactored. Yet the expected outcome such as a user successfully signing in often remains exactly the same.

By maintaining Gherkin scenarios instead of implementation-specific tests, we reduce the amount of information that must be updated as the application evolves. Large Language Models can then transform these scenarios into executable Playwright tests, allowing teams to regenerate tests as the implementation changes while keeping the business specification stable.

From Gherkin to Playwright

The Gherkin feature defines what the application should do, but not how it does it. Before an AI agent can generate a Playwright test, it must understand how each scenario is implemented in the application.

Rather than relying on browser recordings or guessing selectors, the agent analyzes the application source code to ground every Gherkin step in the actual implementation. For our SonarQube example, this means inspecting the sonarqube-webapp frontend to identify the components responsible for authentication and the post-login user experience.

Gherkin Step	Implementation Grounding
Given the SonarQube application is running	Unauthenticated requests to / are redirected to /sessions/new by the authentication middleware and session routes.
Given a registered user exists	The test uses an existing SonarQube account with valid credentials supplied through environment variables.
When the user logs in with valid credentials	LoginForm.tsx exposes the login form, including the #login-input, #password-input, and the Log in button.
Then the user should be authenticated	GlobalNavUser.tsx renders the authenticated navigation, exposing the Account control.
And the dashboard should be displayed	DefaultPageSelector redirects new administrators to /projects/create, where the onboarding page and global navigation become visible.

This grounding phase is essential. Instead of treating the application as a black box, the AI connects each business step with the components that implement it. As a result, the generated test is based on the application's source code rather than fragile browser recordings or manually discovered selectors.

To produce high-quality Playwright code, the agent is also provided with the official Playwright Agent Skills. These skills contain framework-specific knowledge, recommended APIs, locator strategies, and testing best practices. Combined with the Gherkin feature and the grounded application source, they enable the agent to generate tests that follow Playwright conventions while remaining faithful to the original specification.

The generation workflow is therefore:

1. Read the Gherkin feature.

2. Analyze the application source to ground each scenario step.

3. Apply Playwright Agent Skills to generate idiomatic Playwright code.

4. Execute the generated test against a running SonarQube instance.

AI Generated Playwright Test

The resulting Playwright test preserves the structure of the original Gherkin scenario while translating each business step into concrete browser interactions and assertions.

// feature: features/user-authentication.feature
// UI grounding: LoginForm / GlobalNavUser / CreateProjectModeSelection

import { expect, test } from '@playwright/test';

const username = process.env.SONAR_USERNAME ?? 'admin';
const password = process.env.SONAR_PASSWORD ?? '1234abcd!AAA';

test.describe('User Authentication', () => {
  test('Successful login', async ({ page }) => {
    // Given the SonarQube application is running
    // And a registered user exists
    await page.goto('/');
    await expect(page.locator('#login-input')).toBeVisible();

    // When the user logs in with valid credentials
    await page.locator('#login-input').fill(username);
    await page.locator('#password-input').fill(password);
    await page.getByRole('button', { name: 'Log in' }).click();

    // Then the user should be authenticated
    await expect(page.getByRole('button', { name: 'Account' })).toBeVisible();
    await expect(page).not.toHaveURL(/\/sessions\/new/);

    // And the dashboard should be displayed
    await expect(page).toHaveURL(/\/projects\/create/);
    await expect(page.locator('#it__global-navbar-menu')).toBeVisible();

    await expect(
      page.getByRole('heading', {
        name: 'How do you want to create your project?',
      }),
    ).toBeVisible();
  });
});

The generated test closely mirrors the original Gherkin feature. The Feature becomes the Playwright describe, the Scenario becomes the test case, and each Given, When, and Then step is translated into browser interactions and assertions. This one-to-one mapping makes the generated artifact easy to understand, debug, and trace back to the original specification.

Most importantly, the engineering team maintains only the Gherkin feature. The Playwright test is a generated artifact that can be regenerated whenever SonarQube's implementation changes. As long as the business behavior remains unchanged, the specification stays stable while AI produces an updated implementation-specific test.

Conclusion

AI is fundamentally changing how software is built. As implementation becomes faster and increasingly automated, the bottleneck shifts from writing code to defining and preserving the system's intended behavior.

This changes where engineering effort should be invested. Instead of spending time maintaining implementation-specific artifacts such as Playwright tests, teams should focus on higher levels of abstraction: architecture, business processes, and functional specifications. These artifacts evolve more slowly than code and provide stable guidance for AI systems to generate implementation-specific outputs.

In this article, we demonstrated one practical example of this approach. Gherkin feature files serve as the source of truth for business behavior, while AI transforms those specifications grounded in the application source and Playwright knowledge into executable Playwright tests. The team owns the specification; AI owns the generated artifact.

However, one important question remains unanswered:

How can we verify that an AI-generated test faithfully implements the behavior described in the original Gherkin specification?

Generating tests is only half of the problem. We also need confidence that the generated artifacts are complete, correct, and traceable back to the requirements they are intended to validate.

That question will be the focus of the next article, where we'll explore techniques for validating AI-generated Playwright tests against their original Gherkin specifications.

Photo by Alice Butenko on Unsplash