Cypress and Gherkin: specs an agent can maintain

Gherkin has a bad name among developers, and a good part of that is our own fault. The original promise — that the business writes the scenarios — almost never happens, and what is left is a translation layer between structured English and code that has to be maintained twice over.

My position, after several projects with Cypress and the Cucumber preprocessor, is that Gherkin is still worth it, but for a different reason than the one usually given. It is not so the business can write tests. It is so the test declares what it verifies independently of how it verifies it. And that separation has become far more valuable now that there are agents touching the code.

The symptom of a badly written feature#Link to The symptom of a badly written feature

This is the degenerate form, the one that gives the format its bad reputation:

Scenario: Login
  Given I visit "/login"
  When I type "carlos@example.com" into "#email"
  And I type "hunter2" into "#password"
  And I click "#submit"
  Then the URL should be "/dashboard"

This is not a specification. It is a Selenium script with an accent. Every DOM selector is embedded in the natural language, so any markup refactor breaks the feature, and reading it tells you nothing you did not already know.

The version that does work describes intent:

Feature: Access to the private dashboard
 
  Background:
    Given a registered user with valid credentials in the role "Admin"
 
  Scenario: Successful sign-in
    When they sign in
    Then they reach their dashboard
    And their session survives a reload
 
  Scenario: Wrong credentials
    When they sign in with an incorrect password
    Then they are told it failed, without revealing whether the email exists
    And they stay on the sign-in screen

Look at the second scenario: "without revealing whether the email exists" is a real security requirement, expressed in a verifiable way, that would never show up in a script of clicks. That is what Gherkin contributes.

The step definitions are the layer that absorbs change#Link to The step definitions are the layer that absorbs change

With the intent in the .feature, everything that knows about the DOM sits below it. The usual advice at this point is to add a page object. That is not what I did, and the thing I did instead is the part of the practice I would actually defend: the reusable unit is the step.

Step definitions live in one flat namespace: a phrase written in a .feature is matched against every step registered in the suite. The Cucumber preprocessor exports a Step helper that runs that same match from inside a step definition — so a step body can call any phrase a feature file could have written. One function, and a Gherkin line becomes composable:

// cypress/e2e/steps/auth.steps.ts
import { Given, When, Step, DataTable } from '@badeball/cypress-cucumber-preprocessor';
 
// The role in the .feature is a key, not a credential: it indexes a map of
// seeded users, so no password ever appears in a spec.
Given(
  'a registered user with valid credentials in the role {string}',
  function (role: string) {
    this.user = resolveUser(role);
  },
);
 
When('they sign in', function () {
  Step(this, 'the user goes to the relative path "/login"');
  Step(this, `the user fills the input with the label "Email" with the value "${this.user.email}"`);
  Step(this, `the user fills the input with the label "Password" with the value "${this.user.password}"`);
  Step(this, 'the user clicks on the button with the label "Sign in"');
});

Compare that to the page-object version of the same step:

When('they sign in', function () {
  loginPage.visit();
  loginPage.fillEmail(this.user.email);
  loginPage.fillPassword(this.user.password);
  loginPage.submit();
});

Four lines either way, and the page-object version has the compiler on its side — which is a real thing to give up. The difference is not length; it is who else can call those lines. loginPage.fillEmail is reachable from TypeScript and nowhere else. the user fills the input with the label "Email" with the value "…" is reachable from every feature file in the suite, because it is resolved out of the same namespace a .feature is resolved out of. That is what the flat namespace buys, and it is the whole trade.

So they sign in is not a function that happens to be called from Gherkin. It is a phrase built out of smaller phrases.

(Note the function keyword rather than an arrow: this is the Mocha context, one object shared by every step in a scenario — it is how this.user survives from the Given into the When — and an arrow has no this to hand to Step.)

they sign in is the middle of three tiers, all made of the same material:

• atomic steps — one interaction: each of the four lines in that body, a fill or a click
• compound steps — they sign in itself: a handful of atomic steps and nothing else
• composite Givens — Given an Admin has created a record ready for review: an entire multi-role workflow behind one line of a Background

Only the first tier touches the DOM.

One constraint comes with all this. Step queues a Cypress command rather than running one, so anything a step writes to this lands after the enclosing body has already returned. Context flows between step definitions — the queue drains in between — never between two Step() calls in the same body. Read this.user from the Given, as above; never from a Step() on the line before.

Where the selectors go, then#Link to Where the selectors go, then

At the bottom of the vocabulary the DOM finally appears, and when it does it is generic:

// cypress/support/selectByLabel.ts — one rule for every labelled field in the app
export function getFieldControlByLabel(label: string) {
  const isFieldLabel = (_index: number, element: HTMLElement) =>
    element.textContent?.replace('*', '').trim() === label;
 
  return cy.get('label').filter(isFieldLabel).siblings('div');
}

Call that the resolver. A page object maps a name to a selector, one entry per field per screen; the resolver maps a whole class of fields with a single rule — any field whose visible label is X. Add a field to a form and nothing in the support code changes: naming the new label is the whole edit.

The trade is explicit in both directions. You depend on the label text, which the requirement depends on anyway, instead of on a data-testid, which only the tests depend on. In exchange, a copy change to a label is a change to the tests — the right amount of coupling when the label is part of the requirement, and a nuisance the day the app is translated.

And it buys that with two holes worth knowing about. Two identical labels on one page is an ambiguous match rather than a clean failure. A control named only by aria-label has no <label> to find at all — which is the one place the Playwright half below is better, since a role-based locator asks for the accessible name and gets it either way. For controls with no label the testid goes into the .feature as a {string} instead: still an argument, still not a map. And the shape of that sibling lookup is a contract with one app's form markup, not a universal rule — in a label-column layout the control is not the label's sibling and the same function returns nothing.

A form is a table, not a paragraph of steps#Link to A form is a table, not a paragraph of steps

Once that vocabulary exists, filling a form stops being one Gherkin line per field:

When the user fills the form with table
  | Label                | Type           | Value            |
  | Reference ID         | Input          | {randomRef}      |
  | Category             | Multi Select   | Premium          |
  | Destination          | Select         | Northside Office |
  | Notes                | Textarea       | Automated check  |
  | Pickup Date and Time | DateTimePicker | +1 day           |

And the step that consumes it is a router, not an implementation:

When('the user fills the form with table', function (dataTable: DataTable) {
  for (const [label, type, value] of dataTable.rows() as Array<[string, string, string]>) {
    const expanded = interpolate(value, this); // {randomRef} → this.randomRef
 
    switch (type) {
      case 'Input':
        Step(this, `the user clears the input with the label "${label}"`);
        Step(this, `the user fills the input with the label "${label}" with the value "${expanded}"`);
        break;
      case 'Select':
        Step(this, `the user selects the option "${expanded}" in the select with the label "${label}"`);
        break;
      // …one branch per widget family, each of them a step a feature could call itself
      default:
        throw new Error(`Unknown field type "${type}" for label "${label}"`);
    }
  }
});

That default is not decoration. Without it a mistyped Type — Multiselect for Multi Select, a trailing space — silently skips the row, leaves the field empty, and lets the scenario pass. A dispatch table with no default is exactly the kind of test that is green because it verified nothing.

The Type column is a dispatch key onto another Gherkin step. In the suite this comes from, that one step carried around a hundred and fifty call sites across some fifty feature files — the most reused line in the whole thing.

And because DataTable is a class you can construct, a composite Given fills the same form from TypeScript through the same step:

Step(this, 'the user fills the form with table', new DataTable([
  ['Label', 'Type', 'Value'],
  ['Reference ID', 'Input', referenceId],
]));

One filler, two callers. A page object with a fillCreateForm(…) method buys the same reuse in TypeScript, but the feature file loses the table — and with it the ability to read, in the spec, exactly which fields a scenario sets.

What it costs#Link to What it costs

One flat namespace is what makes all of the above work, and it is also the bill. Every definition is a candidate for every step in the suite, so a phrasing loose enough to be convenient at fifty features can start colliding at three hundred — and a collision is a hard MultipleDefinitionsError, not silent shadowing. Better than shadowing, but it is thrown when the step runs, so a careless new definition in one app's step file breaks another app's features and you find out scenario by scenario.

Which is why a second app in the same suite has to carry its scope in the wording — the user is logged in to the chat app as …. That reads like a style preference and is not one: it is collision avoidance, because step resolution has no other namespace to offer.

So: three things, each with one reason to change, but only two of them grow with the app. The .feature changes when the requirement changes. The vocabulary changes when the interaction changes. The resolver changes when the markup changes — and there is one of it for the whole suite, not one per screen.

Why this matters more with agents#Link to Why this matters more with agents

An agent you ask to "fix the failing test" has a perverse incentive: the fastest way to make a test pass is to weaken it. Swap a should('be.visible') for a should('exist'). Raise a timeout. Add a .first() to an ambiguous selector. All of those edits are local, plausible, and they switch the verification off.

When the intent lives in a .feature, the agent does not need to touch it in order to fix a selector; you have an anchor. Reviewing the PR collapses into one question: did any .feature change? If not, the contract still stands, and the diff is mechanical. If it did, it has to be read carefully, because what the system promises has been modified.

In practice, I make this explicit in the repository instructions:

The files `cypress/e2e/features/**/*.feature` describe product requirements. Do not modify them to make a test pass. If a scenario fails and you believe the scenario is wrong, say so in the PR and wait for human review.

It is a simple rule and it works surprisingly well, because it turns a fuzzy judgment ("does this change weaken the test?") into a binary check on which files the diff touches.

The minimal setup I use#Link to The minimal setup I use

.cypress-cucumber-preprocessorrc.json is the short half, and the load-bearing one:

{
  "stepDefinitions": ["cypress/e2e/steps/**/*.ts"]
}

That one glob is what makes resolution global, and it is easy to get wrong by leaving it out. The default patterns interpolate [filepath] — the feature file's own path — so out of the box each feature sees only its own paired step directory plus cypress/support/step_definitions. A pattern with no [filepath] in it resolves to the same file set for every feature. That is the entire reuse story above, and it is one line.

Nothing needs configuring to make a missing step fail loudly: the registry throws MissingDefinitionError when no expression matches, and MultipleDefinitionsError when more than one does. There is no pending state to quietly fall into.

The runner config is longer and less interesting — specPattern pointing at .feature files and setupNodeEvents wiring the preprocessor into the bundler are the two lines that matter, the rest is the shape any Cypress config has:

// cypress.config.ts
import { defineConfig } from 'cypress';
import createBundler from '@bahmutov/cypress-esbuild-preprocessor';
import { addCucumberPreprocessorPlugin } from '@badeball/cypress-cucumber-preprocessor';
import createEsbuildPlugin from '@badeball/cypress-cucumber-preprocessor/esbuild';
 
export default defineConfig({
  e2e: {
    baseUrl: 'http://localhost:3000',
    specPattern: 'cypress/e2e/features/**/*.feature',
    supportFile: 'cypress/support/e2e.ts',
    async setupNodeEvents(on, config) {
      await addCucumberPreprocessorPlugin(on, config);
      on('file:preprocessor', createBundler({
        plugins: [createEsbuildPlugin(config)],
      }));
      return config;
    },
  },
});

Where the markup coupling lives is a size question#Link to Where the markup coupling lives is a size question

Everything above is Cypress, and it is what a suite of roughly three hundred feature files across two apps pushed me towards. The acceptance suite for this blog is nothing like that size — nine scenarios behind three page objects, Playwright with playwright-bdd — and it does have a pages/ layer:

apps/e2e/features/acceptance/blog.feature   ← intent, no selectors
apps/e2e/steps/acceptance/blog.steps.ts     ← intent → action
apps/e2e/pages/blog-index.ts     ← the only layer that may know about markup

That is not a contradiction. It is the same rule answered at a different size, and the rule is this: no line of a .feature names a selector. Below that line the only open question is where to concentrate the markup coupling.

The number to look at is not the feature count; it is how many different flows name the same field. While that is one, a map per screen is one name per field and costs nothing. Once it is a dozen, the map is a dozen names for one input — where a resolver keyed on the visible label has exactly one. That is a check you can run against your own suite this afternoon, and it is why these two suites landed where they did.

playwright-bdd exports a Step too, but it is a step constructor that does not care which keyword you wrote, not a way to call one step from inside another; here the sharing happens through fixtures instead, and steps/fixtures.ts hands every step of a scenario the same page-object instance. So the runner does not decide this either — the shape of the suite does.

The blog feature file — three of the nine scenarios — is short enough to read in full:

Feature: Blog publication
  As a site visitor
  I want to read the published articles
  so I can judge the author's technical judgment.
 
  Background:
    Given at least one published article exists
 
  Scenario: The index lists the articles
    When I visit the blog index
    Then I see the list of articles
    And each article shows its date and reading time
 
  Scenario: Read a full article
    When I visit the blog index
    And I open the first article
    Then I see the article body
    And the article title is the page's main heading
 
  Scenario: Drafts are not published
    When I visit the blog index
    Then no listed article is marked as a draft

Not one line names a CSS class, a URL or a specific post. At this size the selectors live one layer down, in the page object, which states that rule in its own doc comment because it is the file the rule is about. Its locators are role-based, so a heading is a heading to a screen reader and to the suite alike, and a class rename does not turn the build red. It also refuses to expose a way in by slug: the only way to an article is a click on the rendered list, identified by position, because a step that hardcodes a slug rots the day the content changes.

The last scenario is the one I would point at first. "No listed article is marked as a draft" is trivial to write and trivially passed by accident: a site with no drafts in it satisfies that line forever while verifying nothing. So the repo carries e2e-draft-fixture.mdx, a post whose entire reason to exist is to be a draft that must never ship, plus a unit test pinning its draft: true flag and its exact title string so that nobody can quietly publish it and leave the scenario green. The spec and the fixture built to make it fail honestly are the same commit.

And this is not a suite I run when I remember to. It is a required job in the merge gate, alongside type checking, lint, unit tests and the visual diff. No branch reaches main without it.

That is the part that makes the whole argument checkable rather than merely stated: the files above are not an illustration of how I would organise a suite. They are the files this page was published through.

When not to use Gherkin#Link to When not to use Gherkin

For component tests, I do not use it. The ceremony does not pay for itself: a component test is already readable, and its contract is the component's API, not a user story. Gherkin wins in flows that cross several screens and where the requirement is debatable.