How good is tagging?
Rhetorical question; Tagging is great; my favourite organisation system, in fact. Test systems with tagging allow you the flexibility to organize your tests according to what code they're testing, while running tests according to what kind of test you're after.
Tagging & Cypress
Unfortunately, Cypress lacks native support for tagging, and this makes me very sad face emoji.
So, I'm going to use
saucectl to retrofit tagging functionality onto our Cypress tests, whether you're running locally in Docker mode, or remotely against Sauce Labs' cloud.
Wait, what's saucectl?
Let's go to the README!
saucectlcommand line interface orchestrates the tests in your framework, providing rich parallelization, test history filtering, and analytics in Sauce Labs.
saucectl is an amplifier for your tests. It drags your Cypress, Playwright, TestCafe or Puppeteer kicking and screaming onto the Sauce Labs cloud, giving you delicious Sauce features (like analytics) along with all the goodies your framework has to offer. Saucectl can also run your tests with Docker, giving you a quick and reliable test environment. It's a great solution for using Cypress with CI.
(Disclosure: I work for Sauce Labs. I still really like
config.yml (the config file for
saucectl), you can specify individual suites see here which let you provide individual config options to Cypress.
By passing the
testFiles option to Cypress, you can control which individual files it runs. We're going to combine these two, to give us individually named test suites, each of which runs tests with a specific set of tags.
Can't you do that with Cypress alone?
Yes, technically, but I prefer this approach for... let's call them ideological reasons.
While you could have individual config files, and call
cypress run --config-file somesuite.json for each one, it's messy. If you make changes to one suite file, you have to make it across all of them, which is easy to forget.
With this solution, all your Cypress config stays as is, and only the tests being executed are changed. Even better, control of which tests run is maintained in the infrastructure configuration, and infrastructure is likely to have the most significant impact over what 'flavours' of test you want to run.
How we're implementing tagging
This solution makes use of globbing, a way of selecting a group of files. It's pretty common; Common enough to have a Wikipedia page, and you've likely seen at least one glob before.
We're going to alter our tests so each test suite is configured to run a set of globs corresponding to our tagged tests.
Say this is our test suite. Each file contains tests that should only run on a specific browser (or combination of browsers). We've named them so each filename includes the relevant tags, separated by underscores, like
_ieOnly. These could just as easily be
_FrenchLanguage or anything you want to tag by.
specs/ ui/ login_chromeOnly_spec.js login_firefoxOnly_spec.js accounting/ jsMathBugFix_ieOnly_spec.js chromiumCantCount_edgeOnly_chromeOnly_spec.js
Here's the suites component of our
config.yml. Each suite has a unique glob for testFiles which finds all tests in your specs folder which include the respective tag/s.
suites: - name: "Where There's Smoke There's" browser: "firefox" platformName: "Windows 10" config: testFiles: ["specs/**/*firefoxOnly*_spec.js"] - name: "Cream on" browser: "chrome" platformName: "Windows 10" config: testFiles: ["specs/**/*chromeOnly*_spec.js"] - name: "Livin' on the" browser: "edge" platformName: "Windows 10" config: testFiles: ["specs/**/*edgeOnly*_spec.js", "specs/**/*ieOnly*_spec.js"]
How it works
Each glob starts with
specs/**/*. This glob means "start in the specs folder (
specs/), then look in any subfolder(
**), for any characters at all(
*) followed by an underscore(``)".
Let's look at how the first glob continues. After
specs/**/* we have
firefoxOnly*_spec.js. This means "_find anything containing the phrase
firefoxOnly, followed by zero or more of any character(*), followed by
_spec.js_". For our tests, that only matches
The second glob is
specs/**/*chromeOnly*_spec.js, so it matches every file in every subfolder of
specs which contains
chromeOnly, followed by zero or more of any character, followed by
_spec.js. In our case, that's
Did you notice?
testFiles parameter doesn't take a glob directly; it takes an array of globs. That lets us pass in multiple entries, which is how suite three works. We're able to pass in two entries, one for any test tagged
edgeOnly and one for any tagged
ieOnly. It will match any file in either of those blobs; in this case
Additionally, test files can have multiple tags. This is really useful when you want to run some tests in multiple scenarios; Say you have a limited set of functional tests that are also used to ensure deploys go well; You could tag them
_functional_postDeploy_spec.js and they'd run when using either the
_functional tag or the
And that's it!
A small change to how you name and select tests, and you're left with a more flexible testing system. And of course, you can still run a default suite with no tags, and make sure you're running everything.