Categories
Technology

Framework review: Playwright

Photo by Fatih Kฤฑlฤฑรง on Unsplash

Playwright enables end-to-end testing. Test modern single page apps, across all modern browsers, using in your preferred language (JS, TS, Java, C#, Python).

โ€” Official website

Code

Example of automation at GitHub.

Use cases ๐Ÿฅˆ

Automate end-to-end (E2E) tests using the UI. It also supports mocks.

Learning curve ๐Ÿฅˆ

By now, I know my way around a test framework, it’s just a matter of learning the syntax. The official doc was not sufficient to get started and while I was coding I needed the reference API by my side. When I was not sure how to do something, I simply enabled the “record mode” and let Playwright generate the test code of my interactions.

Language ๐Ÿฅˆ

Tests are written in JavaScript or TypeScript. Since the API is very low level, more than basic knowledge is required to code and understand the tests.

Ecosystem ๐Ÿ˜ญ

JavaScript. I used VS Code to write tests. Autocomplete is limited. You can debug your test code using breakpoints in VS Codeโ€ฆ but you need multiple watches, as you can’t evaluate expressions. Part of Playwright’s pitch is cross-browser testing. It’s super easy to enable it but super flaky to use. In my case, Firefox was (mostly) fine, Chrome returned page errors (which I had to ignore) and Safari was 3x slower (when it didn’t fail straight away) โ€“ both issues were false positives.

Readability ๐Ÿ˜ญ

Verbose and rough. A simple assertion takes three lines, thankfully expect-playwright reduces it to one. Something concise and intuitive in Cypress (page.get.first) is obscure in Playwright (page.$$[0]). The test code reads like back-end code, with async and await keywords sprinkled every other line. Also you need to explicitly wait for page loads (async page.waitForNavigation()) or selectors to be available. Yet, sometimes you get timeouts.

Extensibility ๐Ÿฅ‰

I expected to be able to run the tests right away, but I needed a test runner like Jest. Doesn’t come with a built-in way to take a screenshot on test failures, nor does it have a vast collection of plugins like Cypress. It has a few tutorials though and it supports the PageObject Model.

Maintainability ๐Ÿฅ‰

Hard to debug. When the toHaveText() assertion fails, it outputs the whole page HTML in the error log, which is a nightmare to review. Slower than expected โ€“ if the tests ran in parallel I didn’t even notice it.

Documentation ๐Ÿฅˆ

I started with the official doc and quickly found out it doesn’t work, because it’s missing the test runner (Jest) which led to a quite different code. I had to follow community guides. On the bright side, the doc is complete and easy to search.

VERDICT: 1/5

Unproductive and unreliable, so I can’t recommend it. There are better alternatives.ย