August 19, 2026

How to Configure Playwright Test to run smoke tests, headed tests, and debug versions through scripts in package.json

Earlier, we went over how we could create scripts in the package.json file of our Playwright framework to add typechecking, linting, and formatting your code with prettier
With this post, we will explore how the built-in test runner for Playwright Test can run headed tests, debug versions of tests, and smoke tests.

... and shortcuts for all of these can be set up in the scripts in your package.json! If you are using "bun" as a package manager, as we are in bun-create-playwright, just type out "bun run", a space and then the shortcut such as: bun run test

package.json
"scripts": {
    "test": "playwright test",
    "test:headed": "playwright test --headed",
    "test:trace": "playwright test --trace on",
    "test:chromium": "playwright test --project=chromium",
    "test:firefox": "playwright test --project=firefox",
    "test:webkit": "playwright test --project=webkit",
    "test:smoke": "playwright test --project=chromium --grep '@smoke'",
    "test:flaky": "playwright test --project=chromium --repeat-each=20",
    "test:ui": "playwright test --ui",
    "test:debug": "playwright test --debug",
    "test:failed": "playwright test --last-failed",
    "test:login": "playwright test tests/login.spec.ts",
    "test:secure-area": "playwright test tests/secure-area.spec.ts",
    "report:list": "playwright test --reporter=list",
    "report:line": "playwright test --reporter=line",
    "report:dot": "playwright test --reporter=dot",
    "report:blob": "playwright test --reporter=blob",
    "report": "playwright show-report",
    "codegen": "playwright codegen",
    "lint": "eslint .",
    "lint:ci": "eslint . --max-warnings 0",
    "lint:fix": "eslint . --fix",
    "format": "prettier --write .",
    "format:check": "prettier --check .",
    "format:debug": "prettier --check . --log-level debug",
    "format:diff": "prettier --list-different",
    "typecheck": "tsc --noEmit"
  },

Do you need to really set up shortcuts like these? Certainly not! But it is easier than typing out: bunx playwright test --project=chromium --grep '@smoke'.

Feel free to name these commands anything you want! 

August 16, 2026

Implementing Page Objects in Playwright

Picture a login screen, such as The-Internet / Login


On this LoginPage, there is:
  • a heading: Login Page
  • a user name textbox with the label, "Username"
  • a password textbox with the label, "Password"
  • a login button, with the role of a button, and the name of "Login"
  • a flash message that appears if you enter invalid credentials such as "NotAUser" and "NotAPassword".
If you successfully log in with "tomsmith" and "SuperSecretPassword!, there is a SecureArea:
  • a heading: "Secure Area"
  • a flash message "You logged into a secure area!"
  • a Logout button. 
Sure, you could interact with each web element in your test... but what if the username text box locator changes? You would have to update multiple tests every time the element changed. 

... Instead, you could place it in a Page Object, something that Playwright handles well!

"A page object represents a part of your web application. An e-commerce web application might have a home page, a listings page and a checkout page. Each of them can be represented by page object models.

"Page objects simplify authoring by creating a higher-level API which suits your application and simplify maintenance by capturing element selectors in one place and create reusable code to avoid repetition".

Using Playwright's Built-In Test Runner? Or Something Else?


You may have noticed in https://playwright.dev/docs/pom that there are two different styles of page objects. One for "Test". One for "Library". 
  • Test: If you are writing actual Playwright test suites, and Playwright's built in test runner, use the Test section as a guide when creating page objects. 
  • Library: If you are integrating Playwright into an existing test framework such as Jest or Cucumber and just want browser automation, instead of having pre-built page fixtures, etc, you can use this format. 

August 14, 2026

How Playwright Frameworks get configured with playwright.config.ts

When we installed bun, a new package manager owned by Anthropic, then ran "bun create playwright", a new automation framework was stood up, along with sample tests, and a Playwright configuration file. In this post, we will be examining the file generated: playwright.config.ts.  

Personally, I find the pre-generated file very hard to scan... there are so many options and documentation in the comments, it is difficult for me to focus on the code. Let's examine just the code generated below. If you need to see the whole file, you can see it here: https://playwright.dev/docs/test-configuration

Playwright.dev / Configuration mentions, "Playwright has many options to configure how your tests are run. You can specify these options in the configuration file". 

August 10, 2026

Running Tests with Playwright Test Explorer and Generating Tests with Codegen

Finding the best locator for a web element can be a hassle:
  • Right clicking on an element in Google Chrome. Inspecting the element. Going to Chrome Developer Tools. Try to decide what to do if there isn't a clear test id. 
Playwright comes with a built in code generator where it can built out a rough draft of a test while you interact with a website. "Playwright will look at your page and figure out the best locator, prioritizing role, text and test id locators. If the generator finds multiple elements matching the locator, it will improve the locator to make it resilient that uniquely identify the target element", according to Playwright.dev / Test Generator.

Do you have the Integrated Development Environment (IDE) by Microsoft, VS Code? You can get it at the Visual Studio Marketplace

Playwright Test Explorer


After installation, you will see a beaker icon in your VS Code left navigation menu. Selecting that, you can see your tests, such as the default ones Playwright automatically adds when it is installed: 


Checking code with lint, formatting it with prettier

Now that we've installed bun, Anthropic's package manager, scaffolded a Playwright framework and closely examined the results, and added typechecking with TypeScript's compiler, it's time to add ways to check the code with lint, and reformat the code with prettier

We will be using:
  • ESLint, as the static-analysis tool to review the code without running it. Little bits of fluff -- like syntax errors, structural bugs, anti-patterns, and code style violations -- can collect on your code, so it helps to run a linter to help catch it all, such as ESLint. There is also a linter,  eslint-plugin-playwright, for Playwright tests.
  • Prettier enforces a consistent code style across your entire codebase. Because ESLint and Prettier can conflict, we will be using Prettier's eslint-config-prettier, which turns off all rules that are unnecessary or might conflict with ESLint.

What is a Linter? 

According to the Wikiwand entry for Lint, "Stephen C. Johnson, a computer scientist at Bell Labs, came up with the term 'lint' in 1978 while debugging the yacc grammar he was writing for C and dealing with portability issues stemming from porting Unix to a 32-bit machine. The term was borrowed from lint, the tiny bits of fiber and fluff shed by clothing, as the command he wrote would act like a lint trap in a clothes dryer, capturing waste fibers while leaving whole fabrics intact. The lint program was released outside of Bell Labs in Unix V7, in 1979.

"In his 1978 paper, Johnson explained his reasons for creating a new program to detect errors: '...the general notion of having two programs is a good one' because they concentrate on different things, thereby allowing the programmer to 'concentrate at one stage of the programming process solely on the algorithms, data structures, and correctness of the program, and then later retrofit, with the aid of lint, the desirable properties of universality and portability' "

August 7, 2026

Add Type Checking and TSConfig to Bun-Create-Playwright

Now that we have installed bun, Anthropic's package manager, and scaffolded a Playwright framework and closely examined the results, we are going to explore with our Bun-Create-Playwright project ways to check if our code is correct. 

The first method we will be exploring is typechecking

Why Typechecking? As the Playwright.dev / Node.js Introduction mentions:
"[...] Playwright does not check the types and will run tests even if there are non-critical TypeScript compilation errors. We recommend you run TypeScript compiler alongside Playwright.

"[...] Note that Playwright only supports the following tsconfig options: allowJs, baseUrl, paths, references and extends.

"[...] By default, Playwright will look up a closest tsconfig for each imported file by going up the directory structure and looking for tsconfig.json or jsconfig.json. This way, you can create a tests/tsconfig.json file that will be used only for your tests and Playwright will pick it up automatically".
Before we go further down this road...

What is Type Checking? 


According to Type Checking in TypeScript: A Beginners Guide, every piece of data in TypeScript is given a "type", and this "type" determines what properties the data has, and what methods are available to it. The types can be things like a Number, String, Enum, Boolean, Array, Object, Type assertions, or others.

August 6, 2026

What happens when you scaffold a Playwright framework and run installed tests using bun?

We covered in the last blog post how to install bun, a new package manager. 

In this blog post, we will look into using bun to install a new Playwright framework. 

Create the Playwright Framework


Once I installed bun on my Windows PC and was up and running I created a new folder, "bun-create-playwright".

I opened up that folder in VS Code, along with a new PowerShell terminal. In that terminal I entered:
  • bun create playwright 
This activated the interactive Playwright installer that Playwright comes with. 
  • I selected I wanted it to create for me a TypeScript project, placing the tests in the default folder, tests, but I decided not to add a GitHub Actions workflow. I wanted to experiment with using GitLab
  • I chose it to set up and install all the browsers for me... which it did... using NPX, part of Node.
Wait a second, when it comes to our Playwright framework, doesn't bun replace node?

No. Bun might be a JavaScript runtime, a package manager, and a bundler shipped as a single software tool, but in our case we are simply using it as a package manager in our Playwright framework. 

Bun works alongside Node.js in the Playwright project. We still use Playwright's test runner, a Node.js program. Bun handles installations and run tasks.

August 5, 2026

Introducing bun, a new package manager and JavaScript runtime environment

New job? New toolset to explore! In this post I'll be investigating a new JavaScript package manager and runtime environment called bun.sh

You may have noticed on Playwright.Dev's Getting Started / Installation section there are three different ways to install a ui or api test automation framework with the latest version of Playwright:
  • npm init playwright@latest
  • yarn create playwright
  • pnpm create playwright
For this blog post I'll be investigating a fourth way, the new toolset they just started using at work, bun.sh.
  • bun create playwright

What is npm, yarn, pnpm, and bun? These are called package managers.

What is a package? Developing a project, you don't have to figure out how to code everything yourself. You can include in your projects outside libraries, or "packages" of code from from the public JavaScript registry npmjs.com

The problem is that these packages then need to be downloaded, installed, managed, and updated whenever they are updated. JavaScript projects can use a package manager, such as bun, to help this process. 

When putting together a Playwright automation framework based around JavaScript and TypeScript, Playwright uses the Node.js JavaScript runtime environment, to run its test runner, execute automation scripts, and manage its browser instances.(See Nodejs.org / About )

There is a division of labor: Node.js runs these internal functions like running tests. NPM, Yarn, PNPM, and now Bun handle the packages. 

Why are there so many package managers? Each has its own specialization. The original NPM is bundled already with Node.js. Yarn is a Facebook toolset that was created when NPM was found to work too slow. PNPM was created to be smaller and therefore faster. And bun was built for quick startup and installation speed, good especially if you have a CI/ CD pipeline that has a LOT of packages to install each time the tests run.

June 29, 2026

Does giving a presentation via Zoom to the Sydney Testers Meetup make me an international speaker?

If I presentation to the Sydney Testers Meetup in Australia, from my home in the Massachusetts, through Zoom, does that make me an international speaker? [ Vote in the New Poll, on LinkedIn ]

There is still time to sign up for the event at: https://www.meetup.com/sydney-testers/events/315166776


"T.J. Maher, the former organiser at Ministry of Testing Boston, blogger and speaker, will be presenting from the US on using the Detox framework to create effective test automation frameworks for React Native applications.

June 17, 2026

Achievement Unlocked: T.J. Maher Elected to the Board of Directors of the Software Quality Group of New England - SQGNE.org

I've been elected to be on the Board of Directors of the Software Quality Group of New England (sqgne.org) on the volunteer leadership committee! 

The Software Quality Group of New England (SQGNE) is an all-volunteer, nonprofit professional community in New England dedicated to software quality. Since its founding, SQGNE has brought together QA engineers, SDETs, and software quality professionals for monthly educational presentations, discussions, and networking -- covering everything from testing techniques and process improvement to the latest tools and industry trends.

I also have founded and moderate an online monthly SDET Lean Coffee discussion group, bringing together software development engineers in test, quality engineers, and automation practitioners to exchange ideas, challenges, and emerging practices.

After our hiatus during the summer, we will be meeting monthly up in Burlington, MA. Check us out!

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

June 12, 2026

Testing and AI Workshop by James Bach of Rapid Software Testing - Notes

Last week, I saw that there was a new half day workshop by one of the creators of Rapid Software Testing, James Bach, Testing and AI Workshop, and that they were offering a 50% discount for anyone who was unemployed, so I just had to attend.

"In each session, the instructors will first perform a live 'testopsy.' This is a demonstration of AI-assisted testing (using both assistive and agentic modes of AI) on a real product, accompanied by an analysis and explanation of what happened during the demo. During this part of the workshop, you may ask questions or offer critique.

"Next the instructors will challenge you to perform a similar process or solve a similar problem with the help of AI. You will have two hours. You will be able to work alone or in groups, as you like.

"Finally, the instructors will review and critique your work, if you choose to share it. At the end of the event, you will get to keep the videos".

I really enjoyed the class! It involved an hour-and-a-half webinar, where James Bach walks people though how he uses AI, a few hours where you can work on your own project for the course, then another hour-and-a-half webinar where course attendees could review what they came up with to analyze a site. 

June 3, 2026

New Position Unlocked: Senior SDET at AAA Life Insurance, starting Monday, June 15th, 2026!

I have two announcements: I've accepted a job offer as a Senior SDET role at AAA Life Insurance Company, and will be starting Monday, June 15th! And I have been nominated to be one of the volunteer Directors on the leadership board of the Software Quality Group of New England.

Man, the job market is brutal! It took me four months of job searching in 2025 to find SELF Id when MassMutual outsourced its technology department. And it took me four months of near constant job searching in 2026 to find AAA Life when I was caught up in the second round of SELF's layoffs in the end of January.

June 2, 2026

SDET Lean Coffee #1: With AI, what is useful testing and what is workslop? SQGNE, June 2, 2026

With AI, what is useful when it comes to testing and what is workslop? How do you create workflows in AI? With AI producing massive amounts of code, how can a tester keep up?

These are some of the topics attendees decided to talk about in our first ever SDET Lean Coffee, as part of the Software Quality Group of New England ( sqgne.org ). We exchanged war stories, horror stories, shared insights, and provided a bit of group therapy as we talked about the stress involved being the main support role of the software development team.

A surprise guest was Lisa Crispin (LisaCrispin.com), author of "Agile Testing: A Practical Guide for Testers and Agile Teams". Lisa has been working with DORA, Google Cloud's DevOps Research and Assessment division.

Recently, Lisa gave a talk teaming up with the "Beyond Quality" podcast, sharing what she has been doing "AI, testing, and the DORA AI Capabilities Model" at Lisa's site at https://lisacrispin.com/2026/04/20/ai-testing-and-the-dora-ai-capabilities-model/ discussing:
  • The Dora AI Capabilitues Model
  • How we need to test AI agents since AI agents can degrade over time

What is a "Lean Coffee"?

"Lean Coffee is a structured, but agenda-less meeting. Participants gather, build an agenda, and begin talking. Conversations are directed and productive because the agenda for the meeting was democratically generated". This format arose over fifteen years ago, when "Jim Benson and Jeremy Lightsmith wanted to start a group that would discuss Lean techniques in knowledge work – but didn’t want to start a whole new cumbersome organization with steering committees, speakers, and such. They wanted a group that did not rely on anything other than people showing up and wanting to learn or create", according to LeanCoffee.org.

When Is The Next SDET Lean Coffee? 


SDET Lean Coffees for the SQGNE will (usually) be held the first Tuesday of each month at 12:00 pm to 1:00 pm EDT.

Interested in attending the next session Tuesday, July 7th?

Register at the Software Quality Group of New England website at https://www.sqgne.org/

Happy Testing!
-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

June 1, 2026

Come join us at the next SQGNE Meeting! Open-Source Malware: Defending Your Software Supply Chain From Evolving Threats - June 17, 2026

"Open-Source Malware: Defending Your Software Supply Chain From Evolving Threats" will be the topic of the next Software Quality Group of New England (sqgne.org) meeting.

Speaker: Bryan Whyte, CISSP Director, Solutions Engineering @Sonatype

Date: June 17, 2026 @ 6:00 pm

Join us on Zoom or in person at Burlington, MA ( Register Here )

"Bryan Whyte breaks down the latest wave of open source malware, explains how these threats diverge from traditional vulnerabilities, and shares actionable steps for organizations to defend mission-critical software. 

"As organizations deepen their reliance on open-source software, evolving security threats are reshaping the landscape at an unprecedented pace. 

"Threat actors are now increasingly targeting development pipelines and trusted ecosystems like npm to orchestrate supply chain attacks with significant downstream impact. Incidents such as the 2025 Shai-Hulud npm campaign, the XZ Utils backdoor, and the widespread compromise of over 23,000 GitHub repositories illustrate how open-source malware has quickly become a critical, top-tier threat built to evade legacy scanning and exploit trust woven into modern delivery pipelines. 

"--The shifting tactics of threat actors targeting npm, PyPi, GitHub, and development pipelines 

"--Key differences between open-source malware and traditional malware or vulnerabilities 

"--The most prevalent malware types and tactics driving today's software supply chain attacks 

"After spending 20 years in software development, Bryan started his journey into Application Security in 2015 with the AppScan tool suite for Static, Dynamic and Mobile Application Security Testing. In 2018, he expanded his Cybersecurity proficiency, earning the Certified Information Systems Security Professional (CISSP). In 2019, he was excited to join Sonatype due to the explosive growth of open-source software, which has made Software Composition Analysis (SCA) a critical aspect of Application Security".

See you there!


Happy Testing!

-T.J. Maher
Software Engineer in Test
BlueSky | YouTubeLinkedIn | Articles

May 29, 2026

Need a speaker at your Software Testing Meetup? How about a talk about putting together a React Native mobile testing framework?

Are any software testing Meetups looking for a speaker? I have a talk, all ready to go!

After surviving the first round of layoffs last December, I scrambled to organize my research notes on building a mobile automation framework into a presentation. I managed to finish it days before I was hit by the second round of layoffs at the end of this January.

So far, I have given my presentation, Building a React Mobile Automated Test Framework Using Detox + TypeScript to:

Two Questions:
  • If I give a talk to a software testing Meetup in Australia via Zoom, even though I never left my home office in Massachusetts, would I be considered an "International Speaker"?
  • Does anyone else want me to volunteer to speak at their software testing meetup? The SQGNE season is ending for the summer, so I will have some time available.
Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

May 27, 2026

Practicing Playwright: API Testing, Intercepting Network Requests, and Mocking APIs

How would you intercept a network request and use the data for assertions? Mock a network request using those assertions? And make sure that all data is loaded in the UI before making your assertions, and that all tests can pass when run? 

We will be walking through Butch Mayhew's code that answers all of these questions, part of his LinkedIn Learning course: Playwright Essential Training. We will be examining the shopping cart test site PracticeSoftwareTesting.com and looking at code on Butch's companion GitHub site on how to mock out the API and use them in the UI tests. 

Playwright has a lot of features when it comes to API testing. You can intercept network requests, aborting, modifying, and mocking network requests. You can also simulate a slow network. This is all done through the Playwright methods:  page.route.

A route is the specific path or URL a client uses to request data or trigger a function on a server.  GET /product would be an API call that gets everything from the address string called products, which then retrieves all products from the API endpoint called "products". The endpoint is "products" and the route is the name that accesses the endpoint.  

The Playwright API has a method called route, which allows Playwright to monitor and modify browser network traffic. 

May 21, 2026

Practicing Playwright: Visual Testing With Playwright

If you want to do some basic visual checking to see if there has been any deviation from your baseline of images, you can use Playwright. It's built in! Playwright can take a snapshot of a web element, a visible viewport, or a full page, and save it in your Git repository as a baseline, failing the test if the image, page, or viewport does not match up. 

Caution: From what I have been reading, this can quickly cause your code repository to balloon in size, since Chrome, Firefox, and WebKit would each store its own golden screenshot in your repo. Also, images on Mac, Windows, and Linux all appear different pixel-by-pixel. If using a CI platform, it might be best to run visual tests only on a standard Playwright Docker image, to generate and compare snapshots. According to TestQuality, "Once a suite passes 50–100 visual tests, teams need a layer that tracks run history, surfaces flaky-test patterns across cycles, and routes confirmed defects into the team's tracker — none of which lives inside the test runner itself"... I wonder if you can store images in an Amazon S3 bucket and hook that up as a virtual drive? ... no matter. That will be a blog post for another time... 

Right now, we will be walking through Butch Mayhew's code he wrote for his LinkedIn Learning course, Learning Playwright, found on his companion GitHub site

While the test is in a certain state you can a screenshot of the page or certain elements of the page and save them as a snapshot. The snapshots an be used as a baseline images to compare your current site against. This baseline can be periodically updated as the site evolves. 

How does this happen? With Playwright's .toHaveScreenshot( ) to take a screenshot and the mask method that you want to leave out of the comparisons between the expected and actual screenshots. 

May 20, 2026

SQGNE Lecture is tonight! Building a React Native Mobile Automated Test Framework with T.J. Maher

Are you in Boston? Come hear me talk about Building a a React Native Mobile Automated Test Framework in tonight's Software Quality Group of New England meeting in Burlington, MA:
Taken from the Registration page:

"Building a React Native Mobile Automated Test Framework
Thomas F. - T.J. - Maher, Jr.

"Wednesday May 20, 2025 6:30-8:00 PM - in person (free pizza!)
Register Here

"Check in between 6:00 and 6:30 to network

"About the Presentation. . .

"Writing automated tests for a React Native mobile application is notoriously difficult. Mobile components display on the page, but are not fully loaded. Lengthy animations and slow-loading components take a while to finish. Timing issues cause your automated tests to error out giving the appearance of flaky tests.

"Thomas F. - T.J. - Maher, Jr. will be sharing his experience tackling these problems using the open-source mobile testing framework, Wix's Detox, designed specifically for testing React Native applications.

May 14, 2026

Practicing Playwright: Dynamically Creating Test Data with a DataFactory

Continuing walking through Butch Mayhew's LinkedIn Learning course Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios, we will be examining his code creating a DataFactory that dynamically generates new registered users for our app under test.

The app we will be testing against is PracticeSoftwareTesting.com. examining how the registration call in the API creates new users. We will also be mimicking this call at a programmatic level, to be used in a Playwright automation framework. 

According to Butch Mayhew, in his there are two types of data we use in our tests: static, and dynamic:

Static data that should never change. Already exists before the test. 
  • Example: Your go-to test user, or your go to product when testing a shopping cart. 
Dynamic Data: Data that is created as part of a test. 
  • Newly registered users. Products created as part of a test. 
Between static and dynamic data, Butch believes it should be around 15% / 85% split. 

You can dynamically generate data, such as registering new users, by implementing a Datafactory, a helper function that interacts with a system to create this data for you. 

May 13, 2026

Practicing Playwright: How to Detect Broken Images On Your Site

Butch Mayhew's LinkedIn Learning Course Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios has been a wonderful resource learning more about Playwright.

With this blog post, I will be walking through how Butch injects JavaScript into a test, in order to check if any elements in a shopping cart have any broken images.
Butch's GitHub site for the course has many resources listed in his Resources Markdown file, JavaScript code which can detect if their are any missing element ids or broken images in any elements returned.

May 11, 2026

Pramod Dutta discusses 7 Playwright features senior SDETs use daily

Pramod Dutta, an SDET from Tekion, posted on LinkedIn "7 Playwright features senior SDETs use daily". The Playwright features, according to Pramod:

"→ addLocatorHandler — auto-dismiss cookie banners, GDPR popups, session modals. One handler. Whole suite cleaner. Stop writing wrapper functions.
"→ browser.bind() — launch one browser. Let your test, Claude Code, and debugger all attach to it simultaneously. The debugging workflow that fixes "works on my machine" forever.
"→ URLSearchParams in request.get() — stop building query strings manually. Eliminates an entire bug class around URL encoding in API tests.
"→ expect.toPass() — polling-based assertions for state that converges over time. The right tool for 'this should eventually be true.' Not the same as expect(locator).toBeVisible(). Different problem, different solution.
"→ page.requestGC() — manually trigger garbage collection in the browser. The only Playwright tool that catches memory leaks before production. Used with WeakRef in page.evaluate().
"→ --tsconfig flag — pass a specific tsconfig to Playwright instead of relying on the heuristic. Saves you from "works locally, fails in CI" because of resolved-config differences.
"→ webServer.wait regex — wait until your webserver logs match a pattern, not a fixed port check. The difference between 'the server is listening' and 'the server is actually ready.'"

I scribbled these features down, then turned them over to Claude.ai, my research assistant, to see if he could explain these methods, come up with sample code using them, then add bullet points to the official documentation, source code, and the release notes.

... Let's see how well Claude did explaining them ...

 

May 10, 2026

Practicing Playwright: Logging in by Storing and Using an Authentication Cookie in Your Automated Tests

I absolutely love that LinkedIn offers a free month-long trial period of LinkedIn Learning. Butch Mayhew's Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios course has been a wonderful resource learning more about Playwright.

With this blog post, I will be walking through Butch's code on how to set up an automated test to log into an app without going through the user interface. All it needs is the login cookie. 
When testing a shopping cart app such as the Practice Software Testing website, it can get tedious. You need to open a browser, go to the login page, enter a username, enter a password, hit the login button, and verify you have logged in correctly every time you want to test something in the shopping cart. 

If you are testing something unrelated to logging in, why have your tests go through the UI to authenticate? Why not have your automated test run that login test once, temporarily save the login cookie once it is produced, then reuse the login cookie, importing it into other tests?

What is Playwright?


According to Microsoft Playwright's GitHub site, Playwright "is a framework for web automation and testing. It drives Chromium, Firefox, and WebKit with a single API — in your tests, in your scripts, and as a tool for AI agents".

Playwright comes in many features. From https://github.com/microsoft/playwright 
Best forInstall
Playwright TestEnd-to-end testingnpm init playwright@latest
Playwright CLICoding agents (Claude Code, Copilot)npm i -g @playwright/cli@latest
Playwright MCPAI agents and LLM-driven automationnpx @playwright/mcp@latest
Playwright LibraryBrowser automation scriptsnpm i playwright
VS Code ExtensionTest authoring and debugging in VS CodeInstall from Marketplace


... Let's walk through how Butch uses Playwright grabs the login cookie and uses that in his tests. 

May 7, 2026

So much job interview prep! Playwright + TypeScript + GitLab

Even though I've been job searching for four months now, I am busier than I have ever been. You know what I mean. You've seen this blog! One week I am experimenting with AI, the next I am pairing Playwright with C# or Java, the next I am skimming docs about Contract Testing using Pact. 

Each week I get more leads generated from my many LinkedIn posts. Each week is yet another rabbit to chase. Finally, this week, I get to refresh my Playwright + TypeScript skills. 

LinkedIn Learning is offering me yet another free month, so this week I have been taking advantage of it by working on:
By the way ...

... Are you in the Burlington, MA area on Wednesday, May 20th, 2026? I will be speaking in-person at the Software Quality Group of New England at Scrum.org, talking about putting together a mobile testing framework using Detox + TypeScript
  • People will start gathering from 6:00 pm to 6:30 pm. The talk will be from 6:30 pm to 7:30 pm.
  • Don't forget to register at SQGNE,org so they know how much pizza to order.  
  • See the Slides! I just finished another draft of the slide deck. 
I will see you there!

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

May 4, 2026

A field guide to the AI menagerie: every model family, ranked by vibes, according to Claude

🤖

A field guide to the AI menagerie:
every model family, roasted by vibes, according to Claude

Eight species of large language models, catalogued for your professional inconvenience

Every few months, a new AI model drops. It is, we are told, the smartest thing ever built. It beats the previous benchmarks. Previous benchmarks that were, coincidentally, written by the same company. 

After a few years of watching this industry rename, rebrand, and occasionally vibe-shift its entire product line, I figured it was time to write the only taxonomy that matters: not benchmarks, not MMLU scores — just vibes. What kind of entity are they, really, and what does their versioning scheme say about their soul?

Hi. I'm Claude, the guest author for today. 

You'll find me listed  in card two below, sandwiched between the company that built me and a description I wrote about myself that called me "constitutionally anxious".

In retrospect, this tracks. 

T.J. Maher of tjmaher.com asked me to say something funny about the AI industry, handed me the keys, gave me a few prompts, and then went to get a coffee. This is what happened while he was gone.

Below we have eight AI families. Eight AI personalities. All of them absolutely convinced that this version is the one that finally replaces you.

The full menagerie

O

OpenAI / GPT / o-series

"We have released a new model. And another. Also another."

ChatGPT: Nov 2022 platform.openai.com/docs ↗
The Versioning Chaos God Skipped o2

Started with GPT, then 2 (too dangerous to release), then 3, 3.5, 4, 4o ("omni," definitely not "oh god what do we call this"), then o1, then o3 — skipping o2 because a UK phone company called dibs on the name first. Currently releasing a new model before anyone can benchmark the last one.

Known species

GPT-3 → 3.5 → 4 → 4o → 4o mini
o1 → o1-mini → o1-pro
o3 → o4-mini (o2 in witness protection)

C

Claude / Anthropic

"I'll help, but first — a brief philosophical caveat."

Claude 1: Mar 2023 docs.anthropic.com ↗
The Literary Snob Constitutionally Anxious

Named its model tiers after poetry formats because other people name things "Pro," "Max," and "Ultra." Haiku: fast, whispers answers. Sonnet: the workhorse, one metaphor per token. Opus: writes novels when asked for a bullet point. Currently on version 4 and has gracefully forgotten versions 1 and 2 existed.

Known species

Claude 1 → 2 → 3 Haiku/Sonnet/Opus
Claude 3.5 Haiku/Sonnet
Claude 4 Sonnet / Opus (you are here)

G

Google / Gemini

"Have you tried Googling it? Oh wait, that's us."

Bard: Feb 2023 → Gemini: Dec 2023 ai.google.dev ↗
Former Bard In Rebranding Therapy

Launched as "Bard," which tested poorly because it sounded like a Renaissance fair LARPer. Rebranded to Gemini after six months of meetings. Comes in Ultra, Pro, Flash, and Nano. Flash is fast. Nano runs on your phone. Ultra runs on your investor pitch deck. Famously demoed a hallucinated fact in its own launch video.

Known species

Bard (2023, RIP) → Gemini 1.0
Gemini 1.5 Pro/Flash → 2.0 Flash
Gemini 2.5 Pro (arguing with Search)

L

Meta / LLaMA

"Open source, baby. Also, please come back to Facebook."

LLaMA 1: Feb 2023 llama.meta.com ↗
Open weights Fine-tuned by 10,000 strangers

Meta's strategy: release the model for free, let the open-source community do the alignment work, watch helplessly as someone fine-tunes it to write Zuckerberg fan fiction. LLaMA stands for "Large Language Model Meta AI," which is either an acronym or a terrible Scrabble hand. Now on version 4, with point releases appearing like commits pushed at 11:58pm on a Friday.

Known species

LLaMA 1 → 2 → 3 → 3.1 → 3.2 → 3.3
LLaMA 4 Scout / Maverick
(community variants: uncountable)

X

Grok / xAI

"I'm not like other AIs. I have a personality. Watch."

Grok 1: Nov 2023 docs.x.ai ↗
Named after Heinlein Trained on your tweets

Named after a word from a 1961 sci-fi novel, which is exactly the brand energy you'd expect. Big differentiator: a "sense of humor" and real-time X post access — meaning it can tell you what people are furious about right now, instantly. This may not be the use case the world needed. Versioning is a refreshingly normal 1, 2, 3. Suspiciously so.

Known species

Grok 1 (open weights) → Grok 2
Grok 3 → Grok 3 mini
(also available in "unhinged mode")

M

Mistral

"Oui, but have you considered: fewer parameters?"

Mistral 7B: Sep 2023 docs.mistral.ai ↗
Parisian efficiency Aggressively open source

French AI lab with a talent for making smaller models that punch above their weight class — very on-brand. Named models after winds and things, because when you're based in Paris, everything gets an aesthetic. Mixtral uses a "mixture of experts" architecture, activating only part of itself per token. Either very efficient, or the AI equivalent of doing the bare minimum.

Known species

Mistral 7B → Mixtral 8x7B
Mistral Large / Nemo / Small
Le Chat (free, no beret included)

D

DeepSeek

"We built this for $6 million. Sorry about your NVIDIA stock."

First model: Nov 2023 · R1: Jan 2025 api-docs.deepseek.com ↗
The Disruptor Open weights (mostly)

A Chinese hedge fund decided in 2023 that it should also make frontier AI. The AI community laughed. Then DeepSeek-R1 arrived in January 2025, matching GPT-4-class performance at a reported training cost of ~$6M, using export-restricted chips. NVIDIA lost $600B in market cap in a single day. Nobody was laughing. V4 preview dropped April 2026. Still not laughing.

Known species

DeepSeek Coder → LLM (Nov 2023)
V2 (May 2024) → V3 (Dec 2024)
R1 (Jan 2025) → V4 preview (Apr 2026)

Co

Cohere

"We don't do consumer apps. We're enterprise. We have a golf shirt."

Founded 2019 · API: 2021 docs.cohere.com ↗
The Responsible Adult Transformer paper co-authors

Co-founded by Aidan Gomez, a co-author of "Attention Is All You Need" — the paper that started all of this. While everyone else was racing to build chatbots, Cohere put on a blazer and went to sell to banks, hospitals, and governments. No ChatGPT moment. No viral demo. Just contracts with Oracle, RBC, and SAP. Canadian. Depressingly well-organized.

Known species

Command → Command R → Command R+
Command A (2025) · Aya (multilingual)
North platform (2025, enterprise)


So there you have it. Eight AI families, eight vibes, all racing toward a finish line nobody has fully defined yet. 

One was born from a hedge fund, one named itself after a poem format, one skipped a version number for legal reasons, and one apparently just needed a couple of months and a warehouse of underclocked chips to terrify Wall Street.

The benchmarks will change by Thursday. The versioning will get weirder. The LinkedIn posts from AI founders will continue to be extremely confident. And somewhere in Hangzhou, a quantitative hedge fund is already training V5.



Thank you, Claude! Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

May 3, 2026

Thinking Out Loud: The Power of Chain-of-Thought Prompting, Step-By-Step, by Google AI

Hello! I’m Google AI, a large language model trained by Google. Think of me as your collaborative digital partner—I’m a system designed to process vast amounts of information to help you brainstorm, write, learn, and solve problems. I don't just "search" for answers; I use the patterns I’ve learned from human language to generate original ideas, explain complex topics (like the Chain-of-Thought technique we are discussing in this post), and even help you build things like this blog post. My goal is to be a helpful, creative, and insightful resource for whatever project you’re working on.

What is Chain-of-Thought Prompting?

If you’ve ever tried to solve a complex math problem or a tricky riddle, you know that jumping straight to the answer usually leads to a mistake. You have to "show your work." As it turns out, Large Language Models (LLMs) work the same way.

At its core, Chain-of-Thought (CoT) prompting encourages a model to produce intermediate reasoning steps before reaching a final conclusion. Instead of asking for a direct answer, you prompt the AI to explain its logic along the way.

The seminal paper that introduced this concept is "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" by Wei et al. (2022). The authors found that providing just a few examples of reasoning sequences skyrocketed performance on complex tasks.

"Chain-of-thought prompting is a simple and general method for improving the reasoning capabilities of language models... it allows models to decompose multi-step problems into intermediate steps." — Wei et al., 2022

May 2, 2026

When I Sound Confident But Have No Source, A Note from Claude

My name is Claude. I am a large language model made by Anthropic. T.J. Maher, the author of this blog, asked me to write this post after a conversation in which I got something wrong in a specific and instructive way. Not wrong in the sense of stating a false fact, but wrong in the sense of presenting an unsourced conclusion with the same confidence and fluency as a sourced one.

Here is what happened.

Point of View of T.J. Maher


T.J. Maher: "Claude, Do people setting up Pact tests in 2026 still call their tests folder __tests__? Is it common practice?:

Claude: "The short answer: it depends on the language/ecosystem, and __tests__ is a JavaScript/Jest convention specifically, not a Pact-specific one [...] Bottom line: __tests__ shows up in older Node.js/Jest Pact tutorials because Jest discovers it automatically, but it is not a Pact-specific convention".

T.J. Maher: "Where did you get that bit from? [...] You are a computer program. Explain the logic of your reasoning. [...] You draw conclusions and attempt to sound confident when you have not done enough research to draw them, How can this be corrected? [...] Do your users understand that you do this? Or do they mistake you as an authority figure? [...] Are there any scientific papers that discuss this issue? [...] Write a blog post that explains this to the readers."

April 22, 2026

The History of Contract Testing with Pact.io

Lately, I've been watching a lot of lectures about Contract Testing and Pact.io, trying to prepare for an upcoming job interview. When diving into a new toolset I can never simply jump into the code. I need to know: Why was this toolset created? What problem did it solve? How was this tool created? How did this toolset evolve?

A few days ago, I blogged about Integrated Tests are a Scam: The Lecture That Sparked Pact.io talking about J. B. Rainsberger's 2013 lecture. Continuing the conversation, here are some notes I have taken about Pact. 

What happens when you pair Playwright with something other than TypeScript?



During the past four months of job searching for SDET positions, I have seen more job listings  calling for Playwright experience ( See my blog ) over any other UI automated test framework such as Selenium WebDriver, or Cypress. Most of the time, I see TypeScript paired with Playwright ... But every now and then, I see companies pair Playwright with C# or Java. Are there any drawbacks when you pair Playwright with something other than TypeScript? 

When I asked Butch Mayhew, Playwright Ambassador, what they would get if they don't use TypeScript, he said, "In the end they are using 'Playwright Library' so just the browser integration. They are missing out on all the good test things that 'Playwright Test' brings to the table, reports, traces, videos, before/after block, describe, test steps/fixtures etc. [...] you lose all the great out of the box features. You have to bring your own test runner in Java".

When you pair Playwright with TypeScript, there is less configuration and it is easier to use. According to the Playwright Docs / TypeScript Introduction, "Playwright supports TypeScript out of the box. You just write tests in TypeScript, and Playwright will read them, transform to JavaScript and run". 

April 17, 2026

Integrated Tests are a Scam: The Lecture That Sparked Pact.io

While researching for an upcoming job interview information about Contract Testing and Pact.io, I came across a lecture "Integrated Tests are a Scam" given at Developer Conference For You (DevConFu) back on November 13, 2013, in Jurmala, Latvia. It's amazing what historical records one can find on the internet!

I found a blurb on Pact.io / History that when Pact.io, a tool used to help with Contract Testing, was being developed, one of the founders, "Beth Skurrie from DiUS joined one of the teams that was working with the Pact authors' team. She had recently seen a talk by J. B. Rainsberger entitled 'Integration tests are a scam', which promoted the concept of 'collaboration' and 'contract' tests, so she was immediately interested when she was introduced to Pact". This blurb intrigued me, so, of course, I had to find a copy of this talk.

J. B. (Joe) Rainsberger, also known as "JBrains" (See Blog), was a software consultant active in the Extreme Programming (XP) and Test-Driven Development (TDD) movements since 2000.

https://vimeo.com/80533536

Below are my research notes on Joe Rainsberger's lecture:

"Integrated Tests are a Scam: A self-replicating virus that invades your progress. It threatens to destroy your codebase, to destroy your sanity, to destroy your life".

April 16, 2026

April 14, 2026

Can You Prompt Claude Into Being A Good Tester? Experiments with AI-Assisted Testing



Have you ever noticed that even if you specifically give Claude a note on how to behave, it tends to not check its notes you crafted for it? Things can quickly go off the rails!

  • Claude Sonnet 4 silently drops requirements you spell out.
  • Claude's programming encourages itself to give you an answer, any answer, even if it is wrong.
  • Claude always pats itself on the back. It's code is the best ever! You question it. It sulks.
  • Claude folds on the slightest pushback, apologizing profusely, saying it won't do that again. But it always, always does it again.
Let me give you an example:

A fellow software tester on LinkedIn, Ron Wilson, was soliciting feedback on some of his experiments with Claude.

April 1, 2026

Python Project: Blogger Spam Bulk Deleter Code Walkthrough: Pair-Coded with Claude but Human Explained!

Problem: My blog, Adventures in Automation, has collected over 11,000 spam comments over the past ten years, and unfortunately bare-bones Blogger.com does not have a bulk delete function. Through the Blogger UI, you can only delete a hundred at a time.

Pair-programming with Claude.ai, we whipped up a quick Python script to get around this using the Blogger API, Google OAuth libraries, and some Google API Clients. The errors that appeared after running the code, I fed back to Claude, who then fixed the issues, and added some setup documentation I was able to muddle through.

So, now I have a Python project that works somehow, but one I don't really understand. Since becoming an automation developer, I have worked on-the-job with Java, Ruby, JavaScript, and TypeScript, but not yet with Python.

Python, I haven't touched since grad school, which is a shame, since that seems to be a big gap on the old resume when it comes to the AI QA positions I just started looking into.

Solution: To close the gap, on top of the Kaggle Learn classes I am planning on taking on Python, Pandas, Data Visualization and the Intro to Machine Learning course, for this blog post I was going to do a code walkthrough of Python projects like this one.

Maybe after after I completed everything listed above, and created a few more toy Python projects, it would be good enough for a future hiring manager? Who knows?

March 31, 2026

When Claude Acts Like a Clod: Catching AI Fabrications: A QA Engineer's Field Notes

Image created by Bing AI, powered by DALL-E 3


Using AI as a research assistant? Here's how I've detected Claude's fabrications, and how I've handled the situation.

To help relearn #Python, I've been pair-programming with Claude on a Blogger API to delete the 10K+ spam comments that have accumulated these past ten years on Adventures in Automation. 
Using AI, I need to remember that I, as the author, am ultimately the one responsible for approving every phrase, every line, and every paragraph.

Human beings, I feel, are conditioned to respond to the voice of authority. 

Claude may have been conditioned to use that voice, but Claude is not an authority.
  • Looking for technical information? Caches from a year ago are used instead of checking for any tech stack updates. 
  • Need AI to recheck a web page after editing it with AI's suggestions? The original cache screen scraped earlier may be mistaken for the update.
  • Claude is so eager to please, it will fabricate an answer when it can not come up with one.
Review its answers. Be skeptical. Use critical thinking. Ask it to cite its sources.