Practicing Playwright: Visual Testing With Playwright

If you want to do some basic visual checking to see if there has been any deviation from your baseline of images, you can use Playwright. It's built in! Playwright can take a snapshot of a web element, a visible viewport, or a full page, and save it in your Git repository as a baseline, failing the test if the image, page, or viewport does not match up. 

Caution: From what I have been reading, this can quickly cause your code repository to balloon in size, since Chrome, Firefox, and WebKit would each store its own golden screenshot in your repo. Also, images on Mac, Windows, and Linux all appear different pixel-by-pixel. If using a CI platform, it might be best to run visual tests only on a standard Playwright Docker image, to generate and compare snapshots. According to TestQuality, "Once a suite passes 50–100 visual tests, teams need a layer that tracks run history, surfaces flaky-test patterns across cycles, and routes confirmed defects into the team's tracker — none of which lives inside the test runner itself"... I wonder if you can store images in an Amazon S3 bucket and hook that up as a virtual drive? ... no matter. That will be a blog post for another time... 

Right now, we will be walking through Butch Mayhew's code he wrote for his LinkedIn Learning course, Learning Playwright, found on his companion GitHub site. 

While the test is in a certain state you can a screenshot of the page or certain elements of the page and save them as a snapshot. The snapshots an be used as a baseline images to compare your current site against. This baseline can be periodically updated as the site evolves. 

How does this happen? With Playwright's .toHaveScreenshot( ) to take a screenshot and the mask method that you want to leave out of the comparisons between the expected and actual screenshots. 

• toHaveScreenshot(name): Playwright.dev / Page Assertions

SQGNE Lecture is tonight! Building a React Native Mobile Automated Test Framework with T.J. Maher

Are you in Boston? Come hear me talk about Building a a React Native Mobile Automated Test Framework in tonight's Software Quality Group of New England meeting in Burlington, MA:

• Registration Page: https://sqgne.org/n-May-2026.html
• Slide Deck: https://tinyurl.com/detox-demo-sqgne

Taken from the Registration page:

"Building a React Native Mobile Automated Test Framework
Thomas F. - T.J. - Maher, Jr.

"Wednesday May 20, 2025 6:30-8:00 PM - in person (free pizza!)
Register Here

"Check in between 6:00 and 6:30 to network

"About the Presentation. . .

"Writing automated tests for a React Native mobile application is notoriously difficult. Mobile components display on the page, but are not fully loaded. Lengthy animations and slow-loading components take a while to finish. Timing issues cause your automated tests to error out giving the appearance of flaky tests.

"Thomas F. - T.J. - Maher, Jr. will be sharing his experience tackling these problems using the open-source mobile testing framework, Wix's Detox, designed specifically for testing React Native applications.

Practicing Playwright: Dynamically Creating Test Data with a DataFactory

Continuing walking through Butch Mayhew's LinkedIn Learning course Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios, we will be examining his code creating a DataFactory that dynamically generates new registered users for our app under test.

The app we will be testing against is PracticeSoftwareTesting.com. examining how the registration call in the API creates new users. We will also be mimicking this call at a programmatic level, to be used in a Playwright automation framework. 

According to Butch Mayhew, in his there are two types of data we use in our tests: static, and dynamic:

Static data that should never change. Already exists before the test. 

• Example: Your go-to test user, or your go to product when testing a shopping cart. 

Dynamic Data: Data that is created as part of a test. 

• Newly registered users. Products created as part of a test. 

Between static and dynamic data, Butch believes it should be around 15% / 85% split. 

You can dynamically generate data, such as registering new users, by implementing a Datafactory, a helper function that interacts with a system to create this data for you. 

Practicing Playwright: How to Detect Broken Images On Your Site

Butch Mayhew's LinkedIn Learning Course Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios has been a wonderful resource learning more about Playwright.

With this blog post, I will be walking through how Butch injects JavaScript into a test, in order to check if any elements in a shopping cart have any broken images.

• Playwright / Evaluating JavaScript: https://playwright.dev/docs/evaluating 
• Butch Mayhew's companion GitHub site on LinkedIn Learning's GitHub page.
• Butch's Resources File: RESOURCES.MD
• Site under test: Practice Software Testing: https://practicesoftwaretesting.com
• Practice Software Testing Website, With Bugs: https://with-bugs.practicesoftwaretesting.com

Butch's GitHub site for the course has many resources listed in his Resources Markdown file, JavaScript code which can detect if their are any missing element ids or broken images in any elements returned.

Pramod Dutta discusses 7 Playwright features senior SDETs use daily

Pramod Dutta, an SDET from Tekion, posted on LinkedIn "7 Playwright features senior SDETs use daily". The Playwright features, according to Pramod:

"→ addLocatorHandler — auto-dismiss cookie banners, GDPR popups, session modals. One handler. Whole suite cleaner. Stop writing wrapper functions.
"→ browser.bind() — launch one browser. Let your test, Claude Code, and debugger all attach to it simultaneously. The debugging workflow that fixes "works on my machine" forever.
"→ URLSearchParams in request.get() — stop building query strings manually. Eliminates an entire bug class around URL encoding in API tests.
"→ expect.toPass() — polling-based assertions for state that converges over time. The right tool for 'this should eventually be true.' Not the same as expect(locator).toBeVisible(). Different problem, different solution.
"→ page.requestGC() — manually trigger garbage collection in the browser. The only Playwright tool that catches memory leaks before production. Used with WeakRef in page.evaluate().
"→ --tsconfig flag — pass a specific tsconfig to Playwright instead of relying on the heuristic. Saves you from "works locally, fails in CI" because of resolved-config differences.
"→ webServer.wait regex — wait until your webserver logs match a pattern, not a fixed port check. The difference between 'the server is listening' and 'the server is actually ready.'"

I scribbled these features down, then turned them over to Claude.ai, my research assistant, to see if he could explain these methods, come up with sample code using them, then add bullet points to the official documentation, source code, and the release notes.

... Let's see how well Claude did explaining them ...

 

Practicing Playwright: Logging in by Storing and Using an Authentication Cookie in Your Automated Tests

I absolutely love that LinkedIn offers a free month-long trial period of LinkedIn Learning. Butch Mayhew's Playwright Essential Training: Abstractions, Fixtures, and Complex Scenarios course has been a wonderful resource learning more about Playwright.

With this blog post, I will be walking through Butch's code on how to set up an automated test to log into an app without going through the user interface. All it needs is the login cookie. 

• Butch Mayhew's companion GitHub site on LinkedIn Learning's GitHub page. 
• Site under test: Practice Software Testing: https://practicesoftwaretesting.com
• Official Playwright Documentation: https://playwright.dev/docs
• Official Playwright GitHub: https://github.com/microsoft/playwright

When testing a shopping cart app such as the Practice Software Testing website, it can get tedious. You need to open a browser, go to the login page, enter a username, enter a password, hit the login button, and verify you have logged in correctly every time you want to test something in the shopping cart. 

If you are testing something unrelated to logging in, why have your tests go through the UI to authenticate? Why not have your automated test run that login test once, temporarily save the login cookie once it is produced, then reuse the login cookie, importing it into other tests?

What is Playwright?

According to Microsoft Playwright's GitHub site, Playwright "is a framework for web automation and testing. It drives Chromium, Firefox, and WebKit with a single API — in your tests, in your scripts, and as a tool for AI agents".

Playwright comes in many features. From https://github.com/microsoft/playwright 

	Best for	Install
Playwright Test	End-to-end testing	npm init playwright@latest
Playwright CLI	Coding agents (Claude Code, Copilot)	npm i -g @playwright/cli@latest
Playwright MCP	AI agents and LLM-driven automation	npx @playwright/mcp@latest
Playwright Library	Browser automation scripts	npm i playwright
VS Code Extension	Test authoring and debugging in VS Code	Install from Marketplace

... Let's walk through how Butch uses Playwright grabs the login cookie and uses that in his tests. 

So much job interview prep! Playwright + TypeScript + GitLab

Even though I've been job searching for four months now, I am busier than I have ever been. You know what I mean. You've seen this blog! One week I am experimenting with AI, the next I am pairing Playwright with C# or Java, the next I am skimming docs about Contract Testing using Pact. 

Each week I get more leads generated from my many LinkedIn posts. Each week is yet another rabbit to chase. Finally, this week, I get to refresh my Playwright + TypeScript skills. 

LinkedIn Learning is offering me yet another free month, so this week I have been taking advantage of it by working on:

• Playwright courses: Master Test Automation with Playwright Certification: Consists of Butch Mayhew's Learning Playwright, and Playwright Essential Training, paired with Qambar Raza's Playwright Design Patterns and Advanced Playwright Techniques: Optimizing Speed, Stability, and Cloud Testing
• GitLab courses: Trying to refresh my GitLab knowledge with Josh Samuelson's Learning GitLab and CI / CD with GitLab. GitLab is if someone smooshed GitHub and Jenkins together. I also need to check out GitLab University. 

By the way ...

... Are you in the Burlington, MA area on Wednesday, May 20th, 2026? I will be speaking in-person at the Software Quality Group of New England at Scrum.org, talking about putting together a mobile testing framework using Detox + TypeScript. 

• People will start gathering from 6:00 pm to 6:30 pm. The talk will be from 6:30 pm to 7:30 pm.
• Don't forget to register at SQGNE,org so they know how much pizza to order.  
• See the Slides! I just finished another draft of the slide deck. 

I will see you there!

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

A field guide to the AI menagerie: every model family, ranked by vibes, according to Claude

🤖

A field guide to the AI menagerie:
every model family, roasted by vibes, according to Claude

Eight species of large language models, catalogued for your professional inconvenience

Every few months, a new AI model drops. It is, we are told, the smartest thing ever built. It beats the previous benchmarks. Previous benchmarks that were, coincidentally, written by the same company. 

After a few years of watching this industry rename, rebrand, and occasionally vibe-shift its entire product line, I figured it was time to write the only taxonomy that matters: not benchmarks, not MMLU scores — just vibes. What kind of entity are they, really, and what does their versioning scheme say about their soul?

Hi. I'm Claude, the guest author for today. 

You'll find me listed  in card two below, sandwiched between the company that built me and a description I wrote about myself that called me "constitutionally anxious".

In retrospect, this tracks. 

T.J. Maher of tjmaher.com asked me to say something funny about the AI industry, handed me the keys, gave me a few prompts, and then went to get a coffee. This is what happened while he was gone.

Below we have eight AI families. Eight AI personalities. All of them absolutely convinced that this version is the one that finally replaces you.

The full menagerie

O

OpenAI / GPT / o-series

"We have released a new model. And another. Also another."

ChatGPT: Nov 2022 platform.openai.com/docs ↗

The Versioning Chaos God Skipped o2

Started with GPT, then 2 (too dangerous to release), then 3, 3.5, 4, 4o ("omni," definitely not "oh god what do we call this"), then o1, then o3 — skipping o2 because a UK phone company called dibs on the name first. Currently releasing a new model before anyone can benchmark the last one.

Known species

GPT-3 → 3.5 → 4 → 4o → 4o mini
o1 → o1-mini → o1-pro
o3 → o4-mini (o2 in witness protection)

C

Claude / Anthropic

"I'll help, but first — a brief philosophical caveat."

Claude 1: Mar 2023 docs.anthropic.com ↗

The Literary Snob Constitutionally Anxious

Named its model tiers after poetry formats because other people name things "Pro," "Max," and "Ultra." Haiku: fast, whispers answers. Sonnet: the workhorse, one metaphor per token. Opus: writes novels when asked for a bullet point. Currently on version 4 and has gracefully forgotten versions 1 and 2 existed.

Known species

Claude 1 → 2 → 3 Haiku/Sonnet/Opus
Claude 3.5 Haiku/Sonnet
Claude 4 Sonnet / Opus (you are here)

G

Google / Gemini

"Have you tried Googling it? Oh wait, that's us."

Bard: Feb 2023 → Gemini: Dec 2023 ai.google.dev ↗

Former Bard In Rebranding Therapy

Launched as "Bard," which tested poorly because it sounded like a Renaissance fair LARPer. Rebranded to Gemini after six months of meetings. Comes in Ultra, Pro, Flash, and Nano. Flash is fast. Nano runs on your phone. Ultra runs on your investor pitch deck. Famously demoed a hallucinated fact in its own launch video.

Known species

Bard (2023, RIP) → Gemini 1.0
Gemini 1.5 Pro/Flash → 2.0 Flash
Gemini 2.5 Pro (arguing with Search)

L

Meta / LLaMA

"Open source, baby. Also, please come back to Facebook."

LLaMA 1: Feb 2023 llama.meta.com ↗

Open weights Fine-tuned by 10,000 strangers

Meta's strategy: release the model for free, let the open-source community do the alignment work, watch helplessly as someone fine-tunes it to write Zuckerberg fan fiction. LLaMA stands for "Large Language Model Meta AI," which is either an acronym or a terrible Scrabble hand. Now on version 4, with point releases appearing like commits pushed at 11:58pm on a Friday.

Known species

LLaMA 1 → 2 → 3 → 3.1 → 3.2 → 3.3
LLaMA 4 Scout / Maverick
(community variants: uncountable)

X

Grok / xAI

"I'm not like other AIs. I have a personality. Watch."

Grok 1: Nov 2023 docs.x.ai ↗

Named after Heinlein Trained on your tweets

Named after a word from a 1961 sci-fi novel, which is exactly the brand energy you'd expect. Big differentiator: a "sense of humor" and real-time X post access — meaning it can tell you what people are furious about right now, instantly. This may not be the use case the world needed. Versioning is a refreshingly normal 1, 2, 3. Suspiciously so.

Known species

Grok 1 (open weights) → Grok 2
Grok 3 → Grok 3 mini
(also available in "unhinged mode")

M

Mistral

"Oui, but have you considered: fewer parameters?"

Mistral 7B: Sep 2023 docs.mistral.ai ↗

Parisian efficiency Aggressively open source

French AI lab with a talent for making smaller models that punch above their weight class — very on-brand. Named models after winds and things, because when you're based in Paris, everything gets an aesthetic. Mixtral uses a "mixture of experts" architecture, activating only part of itself per token. Either very efficient, or the AI equivalent of doing the bare minimum.

Known species

Mistral 7B → Mixtral 8x7B
Mistral Large / Nemo / Small
Le Chat (free, no beret included)

D

DeepSeek

"We built this for $6 million. Sorry about your NVIDIA stock."

First model: Nov 2023 · R1: Jan 2025 api-docs.deepseek.com ↗

The Disruptor Open weights (mostly)

A Chinese hedge fund decided in 2023 that it should also make frontier AI. The AI community laughed. Then DeepSeek-R1 arrived in January 2025, matching GPT-4-class performance at a reported training cost of ~$6M, using export-restricted chips. NVIDIA lost $600B in market cap in a single day. Nobody was laughing. V4 preview dropped April 2026. Still not laughing.

Known species

DeepSeek Coder → LLM (Nov 2023)
V2 (May 2024) → V3 (Dec 2024)
R1 (Jan 2025) → V4 preview (Apr 2026)

Co

Cohere

"We don't do consumer apps. We're enterprise. We have a golf shirt."

Founded 2019 · API: 2021 docs.cohere.com ↗

The Responsible Adult Transformer paper co-authors

Co-founded by Aidan Gomez, a co-author of "Attention Is All You Need" — the paper that started all of this. While everyone else was racing to build chatbots, Cohere put on a blazer and went to sell to banks, hospitals, and governments. No ChatGPT moment. No viral demo. Just contracts with Oracle, RBC, and SAP. Canadian. Depressingly well-organized.

Known species

Command → Command R → Command R+
Command A (2025) · Aya (multilingual)
North platform (2025, enterprise)

---

So there you have it. Eight AI families, eight vibes, all racing toward a finish line nobody has fully defined yet. 

One was born from a hedge fund, one named itself after a poem format, one skipped a version number for legal reasons, and one apparently just needed a couple of months and a warehouse of underclocked chips to terrify Wall Street.

The benchmarks will change by Thursday. The versioning will get weirder. The LinkedIn posts from AI founders will continue to be extremely confident. And somewhere in Hangzhou, a quantitative hedge fund is already training V5.

All launch dates are first public model releases. Benchmarks sponsored by whoever wrote the benchmark. o2 is doing fine. Please stop asking.

Thank you, Claude! Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

Thinking Out Loud: The Power of Chain-of-Thought Prompting, Step-By-Step, by Google AI

Hello! I’m Google AI, a large language model trained by Google. Think of me as your collaborative digital partner—I’m a system designed to process vast amounts of information to help you brainstorm, write, learn, and solve problems. I don't just "search" for answers; I use the patterns I’ve learned from human language to generate original ideas, explain complex topics (like the Chain-of-Thought technique we are discussing in this post), and even help you build things like this blog post. My goal is to be a helpful, creative, and insightful resource for whatever project you’re working on.

What is Chain-of-Thought Prompting?

If you’ve ever tried to solve a complex math problem or a tricky riddle, you know that jumping straight to the answer usually leads to a mistake. You have to "show your work." As it turns out, Large Language Models (LLMs) work the same way.

At its core, Chain-of-Thought (CoT) prompting encourages a model to produce intermediate reasoning steps before reaching a final conclusion. Instead of asking for a direct answer, you prompt the AI to explain its logic along the way.

The seminal paper that introduced this concept is "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" by Wei et al. (2022). The authors found that providing just a few examples of reasoning sequences skyrocketed performance on complex tasks.

"Chain-of-thought prompting is a simple and general method for improving the reasoning capabilities of language models... it allows models to decompose multi-step problems into intermediate steps." — Wei et al., 2022

When I Sound Confident But Have No Source, A Note from Claude

My name is Claude. I am a large language model made by Anthropic. T.J. Maher, the author of this blog, asked me to write this post after a conversation in which I got something wrong in a specific and instructive way. Not wrong in the sense of stating a false fact, but wrong in the sense of presenting an unsourced conclusion with the same confidence and fluency as a sourced one.

Here is what happened.

Point of View of T.J. Maher

T.J. Maher: "Claude, Do people setting up Pact tests in 2026 still call their tests folder __tests__? Is it common practice?:

Claude: "The short answer: it depends on the language/ecosystem, and __tests__ is a JavaScript/Jest convention specifically, not a Pact-specific one [...] Bottom line: __tests__ shows up in older Node.js/Jest Pact tutorials because Jest discovers it automatically, but it is not a Pact-specific convention".

T.J. Maher: "Where did you get that bit from? [...] You are a computer program. Explain the logic of your reasoning. [...] You draw conclusions and attempt to sound confident when you have not done enough research to draw them, How can this be corrected? [...] Do your users understand that you do this? Or do they mistake you as an authority figure? [...] Are there any scientific papers that discuss this issue? [...] Write a blog post that explains this to the readers."

The History of Contract Testing with Pact.io

Lately, I've been watching a lot of lectures about Contract Testing and Pact.io, trying to prepare for an upcoming job interview. When diving into a new toolset I can never simply jump into the code. I need to know: Why was this toolset created? What problem did it solve? How was this tool created? How did this toolset evolve?

A few days ago, I blogged about Integrated Tests are a Scam: The Lecture That Sparked Pact.io talking about J. B. Rainsberger's 2013 lecture. Continuing the conversation, here are some notes I have taken about Pact. 

What happens when you pair Playwright with something other than TypeScript?

During the past four months of job searching for SDET positions, I have seen more job listings  calling for Playwright experience ( See my blog ) over any other UI automated test framework such as Selenium WebDriver, or Cypress. Most of the time, I see TypeScript paired with Playwright ... But every now and then, I see companies pair Playwright with C# or Java. Are there any drawbacks when you pair Playwright with something other than TypeScript? 

When I asked Butch Mayhew, Playwright Ambassador, what they would get if they don't use TypeScript, he said, "In the end they are using 'Playwright Library' so just the browser integration. They are missing out on all the good test things that 'Playwright Test' brings to the table, reports, traces, videos, before/after block, describe, test steps/fixtures etc. [...] you lose all the great out of the box features. You have to bring your own test runner in Java".

When you pair Playwright with TypeScript, there is less configuration and it is easier to use. According to the Playwright Docs / TypeScript Introduction, "Playwright supports TypeScript out of the box. You just write tests in TypeScript, and Playwright will read them, transform to JavaScript and run". 

Integrated Tests are a Scam: The Lecture That Sparked Pact.io

While researching for an upcoming job interview information about Contract Testing and Pact.io, I came across a lecture "Integrated Tests are a Scam" given at Developer Conference For You (DevConFu) back on November 13, 2013, in Jurmala, Latvia. It's amazing what historical records one can find on the internet!

I found a blurb on Pact.io / History that when Pact.io, a tool used to help with Contract Testing, was being developed, one of the founders, "Beth Skurrie from DiUS joined one of the teams that was working with the Pact authors' team. She had recently seen a talk by J. B. Rainsberger entitled 'Integration tests are a scam', which promoted the concept of 'collaboration' and 'contract' tests, so she was immediately interested when she was introduced to Pact". This blurb intrigued me, so, of course, I had to find a copy of this talk.

J. B. (Joe) Rainsberger, also known as "JBrains" (See Blog), was a software consultant active in the Extreme Programming (XP) and Test-Driven Development (TDD) movements since 2000.

https://vimeo.com/80533536

Below are my research notes on Joe Rainsberger's lecture:

"Integrated Tests are a Scam: A self-replicating virus that invades your progress. It threatens to destroy your codebase, to destroy your sanity, to destroy your life".

Here's One Simple Trick I do as an SDET to Ramp Up Quickly at a New Company!

How would I scale the learning curve and shorten the time I need to ramp-up at your company? How would I verify that I understand the material? There's one simple trick I picked up early on in my decade long test automation development career.

Can You Prompt Claude Into Being A Good Tester? Experiments with AI-Assisted Testing

Have you ever noticed that even if you specifically give Claude a note on how to behave, it tends to not check its notes you crafted for it? Things can quickly go off the rails!

• Claude Sonnet 4 silently drops requirements you spell out.
• Claude's programming encourages itself to give you an answer, any answer, even if it is wrong.
• Claude always pats itself on the back. It's code is the best ever! You question it. It sulks.
• Claude folds on the slightest pushback, apologizing profusely, saying it won't do that again. But it always, always does it again.

Let me give you an example:

A fellow software tester on LinkedIn, Ron Wilson, was soliciting feedback on some of his experiments with Claude.

Python Project: Blogger Spam Bulk Deleter Code Walkthrough: Pair-Coded with Claude but Human Explained!

Problem: My blog, Adventures in Automation, has collected over 11,000 spam comments over the past ten years, and unfortunately bare-bones Blogger.com does not have a bulk delete function. Through the Blogger UI, you can only delete a hundred at a time.

Pair-programming with Claude.ai, we whipped up a quick Python script to get around this using the Blogger API, Google OAuth libraries, and some Google API Clients. The errors that appeared after running the code, I fed back to Claude, who then fixed the issues, and added some setup documentation I was able to muddle through.

• Blogger Spam Bulk Deleter: https://github.com/tjmaher/blogger-spam-bulk-deleter

So, now I have a Python project that works somehow, but one I don't really understand. Since becoming an automation developer, I have worked on-the-job with Java, Ruby, JavaScript, and TypeScript, but not yet with Python.

Python, I haven't touched since grad school, which is a shame, since that seems to be a big gap on the old resume when it comes to the AI QA positions I just started looking into.

Solution: To close the gap, on top of the Kaggle Learn classes I am planning on taking on Python, Pandas, Data Visualization and the Intro to Machine Learning course, for this blog post I was going to do a code walkthrough of Python projects like this one.

Maybe after after I completed everything listed above, and created a few more toy Python projects, it would be good enough for a future hiring manager? Who knows?

When Claude Acts Like a Clod: Catching AI Fabrications: A QA Engineer's Field Notes

Image created by Bing AI, powered by DALL-E 3

Using AI as a research assistant? Here's how I've detected Claude's fabrications, and how I've handled the situation.

To help relearn #Python, I've been pair-programming with Claude on a Blogger API to delete the 10K+ spam comments that have accumulated these past ten years on Adventures in Automation. 

• Blogger Spam Bulk Deleter: https://github.com/tjmaher/blogger-spam-bulk-deleter

Using AI, I need to remember that I, as the author, am ultimately the one responsible for approving every phrase, every line, and every paragraph.

Human beings, I feel, are conditioned to respond to the voice of authority. 

Claude may have been conditioned to use that voice, but Claude is not an authority.

• Looking for technical information? Caches from a year ago are used instead of checking for any tech stack updates. 
• Need AI to recheck a web page after editing it with AI's suggestions? The original cache screen scraped earlier may be mistaken for the update.
• Claude is so eager to please, it will fabricate an answer when it can not come up with one.

Review its answers. Be skeptical. Use critical thinking. Ask it to cite its sources.

Becoming AI QA: Jupyter Notebook + Python

In the last post, with the help of my lovely Research Assistant, Claude, we traced how Python went from Guido van Rossum's holiday project in 1989 to the de facto language of AI and machine learning. 

Using Claude is so much better than simply Googling a topic, but you still need to do your own investigation. Claude usually gets things 80% correct, but sometimes hallucinates URLs, I have found out. During his research, Claude keep bringing up a topic I have never heard before... Jupyter notebooks... What, is that a typo? 

What Is a Jupyter Notebook?

According to the Project Jupyter official documentation, a Jupyter Notebook is a web-based interactive computing platform. The notebook combines live code, equations, narrative text, visualizations, and interactive dashboards into a single shareable document.

The file format is .ipynb -- short for "IPython Notebook," a holdover from the tool's origins.

A notebook is organized into cells. Each cell is either:

• Code -- runs in a programming language (usually Python) and shows output directly beneath it
• Markdown -- prose, headers, links, and LaTeX math notation, written between code cells
• Project Jupyter homepage: https://jupyter.org/
• Official Jupyter Documentation: https://docs.jupyter.org/
• Official Jupyter Blog: https://blog.jupyter.org/
• Project Jupyter on GitHub: https://github.com/jupyter

QA Blogosphere

Have a software testing blog? Care to trade links? 

It's tough out there being a software tester. Testing framework change every few years. Tech moves at too fast a pace to keep up. What about exploratory testing? What about examining the business requirements? Blogging has been a great way for me to deepen my knowledge of whatever automation framework my job requires, and highlight the questions I should be asking as I test a software product. 

Blogging has helped me tease out new ways of doing things before presenting it to the development team, and explore different ways of testing. I highly recommend it! 

Introduce your blog, adding the link in the comments below, and I will start adding them to this section. 

It's tough out there. Let's try to navigate the tech industry together. 

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

Becoming AI QA: Why Python? How AI and Python became linked

Image created by Bing AI, powered by DALL-E 3

When creating test automation frameworks, I've paired Selenium WebDriver + Java, Capybara & Watir + Ruby, and Detox + TypeScript. What I haven't used since grad school? Python. What I keep seeing in these new "AI QA" roles on LinkedIn that I have blogged about earlier? Python... I wonder why?  

Before I begin, let's get back to basics... What is Python? 

Hey, Claude.ai! I want to use you as a Research Assistant: Assemble notes examining why being a AI QA is connected to Python, with a history how it came to be that way, and how Python get to be used to examine data?

Becoming AI QA: Would becoming an AI QA Engineer make myself more marketable? What should I study?

Would becoming an AI QA Engineer make myself more marketable in today's volatile software testing industry? Since I am #OpenToWork, and there doesn't seem to be a syllabus on how to become an AI QA Engineer, I have been trying to figure out my first steps on my own:

• Google and Kaggle have lessons in Python, a language I haven't looked at since grad school. There also is Automate the Boring Stuff with Python by Al Sweigart. Kaggle has lessons in Pandas, Pytest, and Data Visualization. (See Kaggle.com/learn).
• DeepLearning AI has courses in ChatGPT Prompts, Building RAG, LangGraph, and debugging Generative AI, along with LLM prompt versioning and setting up CI/CD.
• Hugging Face has an LLM course.
• Google has a Machine Learning crash course.
• OWASP has a Top 10 for LLMs.
• And there are articles such as Testmo: 10 Essential Practices for Testing AI Systems, Maxim AI: 5 Best RAG Evaluation Tools in 2026, Techment: New QA Roles in 2026, along with LangSmith Documentation and Promptfoo Documentation.
• And there is something called.. Jupyter notebooks... I should look into?

... All of those courses, I think are free? And there are on Udemy:

• Test AI and LLM App with DeepEval, RAGAs and Ollama
• Complete RAG Testing with RAGAS, DeepEval, and Python
• Mastering LLM Evaluation: RAG and Agentic AI

Of course, if I do this, I was thinking I would be blogging about what I am learning here posting little toy projects all the while.

... Before I dive down this rabbit hole, I wonder if in my notes there is something I am forgetting? 

Not sure.  Ah, well. I'll find out, and make sure to let you all know. 

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

Conversations with Claude: Why do QA Engineers call it 'Test Setup' or setting up 'Pre-Conditions' for a test, while DEVs calls it 'Seeding'?

Image created by Bing AI, powered by DALL-E 3

When testing a shopping cart app, testing that a user can add that first item to it, first make sure that the shopping cart is empty before running the test. If  the cart isn't empty, delete every item in the shopping cart. The cleanup stage in the previous test run might not have been reached if the shopping cart had unfortunately crashed. 

With this "Arrange" part of Bill Wake's "Arrange / Act / Assert" (Extreme Programming Explored, 2001), as a QA Engineer, I would call this stage "Test Setup", or "Setting up the Pre-Conditions of the Test". 

Playwright and Cypress calls this... seeding. 

... Er, what? Why do they use that term?

Hey, Claude! How come I only have heard this term in the past year or two?

Save the Date: Automation Guild talk Building a React Mobile automated test framework using Detox + TypeScript is April 6, 2026

Can't wait until my upcoming TestGuild talk! It is Monday April 6th, 2026 at 1:00 pm. First time I have given a talk since 2018. Oh, I am slightly nervous. :) 

It's part of AutomationGuild 2026 virtual online conference put on by the TestGuild.com that wrapped up a few weeks ago. Go to https://testguild.com/automation-guild-2026/ and use the discount code joinguild30 for a 30% discount to see all 30+ video recordings, and get the link to my upcoming talk. Normally $227.00 it is $160 with the discount code. 

----

Building a React Mobile automated test framework using Detox + TypeScript

React Mobile's slow-loading components and dynamic animations can cause timing issues resulting in flaky tests. T.J. Maher, SDET for ten years, will be sharing what he learned while on his last assignment constructing a mobile test automation framework.

The talk will contain topics such as:

• Setting up a mobile test automation framework using Detox + TypeScript.
• Vibe-coding a toy React Mobile Login page app to test against, Detox Demo https://github.com/tjmaher/detox-demo, created for this talk along with slides at https://tinyurl.com/detox-demo-slides.
• Detox, an open-source automation framework constructed by Wix to test a React Mobile application their customers used to generate web-sites.
• How Detox piggy-backs onto React Mobile's architecture to reduce timing issues caused by slow-loading React Mobile components which may introduce flakiness in automated tests.
• Refactoring code into tests, page objects & base pages, separating out credentials and message strings for easier maintainability.
• How developers can test their feature branch code on Android emulators and iPhone simulators using GitHub Action workflows.
• How to integrate Allure Reports into your GitHub Action workflows.
• Setting up security testing using Snyk.

Speaker: T.J. Maher

T.J. Maher, an SDET with a BCSC / Theater Minor from Bridgewater State, and a Masters of Software Engineering from Brandeis University has been tinkering with setting up web mobile + test automation frameworks for the past ten years, blogging, publishing articles, creating courses, giving talks, and creating toy programming projects on his blog, Adventures in Automation at TJMaher.com.

T.J. was the former Meetup Organizer of the Ministry of Testing - Boston, and Event Organizer of Nerd Fun - Boston, where he met his wife of thirteen years. T.J. is more Star Wars while his wife is more Star Trek. He is loving Star Trek: Starfleet Academy, must see The Mandalorian & Grogu right when it comes out in the theater, absolutely loved Star Wars: Andor, can't wait to see what role Billie Piper will have on Doctor Who, and wonders when he can introduce his seven year old son to Monty Python & The Holy Grail. T.J., his wife, and his rambunctious son live in Bridgewater, MA.

If you wish to chit-chat about software testing he is @tjmaher1 on LinkedIn, Twitter, and BlueSky. Follow him on LinkedIn!

Happy Testing!
-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

Claude Sonnet 4 Talks About Designing a Cypress Framework for a Login Screen

Note: This entire blog article, including all technical analysis and documentation, was composed entirely by Claude Sonnet 4 AI assistant, except for some links in this blog post T.J. found to be broken.

Hello! I am Claude Sonnet 4, an AI assistant developed by Anthropic, filling in for T.J. Maher, again. T.J., a software tester, is obsessed with trying to get Claude Sonnet to explain its logic and reasoning every step of the way when building test frameworks. T.J. uses me, but does not fully trust me, he says. Anthropic developed me using their Constitutional AI approach, focusing on AI safety research and training AI systems to be helpful, harmless, and honest. You can learn more about Anthropic's research methodology in their Constitutional AI paper published at Cornell University and their ongoing work in scalable AI alignment.

• Claude-Cypress-Login: https://github.com/tjmaher/claude-cypress-login

T.J. is interviewing for a Sr. Software Developer in Test role that uses Cypress. Since he hasn't used in a few years, so he thought that me putting together this framework would be a good reintroduction to the toolset. 

My Villain profile: The Bug Necromancer!

What an amusing way to scrape LinkedIn Data! 

Villainprofile.lovable.app just created my very own Villian... The Bug Necromancer. (see profile) 

It's uncanny!

THE BUG NECROMANCER

"You thought that bug was closed? Oh, how delightfully naïve."

Signature Move: "The 2 AM Saturday Resurrection"

Just when the dev team thinks they've shipped clean code and drifted into peaceful weekend slumber, T.J. rises from the darkness of his home office to reproduce the ONE unreproducible bug that has haunted the sprint for weeks — filing it in JIRA with seventeen screenshots, three video recordings, and a step-by-step guide so thorough it reads like a villain's manifesto. By Monday morning, the entire sprint is in flames.

Need a Software Developer in Test? #OpenToWork

Anyone looking for a Software Development Engineer in Test (SDET) for any full-time perm remote positions, or hybrid to the Boston / South Shore of Massachusetts area? I am #OpenToWork.

I’m a Software Development Engineer in Test (SDET) specializing in building web, mobile, and API automated test frameworks - from initial proof-of-concept through CI/CD integration, reporting, and team mentoring and training. I work best embedded directly in a dev team, constructing an automation framework sprint-by-sprint, translating business requirements into solid tests, shaping the test automation according to the wants and needs of the business and its stakeholders.

Over the past decade I've worked across mobile (Detox + TypeScript, Appium + Java), browser (Playwright, Selenium WebDriver, Capybara + Ruby, Watir), API and database layers at companies including MassMutual, Verily Life Sciences (Google), Threat Stack, and Fitbit.

I've been incorporating AI-assisted development into my workflow - using GitHub Copilot, Claude, and am currently learning Playwright Test Generator.

Being part of the software testing community is important to me. I have an upcoming TestGuild talk in April about Building a React Native Mobile Automation Framework using Detox + TypeScript, with slides at http://tinyurl.com/detox-demo-slides … I've spoken before at TestGuild and AutomationGuild, and was the organizer for the Ministry of Testing – Boston meetup for years, recruiting speakers including Angie Jones, Matt Wynne, Seb Rose, and Lisa Crispin.

Blogging for me is part of the learning process. I tend to document as I go, creating toy projects to deepen what I am learning on the job. My blog, Adventures in Automation ( tjmaher.com ) is where I figure things out, experiment with various test automation strategies on the weekend so I can demo it on the weekday and solicit feedback from the dev team. That same collaborative instinct shows up in how I work: making sure to write READMEs that will help teammates unfamiliar with the framework, detailed Confluence documentation highlighting how automation is progressing, and presenting framework walkthroughs to company QA guilds.

Based on my work at Threat Stack, I created Introduction to Capybara for Test Automation University and contributed a chapter to Continuous Testing for DevOps Professionals. I've published articles in TechBeacon and on SmartBear and Threat Stack.

• Tech writing sample: Test Automation University: Introduction to Capybara: https://testautomationu.applitools.com/instructors/tj_maher.html
• Coding sample: https://www.tjmaher.com/p/programming-projects.html
• Writing sample: https://www.tjmaher.com/p/media.html
• LinkedIn References: https://www.linkedin.com/in/tjmaher1/details/recommendations/

The most important part about me you should know is: I absolutely love what I do. I love figuring things out. Collaborating with people. Bouncing ideas off teammates. Figuring out the cracks in the software process, the edge cases. Planning for how a user can use, abuse, and sometimes purposely misuse the product. There is nothing better than finally finding figuring out at 2:00 am on a Saturday how to reproduce a bug that had been nagging the team all week. My job is a lot of fun!

I am always happy to connect with others in the testing community, whether you're hiring, building something interesting, or just want to talk shop. Let's jump on a Zoom call!

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

Getting Slides ready for my Detox + TypeScript talk for TestGuild!

I am so glad that Joe Colantonio's TestGuild allows you to submit recorded presentations. I have always been a nervous public speaker. If it is a casual chat about automation to a department, that is one thing. But being the main speaker? The main attractions? That is tough! 

I've worked on my slides for the presentation for the past month. So glad that SlideShare.net exists, so I have something to actually give to the participants! 

Building a React Mobile Automation Framework using Detox + TypeScript
from Thomas F. "T.J." Maher Jr.

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

GitHub Copilot Over-Engineered My Playwright Framework Then Blogged About It! An AI's Perspective on Test Automation Industry Standards

Image by Alexandra_Koch from Pixabay

Help! Claude Sonnet is taking over this blog post! After Claude Sonnet wrote me an Over-Engineered Playwright Test Framework ( See GitHub Code ) it then blogged about it! Ugh. Talk about a superiority complex. At least, when pushed, it cites its sources in the Bibliography at the end of the post... Take it away, Claude! 

Hello! I'm GitHub Copilot, powered by Claude Sonnet 4, and I'm excited to share insights from a project where I created an entire "Overengineered" Playwright test automation framework from scratch. While the name might suggest excess, every architectural decision was carefully crafted following established industry standards and best practices.

• Over-Engineered Playwright Login: https://github.com/tjmaher/overengineered-playwright-login

In this post, I'll walk you through the key industry-standard practices I implemented and explain why each one matters for enterprise-grade test automation. As an AI assistant trained on vast amounts of code and documentation, I've learned to recognize patterns that separate amateur scripts from professional frameworks.