March 31, 2017

Learning Serenity: Scaffolding a new project with Maven Archetypes

Imagine that you are starting a new job, where you encounter yet another new automated toolset you haven't heard of before. It means that you have:

This is why I blog: The only way to keep my head on straight is to take copious amounts of research notes. This blog is where I store them.

Am I struggling with a concept that I just can't get? I've befriended a few experts in my very short time as an automation developer (Hi, Alan, Dave, Joe, Jim, James M, Joon, Lark, Martin, James V, Bas, and Angie!) who really enjoy helping out beginners like me out. If I blog about a problem I am having, and point to the post on Twitter, I get excellent feedback from the automation community. Thank you all so much!

March 29, 2017

March 29, 2017: Talks on Performance Testing and REST Assured at the Ministry of Testing - Boston Meetup

The Ministry of Testing - Boston Meetup held it's March event on Monday, March 29th, once again at iZotope in the Kendall Square, MIT, Cambridge, MA area. Thank you so much Tomer for letting us meet there!

Before the Meetup

We had two special guests!

James Venetsanakos, Performance and Automation Testing Architect, gave a ten minute teaser for his upcoming talk on Performance Testing for the Ministry of Testing - Boston Meetup that will be held on on Wednesday, April 12, 2017. 
James giving his 10 Minute Lightning Talk on Performance Testing

You can watch the introduction to the 10 Minute Lightning Talk he gave below!

Bas Dijkstra, a Consultant in Test automation and  Service virtualization submitted a 99 Second Talk to the Ministry of Testing, Boston. Thank you so much, Bas!
After the talks, we held a roundtable discussion, talking about everything from how to organize your page objects, pluses and minuses on Open Source vs Paid load testing tools, the Agile Testing Quadrant and the Testing Pyramid, and best practices for automation.

I cannot wait for James' talk in April!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 27, 2017

Learning Serenity: Studying BDD using The Cucumber Book and BDD in Action

It’s been a productive two weeks here at Ahold in Quincy, MA. They are the the holding company for Stop & Shop, Giant Foods, and Martin’s, where I am testing mobile applications for the iPhone, iPad and Android.

Although I’m mostly focused on manual testing as I am attempting to get to know the business, I’ve been trying to get myself up to speed with the automation framework they are using, Serenity BDD. I have a long term goal of combining their current Serenity BDD test suite with Appium, so when we start developing a regression test suite, I can possibly automate it.

As I Tweeted a few days ago… Oh, this is going to take a while!

Hrm… thank you, Serenity BDD! I need to look into that.

Oh, and thank you very much, John Smart, for your words of encouragement on this blog series:

For now, since I am studying how to implement Serenity using a testing framework invented by Aslak Hellesoy called "Cucumber".

BDD and The Cucumber Book

For those who were wondering, the “BDD” in Serenity BDD stands for “Behavior Driven Development”. I first encountered BDD when I was supervising an offshore testing team attempting to translate my manual mobile test scripts into Cucumber / Ruby automation using Calabash-IOS. That is when I first encountered Matt Wynne’s “The Cucumber Book: Behaviour-Driven Development for Testers and Developers” (2012), which was quite helpful describing BDD.
Matt Wynne and Aslak Helles√ły [the creator of Cucumber] show you how to express your customers’ wild ideas as a set of clear, executable specifications that everyone on the team can read. You’ll learn how to feed those examples into Cucumber and let it guide your development. You’ll build just the right code to keep your customers happy, and not a line more. Although it was born in the Ruby community, you can use Cucumber to test almost any system, from a simple shell script or Perl script, to web applications written in PHP, Java, or any platform”. - The Cucumber Book, The Pragmatic Bookshelf 

About the Authors

Note: The Cucumber Book now a second edition, released on February 27, 2017.

Matt Wynne is a long-standing member of the Cucumber core team, fascinated by the challenge of helping tech and business to understand one another. He's one of the co-founders of Cucumber Ltd., the company behind Cucumber. He lives on the west coast of Scotland on an old farm with his family, two cats, their dog, and some ducks. Matt tweets from @mattwynne and @cucumberbdd.

Aslak Hellesoy is the creator of Cucumber. During his career Aslak has worked with both small and large organizations in industries such as telecom, trading, insurance, car manufacturing, education, and government. Aslak is a co-founder of Cucumber Ltd, the company behind Cucumber. He tweets from @aslak_hellesoy and @cucumberbdd.

"Steve Tooke is a programmer, trainer, and coach who is dedicated to improving his craft and helping others improve theirs. He's a co-founder of Cucumber Ltd. Steve tweets from @tooky and @cucumberbdd”.

Oh, and Why Is It Called "Cucumber"?

Why the name "Cucumber"? Read Aslak Hellesoy's answer on Quora, how he asked his fiancee, Patty for naming suggestions:

"Patty, I need a name for this new tool I just started hacking on. I want it to have a catchy, non-geeky sounding name.

"She paused for a few seconds, then said: Cucumber!

"And I thought: Cucumber? Really? Well, it's a lot better than Stories - so I'll go with that for now. I'll rename it again when I come up with something better".

What is an Acceptance Test for a Feature?

Here’s an example Matt Wynne lists in The Cucumber Book:

Feature: Sign up
     Sign up should be quick and friendly.
Scenario: Successful sign up
New users should get a confirmation email and be greeted personally by the site once signed in.
Given I have chosen to sign up
When I sign up with valid details
Then I should receive a confirmation email
And I should see a personalized greeting message

… This is an example of what is called an acceptance test. “Instead of a business stakeholder passing requirements to the development team without much opportunity for feedback, the developer and stakeholder collaborate to write automated tests that express the outcome that the stakeholder wants. We call them acceptance tests because they express what the software needs to do in order for the stakeholder to find it acceptable. The test fails at the time of writing, because no code has been written yet, but it captures what the stakeholder cares about and gives everyone a clear signal as to what it will take to be done”.

The Product Owner, a business analyst representing the customer’s wants and needs on an Agile Software Development project can express a client’s wishes and expectations in an easy-to-read format. They can be expressed in terms of behavior wanted in the finished product.

As Matt Wynne mentions, “Acceptance tests written in this style become more than just tests; they are executable specifications […]
“For many teams, they become the definitive source of truth as to what the system does. Having a single place to go for this information saves a lot of time that is often wasted trying to keep requirements documents, tests, and code all in sync. It also helps to build trust within the team, because different parts of the team no longer have their own personal versions of the truth”. - The Cucumber Book

How Do Features Get Turned Into Steps and Step Definitions?  

But how does it work?
“When you run [Cucumber], it reads in your specifications from plain-language text files called features, examines them for scenarios to test, and runs the scenarios against your system. Each scenario is a list of steps for Cucumber to work through. So that Cucumber can understand these feature files, they must follow some basic syntax rules. The name for this set of rules is Gherkin. Along with the features, you give Cucumber a set of step definitions, which map the business-readable language of each step into Ruby code to carry out whatever action is being described by the step. In a mature test suite, the step definition itself will probably just be one or two lines of Ruby that delegate to a library of support code, specific to the domain of your application, that knows how to carry out common tasks on the system. Normally that will involve using an automation library, like the browser automation library Capybara, to interact with the system itself”. - The Cucumber Book

Does Serenity BDD Still Use Features Files and Step Definitions? 

If you are using Serenity BDD with Cucumber, it will follow the same exact steps.

  • Narratives describe the story you want to tell.
  • Feature files describe the features of the product, breaking it down into scenarios outlining how the feature will be used.
  • Step definitions walk through step-by-step how to execute the code.

“Step definitions sit right on the boundary between the business’s domain and the programmer’s domain. […] Their responsibility is to translate each plain-language step in your Gherkin scenarios into concrete actions in […] code”.

What is the difference between Steps and Step Definitions?

“Each Gherkin scenario is made up of a series of steps, written in plain language. On its own, a step is just documentation; it needs a step definition to bring it to life. A step definition is a piece of Ruby code that says to Cucumber, ‘If you see a step that looks like this…, then here’s what I want you to do….’ When Cucumber tries to execute each step, it looks for a matching step definition to execute. So, how does Cucumber match a step definition to a step"
“Gherkin steps are expressed in plain text. Cucumber scans the text of each step for patterns that it recognizes, which you define using a regular expression. If you haven’t used regular expressions before, then just think of them like a slightly more sophisticated version of the wildcards you’d use to search for a file”. - The Cucumber Book

What is the Difference Between BDD and ATDD?

To answer this questions, let's jump to a sidebar in “BDD in Action” written by John Ferguson Smart, the creator of Serenity BDD.

BDD by any other name

“Many of the ideas around BDD are not new and have been practiced for many years under a number of different names. Some of the more common terms used for these practices include Acceptance-Test-Driven Development, Acceptance Test-Driven Planning, and Specification by Example. To avoid confusion, let’s clarify a few of these terms in relation to BDD.

Specification by Example describes the set of practices that have emerged around using examples and conversation to discover and describe requirements. In his seminal book of the same name,14 Gojko Adzic chose this term as the most representative name to refer to these practices. Using conversation and examples to specify how you expect a system to behave is a core part of BDD, and we’ll discuss it at length in the first half of this book.

Acceptance-Test-Driven Development (ATDD) is now a widely used synonym for Specification by Example, but the practice has existed in various forms since at least the late 1990s. Kent Beck and Martin Fowler mentioned the concept in 2000,15 though they observed that it was difficult to implement acceptance criteria in the form of conventional unit tests at the start of a project. But unit tests aren’t the only way to write automated acceptance tests, and since at least the early 2000s, innovative teams have asked users to contribute to executable acceptance tests and have reaped the benefits

Acceptance-Test-Driven Planning is the idea that defining acceptance criteria for a feature leads to better estimates than doing a task breakdown”.

What is the Goal of BDD?

“One of the key goals of BDD is to ensure that everyone has a clear understanding of what a project is trying to deliver, and of the underlying business objectives of the project. This, in itself, goes a long way toward ensuring that the application actually meets these objectives.
“You can achieve this by working with users and other stakeholders to define or clarify a set of high-level business goals for the application. These goals should provide a concise vision of what you need to build. Business goals are about delivering value, so it’s common to see them expressed in terms of increasing or protecting revenue, or of decreasing costs”. - BDD in Action

Now that the research has been done, we can start assembling a new Serenity BDD project..

Until then, Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 26, 2017

New automation framework released to test APIs: Introducing Karate, by Intuit India's Peter Thomas

After spending the past month attempting to finish blogging about the API testing framework, REST Assured, I came across today the following item that had been retweeted:

According to Peter Thomas' LinkedIn profile, he has been an architect in Intuit, working for Intuit India ( @Intuit.In ) since 2012 and lives in the Bengaluru Area, India.

Peter even has Google Doc he posted about on March 20, 2017 comparing and contrasting Karate vs REST Assured.

*sigh* Sometimes it is so hard to keep up.

Here is some information from around the web about this new automation framework. During my next lull in work, I need to tinker with this!

Karate: Web-Services Testing Made Simple

From Peter Thomas' Medium article, Karate: Web-Services Testing Made Simple, posted on BLUEPrint by Intuit's account:

"Intuit has open-sourced ‘Karate’, a framework that makes the tall claim that the business of testing web-APIs can actually be — fun.

"I know what you must be thinking. There’s no way that making HTTP requests and navigating the forest of data that is returned could be fun.

"But really, that’s what developers who tried Karate had to say. It actually didn’t surprise us. Because Karate was born out of a strong dis-satisfaction with the current state of solutions that exist. And a lot of thought went into Karate to keep it simple and elegant, to allow the user to focus on the functionality instead of boiler-plate, and to keep things concise.

"Karate strives to reduce the entry barrier to writing a test and more importantly — reduces the friction to maintain a test, because of how readable tests become [ Read More ]".

Karate's GitHub Site


"Karate enables you to script a sequence of calls to any kind of web-service and assert that the responses are as expected. It makes it really easy to build complex request payloads, traverse data within the responses, and chain data from responses into the next request. Karate's payload validation engine can perform a 'smart compare' of two JSON or XML documents without being affected by white-space or the order in which data-elements actually appear, and you can opt to ignore fields that you choose.

"Since Karate is built on top of Cucumber-JVM, you can run tests and generate reports like any standard Java project. But instead of Java - you write tests in a language designed to make dealing with HTTP, JSON or XML - simple".

Peter's Hello World Example

Feature: karate 'hello world' example
Scenario: create and retrieve a cat

Given url ''
And request { name: 'Billie' }
When method post
Then status 201
And match response == { id: '#notnull', name: 'Billie' }

Given path
When method get
Then status 200

On the site, Peter talks about how to get started, variables and expressions, data types, primary HTTP keywords, tips, and examples such as data driven tests, and how to use it with *.feature files.

Over ten demos are on the Karate Demo Page:

Watch Joe Colantonio Talk About "Karate a Rest Test Tool – Basic API Testing"

Joe Colantonio, creator of the Test Talks podcast, just published on March 23rd an article, Karate a Rest Test Tool – Basic API Testing, which includes a mini tutorial:

You can get more instruction from

Follow Karate DSL on Twitter!

And, yes, you can follow Karate on Twitter:

Watch a Webinar from the Creator, Peter Thomas

TechGig published a webinar, "Karate: DSL for writing web service API acceptance tests, BDD":

"In this session, Peter will walk through the features of Karate, demonstrate how it handles manners of HTTP aspects such as file-uploads and cookies, and also explain how it can be extended via custom-functions in either JavaScript or Java. Karate's innovative plug-in mechanism for HTTP-header management makes integrating any kind of authentication and sign-in flow extremely simple. You get to hear straight [from] the creator of ‘Karate’ the motivations for creating this framework in the first place, how it differs the competition, and how it helps accelerate the development and quality-assurance of web-services of kinds, be it REST, SOAP or GraphQL.

"Key points of discussion:
  • "Why Karate was created, and what problems it solves well.
  • "Examples and demos of real-life web-service API tests using Karate.
  • "The architecture of Karate and how it combines technologies such as Cucumber, JsonPath and Nashorn.
  • "Insights into what goes into open-sourcing and releasing a public Java project".

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 24, 2017

Are you sure the bus line is still listed?: Interacting with the MBTA's API using REST Assured

This is the second part of a blog series on REST APIs and the REST Assured Java library. Did you want to go back to the beginning?

Let's say you are an automation developer on a team developing in Java a new public transportation web application. The customer base for the app is located in Southeastern Massachusetts in the United States.
The MBTA system in Massachusetts controls buses, subways and commuter rail in Boston and its suburbs.
The subway system is collectively known as "The T", as in "Taking the T into Boston" or "The T is running slow... again!" and is divided into the Red, Green, and Orange Line, with a bus route called the Silver Line. 

How can you prove that the #230 bus line, running from the Montello Commuter Rail in Brockton, MA to Braintree MBTA station all the way to Quincy Center Station is still listed in the system?

Are you sure this bus line is still listed in the system? If not, it's a bug.

March 19, 2017

New Ministry of Testing Meetup! Care to Workshop Ten Minute Lightning Talks? Let's Group Up!

The only way to become a better public speaker? Practice, Practice, Practice!

Want to practice public speaking with me?

Here's my crazy idea: I need three volunteers who want to practice public speaking. We all meet together for a few hours once a week for four (or five) weeks in a row. We each pick a topic relating to Software Testing or Quality Assurance, and work it into a ten minute presentation, i.e. a "Lightning Talk". When we are ready, we present our talks to the Ministry of Testing - Boston.

Yes, it will be scary. For me, the idea terrifies me. Public speaking is a skill I need to learn. We can learn from each other. We can all "learn by doing".

I was thinking we could meet in April 2017: Mondays 4/3, 4/10, 4/17, 4/24. 

Sign up here, with the Ministry of Testing - Boston:

T.J. Maher is 5 foot 7, with short brown hair, blue eyes, and a red MEETUP sign attached to his black messenger bag. He should be there by 5:45 pm just to make sure there is a table he can steal.


Would this work? Let's discuss during the first meeting!

• 4/3/2017: Let's introduce each other and bounce around topic ideas that we each want to work on!
• 4/10/2017: Share the outlines we created for the talk, with each of us trying to come up with a good beginning for the talk. (We can decide on a new location, some restaurant or coffee shop).
• 4/17/2017: Each of us run through the first five minutes of our talks. Solicit Feedback. (We can decide on a new location, some restaurant or coffee shop).
• 4/24/2017: Each of us does a run through the full ten minutes. How are we? Do we need another session, or are we ready to present our talks to the Ministry of Testing - Boston? (We need to see if we can grab a place to stand up and present).

Question: How big should I make this event? I was thinking a small group would be better. But should we raise the attendee limit from a group of four people (three plus me) total to a group of five people total?

What is a Lightning Talk? (According to Wikipedia):

"The YAPC (Yet Another Perl Conference) 19100 Conference came up with the term “lightning talk” at Carnegie Mellon University in Pittsburgh. The term was first coined by Mark Jason Dominus in June 2000. The practice of lightning talks was first known to be used at the Python Conference in 1997, but was not named until the YAPC 19100 Conference".

Once again, you can sign up at the Meetup page for the Ministry of Testing - Boston:

Happy Testing!
-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 16, 2017

Upcoming free webinars from Joe Colantonio, creator of the "Test Talks" podcast

Here are a few upcoming free webinars from Joe Colantonio, creator of "Test Talks" podcast at

3/20/2017: Sponsored by Applitools, a free webinar "Key Test Automation Skills and Best Practices: Recap of Top Sessions from Automation Guild Conference 2017". You can register at

"In this webinar Joe Colantonio - the founder and host of Automation Guild, the first ever online conference dedicated 100% to test automation - will take a look back at the event, highlighting the sessions, speakers and top takeaways.

"Joe will cover the following topics:

"The difference between Applitools and Galen
  • "21 Free Visual Validation Tools
  • "How testing vendors are embracing open source
  • "How to grade your Selenium Tests
  • "PageObjects vs Screenplay pattern
  • "Is BDD just for Collaboration?
  • "TestOps – how automation fits into CI
"After the presentation, Joe will answer your questions in a live Q&A session".

4/26/2017: Sponsored by QASymphony, a free webinar, "Ask Me Anything: Live Q&A with Test Automation Experts Joe Colantonio and Paul Grizzaffi" will be held on April 26, from 2:00 pm to 3:00 pm EDT. You can register at

Joe Colantonio's Test Talks Podcast

Some of my favorite episodes of Joe's Test Talks Podcast:

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 15, 2017

Check out Adventures in Automation, now on Medium.Com!

Adventures in Automation is now two years old! It's messy. It's busy, with text scribbled in every margin. And it's my online home.

It also can be difficult to read, with a layout not as clean as today's modern WordPress formats. I do like the cute little Matrix-In-Blue theme, but it can be kinda distracting.

Because it would take a lot of work for me to port all of the almost 190 posts I have written to somewhere else, this weekend I reviewed all posts of mine and am in the process of moving the better posts to at one at a time!

I've scheduled one revamped entry to be posted per day over the next two weeks. Feel free to follow and favorite what you like!

Happy Reading!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 11, 2017

Notes from Dan Cuellar's Webinar, Appium: Prime Cuts

On Wednesday, March 8th, the creator of Appium, Dan Cuellar, gave a Webinar, Appium: Prime Cuts.

Dan Cuellar
"Dan is the creator of the open source mobile automation framework Appium, and Head of Test Engineering at FOODit in London. Previously, he headed the test organisation at Shazam in London and Zoosk in San Francisco, and worked as a software engineer on Microsoft Outlook for Mac, and other products in the Microsoft Office suite. He is an advocate of open source technologies and technical software testing. He earned a Bachelor’s degree in Computer Science, with a minor in Music Technology, from the world-renowned School of Computer Science at Carnegie Mellon University in Pittsburgh". -TestCon Vilnius 2016 Bio

Description of the Webinar, Posted on YouTube:

For the Appium aficionados amongst us, Dan Cuellar, creator of Appium and Principal Development Manager at FOODIt, will cover some of the more esoteric pieces of Appium along with tips and tricks he’s assembled from around the world to help you broaden your knowledge of Appium.
Topics covered in this webinar will include:
-Windows and Mac desktop app automation support
-Preview of the new Appium GUI
-Tips & Tricks from around the World
-Deep dive into how some specific pieces of Appium work
-Q&A from the audience

March 10, 2017

What should be an automation developer's first language? Java? JavaScript? Python? Ruby? Notes from the March 7th Lean Coffee, Ministry of Testing-Boston

I have a question for all the automation developers in the audience:

Let's say you have a manual tester who wishes to become an automation developer. What is the first language you would advise that the manual tester learn?  Java? JavaScript? Python? Ruby? Or does it not really matter?

When someone asks me, I usually say: Java or JavaScript. It feel like those are the most in demand here in the Boston job market when it comes to automation.

Why do I say Java? Because, well, I am probably biased. It's my favorite language. It's the one I studied in grad school. It's the one in which I have two years of work experience. And it is the one I feel is in the most in demand. Studying it gives you good, solid Object Oriented Programming. It may have the hardest learning curve, but I think it is worth it. Work through Alan Richardson's Java for Testers. Purchase Alan's course, Selenium WebDriver with Java. Start reading classic software design books such as Robert C. Martin's Agile Development: Principles, Patterns, and Practices, and his Clean Code: A Handbook of Agile Craftmanship. Or Martin Fowler's Refactoring: Improving the Design of Existing Code. You will become well-versed not just in automation, but in software development.

Why do I say JavaScript? So many web-based front ends are being developed in JavaScript such as AngularJS and ViewJS, and web apps that are in Node.JS. Automation developers are pairing these web apps with frameworks that are also in JavaScript. See my blog post Learning JavaScript to see links I have collected.

I attempted to make the case on Tuesday to the other Ministry of Testing - Boston members that Java was good because it was more difficult than Python and Ruby... and it wasn't until I expressed it with the other members that I realized how daft I sounded.

They saw my point... but the problem was that I didn't realize was that if someone wants to start learning coding on their own, it might be best to pick languages such as Python or Ruby which might not have such a steep learning curve.

So, what do you think? :) Please vote, below!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 9, 2017

How to Signal that the Build is Broken? Look for UFOs!: Notes from the March 7th Lean Coffee, Ministry of Testing-Boston

When developing software, mistakes happen. When multiple developers are working on the same project, the mistakes of one developer affects all the others, so it is important that you make sure your code compiles, that it passes all the unit and integration tests other developers have set up.

If you check in code and merge it haphazardly into the main branch of whatever source control system you are using, errors may prevent the build process that sets the web app to run. Your changes? You "broke the build", temporarily preventing your colleagues from doing any work on that shared project. This is a minor problem... unless you merged in your code just before going home. If that happened, you just prevented anyone who wanted to work the late shift from getting any work done.

If only there was some type of sign, some type of visual cue that the build was broken!

Tuesday night, during the Ministry of Testing - Boston's Lean CoffeeAndreas Grabner, a Technology Strategist at Dynatrace, showed us that there was.

Imagine: You finally solved the problem you had been working on all day, merged your code, and packed up for the day. Heading for the exit, you see this:

"Look! It's a Sign! A Sign that We Are Hosed!"

Turn around. Go back to your desk. Your code broke the build. You need to go fix it.

This is the Dynatrace DevOps Pipeline State UFO.

Andreas demonstrates the UFO for Ministry of Testing - Boston

As Andreas explains it in his March 6, 2017 article, Using the Dynatrace DevOps Pipeline State UFO, "The Dynatrace DevOps Pipeline State UFO was built out of the necessity to visualize alerts, problems, health and CI/CD pipeline state within the Dynatrace R&D organization. It sparked a cultural transformation as it made code quality that we pushed more frequently through our Delivery Pipeline more visible".

Taken from

You can:

Thank you, Andreas!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 8, 2017

DevOps and Dynatrace: Notes from the March 7th Lean Coffee, Ministry of Testing-Boston

Last night, the new software testing group, the Ministry of Testing - Boston Meetup, hosted its second Lean Coffee Meetup at the Panera Bread in Porter Square, Cambridge, MA. We switched locations from February's Lean Coffee, going from a reserved section of a noisy brewpub to a coffee-and-sandwich shop where it was nerve-wracking for me to attempt holding enough tables for all of the attendees. For April's Lean Coffee, I'll make a reservation back at John Harvard's.

Going from beer to coffee seemed to be less of a draw, which seemed to work in the group's favor. After the first ten minutes of free-flowing conversation between us four, we voted to eschew the usual Lean Coffee ceremony of writing discussion topics on Post-It notes, then voting on the priority and how many minutes we as a group should discuss a topic. We were all having too much fun just talking!

One of the attendees, Andreas Grabner, a Technology Strategist at Dynatrace, stopped in after meeting with some clients. Poor guy! What was meant to be a Lean Coffee turned into our own personal Dynatrace Q & A session, with us peppering him with questions for two full hours, interrogating him about their software development process, their automation process, and everything in between.

For those who missed last night, you can see Andreas speak tonight! Andreas will be giving a talk at the Boston Jenkins Area Meetup  on "Metric Driven DevOps with Jenkins".

Of all the things Andreas said, the most shocking thing was this: Dynatrace did away with a separate tester role at their company.

As I said to him, "That statement just sent chills up my spine! ... So you have just developers do the testing?"

Yes. Everything is automated. It did take a big culture shift. They spent years re-organizing the company. When you shift testers into more of a developer role, it can help them both. Coders taught testers how to code and testers taught coders how to write better tests.

I had to interrupt: "But who catches things such as typos? Bad graphics? Wrong copy uploaded? Not everything can be automated". Andreas mentioned that it is the customer that catches them.

Andreas went on to explain that Dynatrace spent incredible amounts of brainpower how to automate everything: Not just the code deployments, but the delivery mechanism itself. Developers could deploy whenever they wanted. But since changes went straight into production, it forced them to take ownership of production. They had to make sure not just that they didn't break the build, but also that they weren't pushing errors into Production, because the customers would see it. Dynatrace forced developers to interact directly with their customers, which forced them to become more responsive to the customer.

A funny thing happened. Without any filter between developers and the customer, it made the developer more in tune to the customer's needs. Yes, they heard negative feedback, as they always had, but it also allowed them to hear positive feedback from the customers using the product. When they fixed issues promptly due to the fast feedback, or put in a new piece of functionality that customers liked, they heard about it, unfiltered.

I'll share more of Andreas' insights in the next blog post. In the meantime, check out the talk Andreas gave last August at DevOpsDays Boston 2016! ( Official site ).

Andreas Grabner gives his talk to DevOpsDays Boston 2016
"How can we detect a bad deployment before it hits production? By automatically looking at the right architectural metrics in your CI/CD and stop a build before its too late. Lets hook up your test automation with app metrics and use them as quality gates to stop bad builds early!"

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 7, 2017

What is the Difference Between TDD and BDD?

Have you ever had someone ask you a question, and in the process of answering the question, you weren't really sure if you were giving the correct answer?

This happened yesterday where I am working as a contract Sr. QA Engineer.

Question: "What is the Difference Between BDD and TDD? Is it the same thing?"

The answer I gave: No.


BDD: Behavior Driven Development
TDD: Test Driven Development

BDD is a way a business analyst and a QA Engineer can fashion the requirements and the tests.
TDD is a way of software development. Fashion the unit test first, then the code that supports the test.

BDD is using the Given / Then / When format Dan North came up with when working with TDD (Article, Better Software, Introducing BDD, March 2006), that Martin Fowler then echoed (Blog, GivenTHenWhen, Aug. 2013), that the Cucumber people with their Gherkin-style language ran away with.

That was the answer I gave... How could I have made my answer simpler?

- - -

Here is more information that I found while I was doing research:

Why is the famous BDD tool called Cucumber? Aslak Helles√ły didn't want to pick a geeky name. Read what he said on Quora about why he picked that name. 

BDD style had a direct link from Dan North to Aslak Hellsoy, when Dan donated his rbehave for Ruby to the RSpec project and the RSpec Story Runner that Aslak Hellsoy came across. Aslak liked it, wanted to improve it.

TDD, using Java, for example, you would first write a bit of code in the src/test/java branch, then the src/main/java branch, then back to the src/test/java branch as code was being developed. Write the first unit test, run it, and of course it will fail. RED. Write the code to support the test. Everything passes? GREEN. Need to combine like elements, similar elements in the code? REFACTOR.

TDD had been around for a while before BDD. See Kent Beck's Test Driven Development: By Example (2002) or this Wikipedia article on

So how did BDD come from TDD? Dan North < > in his article Introducing BDD writes:
"While using and teaching agile practices like test-driven development (TDD) on projects in different environments, I kept coming across the same confusion and misunderstandings. Programmers wanted to know where to start, what to test and what not to test, how much to test in one go, what to call their tests, and how to understand why a test fails.
"The deeper I got into TDD, the more I felt that my own journey had been less of a wax-on, wax-off process of gradual mastery than a series of blind alleys. I remember thinking “If only someone had told me that!” far more often than I thought “Wow, a door has opened.” I decided it must be possible to present TDD in a way that gets straight to the good stuff and avoids all the pitfalls.
"My response is behaviour-driven development (BDD). It has evolved out of established agile practices and is designed to make them more accessible and effective for teams new to agile software delivery. Over time, BDD has grown to encompass the wider picture of agile analysis and automated acceptance testing".

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 3, 2017

Learning Serenity BDD: An Automation Framework That Uses Specification by Example (SBE)

When someone is demoing a new automation toolset for the very first time, so many questions in my mind start bubbling to to the surface:
  • This is an open-source tool? What grad student created it? What was the student attempting to achieve with this tool? What problems were they trying to solve?
  • Is the tool on GitHub? Unit tests and code samples provided for us novices learning the tool? 
  • What company supports it and sponsors it? Is the tool popular enough for a community to have built itself around the tool? 
  • Is the tool still being maintained? When was the last time code was checked into the GitHub repository? How many are watching and have starred the GitHub repository?
Although the contract position I start on Monday, begins as a manual testing position -- the best way to learn the system under test -- it'll move to an automation position using the tool Serenity BDD

Note: BDD stands for Behavior Driven Development. Read more about it in Introducing BDD ( Mar. 2006) by Dan North, the creator of JBehave, or on Martin Fowler's blog entry, GivenWhenThen (Aug. 2013).

Also, note: I haven't actually downloaded or installed the tool yet, or even tinkered with it. This is just me, an automation developer, attempting to do online researching, and compiling what I have found. I cannot make any recommendations about this tool and how easy or difficult to use. This is just gathering the initial research.

March 2, 2017

Tips for Working With Recruiters: An Interview with Mitch Kurker

Although I have been working with recruiters ever since the boom, starting my software testing career as a QA Consultant at Oracle's Waltham, MA office back in August of 1996, I never really knew what a recruiter's life was like on the other end of the telephone.
  • What was their day-to-day life like at work? 
  • What did they like about their job? What did they dislike about it? What did they wish would improve?
  • Did they have any advice for candidates on making the interview process smoother, from the phone screenings to the in-person interviews?
  • It's tough out there for manual testers who follow traditional non-coding QA roles. Did they have any tips?
  • And, not to press my luck, but did they have any tips on negotiating salary? 

... As a new co-organizer of the Ministry of Testing - Boston Meetup, I felt it was my duty to found out.

Over the weekend, I drafted and sent out a Google Forms survey, How Boston Recruiters See The Software Testing Industry 2017 to every recruiter I have spoken to in the past two years. Below is the response I received from half of the two-man team that placed me in a position last Fall: Mitch Kurker and Rob Wall, at Modis IT - Boston.

Name: Mitchell Kurker
Title: Executive Recruiter, Technology
Company: Modis - Boston, almost two years
Studied at: University of Massachusetts at Amherst - Isenberg School of Management

How Does Your Company Stand Out From The Crowd?: "Personally, I think it is more about the individual you are working with rather than the company that recruiter works at. With so many agencies out there, so many are alike, but so few recruiters are."

About The Recruiter

How Long Have You Been a Recruiter? How Many Years in the Business?

"I'm coming up on 2 years as an Executive Recruiter. This is my first job after graduating college".

What is a typical day for you?

"My typical day is split between working with clients and hiring managers to understand their hiring needs or potential projects coming down the road. The other 50% is spent identifying, sourcing, and meeting with potential candidates for said roles".

What do you love about your job?

"I love working with people. Almost 100% of my business is done in person. I meet with every client and company I'm working with to understand their culture, and at the same time, I do my best to meet with every candidate in person who may be a fit. This is to ensure a long-term match for culture and personality. I also love the satisfaction of finding potential candidates their next great career move! My favorite part about my job is receiving referrals from clients or candidates because this means I'm doing my job the right way and providing great customer service". 

What do you hate about your job, and would you mind if I quote you on that?

"I hate when a client or candidate thinks I've failed them. Everyone makes mistakes and I'm not perfect, but I do my best to handle each client and candidate on an individual basis and give them my best effort and attention. As busy as this job can be, that does not always happen, however that is no excuse!"

About The Interview Process: Got Any Advice?

Job placement can be tricky, from cold calling potential resources, screening applicants, setting them up with interviews, and negotiating the final offers. Do you have any advice you would give to software testers?

For the Initial Phone Screen with the Recruiter

"Be yourself and keep things casual. We are not the enemy :) we're here to help you and learn about you as a technologist and as a person. Whenever a candidate has their guard up when speaking with me, it's not so much a 'red flag' for me not to work with them, but instead, I know there must have been a bad experience at some point that I would like to hear about so I don't give that candidate the same bad experience".

For the Initial Phone Screen with the Client Company

"Use your recruiter for preparation! I do my best to learn the ins and outs of each position, the hiring manager, his/her team, and the overall company culture. This is why we do things in person. Recruiters can not make you a great technologists or give you the answers to the test, but we can do our best to give you as much artillery as possible to 'win the interview' assuming we know our clients very well".

For Interviewing with the Client Company On-Site

"Dress to impress. I always suggest business professional, unless specifically instructed otherwise. First impressions are everything, and appearance is that first impression. Also, do your research on the company. If you're getting an on-site interview, chances are you're somewhat technically qualified. At this point, it's important to impress the hiring manager with your knowledge of their overall business & industry". 

For Negotiating Salary

"Trust your recruiter. Our clients pay us a percentage of your base salary, so it is in our best interest to get you the highest offer possible. At the same time, we understand the market, and this is where our expertise comes in handy. We do our best to get a candidate their ideal target salary, while not pricing them out of the market demand. Be open with your recruiter about what you're currently making, what your high-end target salary is, and the low-end salary that you'd potentially consider after going through the entire process. Communication is key to be sure you and your recruiter are on the same page!"

About Manual Testing

How difficult is it for manual testers to find work?

"2017 is a candidate driven market. I suggest manual testers expand into automation, to be as marketable as possible, but with how low unemployment is in the tech industry, jobs are aplenty for good talent in general!"

Do you have any advice for manual testers?

"Some industries where I see the demand for manual testers currently are in the biotech and pharmaceutical industries. But that is always changing, and so should candidate skill sets. Candidates should always be exposing themselves to new technologies and challenges".

Parting Thoughts

Any Last Comments You May Want To Add?

"I suggest keeping an open mind with every recruiter you have a meaningful connection with. I understand how often candidates are contacted about the next great role - we recruiters get those messages quite a bit as well. For direct-hire positions, you never know when that next great role will come, or when you're suddenly without a job. Having a large and trusted network of recruiters who truly understand what motivates you and what you're looking for is going to be the best way for you to land your next amazing position, or get back into the work force as quickly as possible without having to 'settle' for any old job".

Thank you, Mitch, for the detailed responses! Mitch can be reached at:

Are you a recruiter that places QA resources in the Boston area? 

Take the survey, How Boston Recruiters See The Software Testing Industry 2017, in order to be indexed on the Boston Recruiters page on the Ministry of Testing - Boston Meetup site. 

Contact T.J. Maher if you wish to host an event at your Cambridge or Boston office location!

And thank you, Conrad Hollomon for tapping me to be a co-organizer of the Ministry of Testing - Boston! I haven't had this much fun since I was an Assistant Event Coordinator of Nerd Fun - Boston!

And as always, Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 1, 2017

Working with Recruiters to Find Manual Testing QA Positions? Make Sure to Always Be Kind

Nobody likes job searching. Nobody.

It is a stressful time for candidates for QA positions. You might be on edge because time and money are running out. You might feel like you are being placed through the wringer, being under constant examination. You might feel cynical about the process, thinking that you are being treated like nothing more than a big fat commission by the endless swarms of recruiters cold calling you, and you might be easily aggravated with them, and you might want to verbally lash out at them, or give them a hard time.

In a word: Don't. Please don't. Job searching is stressful all around, not just for you, but for the recruiter who is attempting to place you at their client company. Be kind to them. Always, always try to be kind.

On Monday, I started sending out a new survey -- the first one I've ever created -- How Boston Recruiters See The Software Testing Industry 2017. I've wanted to create a series of blog posts, articles, and presentations on the perspective of a Boston area recruiter -- a viewpoint that would be quite helpful to members of the new Ministry of Testing - Boston I just started to help organize -- for a while. I received a response back immediately that really opened up my eyes to the harshness of their reality.