June 20, 2017

Are you a software tester? Take the State of Testing Survey 2017, by SmartBear!

Tonight was wonderful! The Ministry of Testing - Boston gathered together in Assembly Square, Somerville, MA at the headquarters of SmartBear, maker of automated testing toolsets such as  SoapUI, SwaggerHub, and TestComplete.

SmartBear walked us through their entire testing process... They gave us so much information, it is going to take a few days for me to compile my notes!

In the meantime, drop what you are doing and fill out their survey at https://www.surveymonkey.com/r/smartbear-state-of-testing-survey-social

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

June 18, 2017

My fifth article was published on TechBeacon! "How QA can build quality into your software pipeline"

I forgot to mention... Back on May 30th, my fifth article from TechBeacon was published: "How QA can build quality into the software pipeline"!

TechBeacon always finds the cutest opening graphics for my articles!

... I do prefer my original title, which referred to the changing definition of quality as you moved from being a manual tester to an automation developer.

I can't believe it was a year ago that Rob Mitchell, Managing Editor of TechBeacon gave me a call February of 2016 after I inquired about writing for HPE's online magazine... but I had never written professionally before. What the heck do I write about?

June 17, 2017

Notes from Eran Kinsbruner talk, The Digital Quality Handbook: Guidefor Achieving Continuous Quality in a DevOps Reality (June 13, 2017)

When the Ministry of Testing - Boston met up on Tuesday, June 13th, 2017, that evening's guest was Eran Kinsbruner, Mobile Technical Evangelist at Perfecto Mobile, talk about his book, The Digital Quality Handbook: Guide for Achieving Continuous Quality in a DevOps Reality, just released on April 28, 2017!

Perfecto sponsored the Ministry of Testing - Boston event. Thank you so much for setting up the event, along with the the refreshments, Lindsay!

About the The Digital Quality Handbook:

According to Amazon:
"As mobile and web technologies continue to expand and basically drives large organizational business in virtually every vertical or industry, it is critical to understand how to take existing release practices for mobile and web apps to the next level, including software development life cycle (SDLC), tools, quality, etc. Organizations which are already enjoying the power of digital are still struggling with various challenges that can be related to many factors, such as:

June 5, 2017

What happens behind the scenes as Appium installs and launches an Android app? Examining and footnoting a log file.

This is part seven of of a seven-part blog series. Care to go back to the beginning?

Have you ever wondered what is happening behind-the-scenes as Appium launches an app? Well, there is one way to find out...

Today, we will be taking a look at the log files captured in the Appium console, generated after I launch the APIDemos-debug.apk Android app, with the help of Appium, onto an Android emulator.

... Actually, this blog article wasn't written in a day. It took more than a few weekends to thoroughly research everything... and I do mean everything!

There are possible errors or questions on the inner workings on Appium, with insightful statements such as ... "Huh? What's this?" ... They are highlighted in red.

Is there anything I am wrong about? Please let me know! I am a complete novice when it comes to Appium (although, I have been hideously busy).  I only started learning it in March of 2017. The reason I am blogging about this is because I actually want you to comment if I am completely off-base, or if I am error about the inner workings of Appium or the world of Android development.

June 4, 2017

Notes: Automation, Selenium WebDriver, and PageObjects with AndrewBoyer (5/31/2017)

Have you ever wanted to learn about Automated Testing but wasn't sure where to start?

Members of the Ministry of Testing - Boston met last week to listen and bring questions to Andrew Boyer, a Software Developer in Test who has worked at both Google and Amazon. Andrew talked with us Wednesday, May 31st, 2017 about Functional Web Automation using Selenium WebDriver. 

Andrew Boyer, SDET

About the Speaker:

Andrew Boyer has worked in the software industry for more than twenty years, for companies as small as ten people, and as large as Google and Amazon. Working at times as a developer, tester, or even a sales engineer, he focuses now on process improvements and creating automation frameworks. He believes good automation is accessible to all job roles, and improves not just the quality of the shipping product, but also the velocity of the engineering organization.

LinkedIn: https://www.linkedin.com/in/andrew-boyer-4a00685/

June 1, 2017

Moshe Milman, COO of Applitools, speaks to the Ministry of Testing - Boston (5/23/2017)

We had a special guest on Tuesday, May 23rd, 2017 coming to speak to the Ministry of Testing - Boston!

Moshe Milman, COO and co-creator of Applitools gave a talk to the Ministry of Testing - Boston called "Advanced Test Automation Techniques For Responsive Apps And Sites". The event was sponsored by Cengage Learning, a twelve minute walk from South Station in Boston, MA, at 20 Channel Center Street, Boston, MA.

May 31, 2017

Build a Basic Appium Framework: Download the tests and run them on your own MacBook!

This is part five of of a seven-part blog series. Care to go back to the beginning?

Now that we have set up a rudimentary framework, now it's time for you to download the framework we have created onto your own local machine. We will be using IntelliJ's built in feature... But you could easily use GitHub Desktop or could even use Git.

... Do you have a GitHub account? Get one! Then go through the tutorials they show you. They can teach you more than this blog ever could.

May 30, 2017

Build a Basic Appium Framework: Set up the Page Objects, Page Factories and Tests

This is part four of of a seven-part blog series. Care to go back to the beginning?

Now, let's wrap things up by setting up a few Page Objects, add the Page Factory pattern to these building blocks we are creating, and then compose the test we designed all the way in Part Two.

BeforeSuite and AfterSuite Annotations

First things first... Remember how we made in the last chapter installing and launching the app itself the test?

Let's change that, placing it in a @BeforeSuite annotation from TestNG.

May 20, 2017

Build a Basic Appium Framework: Install and Launch an App Using Desired Capabilities

This is part three of of a seven-part blog series. Care to go back to the beginning?

INTERLUDE: Overheard at my workplace... 

"Now, type on the next line, Capital-D, DesiredCapabilities, one word! Then space, the word 'caps', space, equals sign, the word 'new', Capital-D DesiredCapabilities, one word! Open-parentheses, then close-parentheses, semi-colon.

"... On the right side of the equation, we just declared a new instance of the class DesiredCapabilities, part of the Selenium WebDriver library. To the left of the equation, we are placing that value in a variable we are calling "caps", and declaring that also to be of type DesiredCapabilities...

"... Why is the word DesiredCapabilities underlined in red in our editor? We haven't imported the library into our 'Smoketest.java' class. On the MacBook, it is CTRL+Enter to import. Press those keys at the same time right after you see the DesiredCapability IntelliJ tooltip. That is IntelliJ IDEA's 'Intellisense' feature..."

So far, I have spent two half-hour pair-programming sessions attempting to train a manual tester on my team how to do automation development. It's not exactly the blind-leading-the-blind, but it is pretty darned close.

I've only been doing automation development for the past two years, hopping around from Selenium WebDriver + Java at Fitbit, Nightwatch Js + JavaScript + NodeJs for a few months at Good Start Genetics. And I only have been teaching myself Appium since March since I signed up for this six month contract at Stop & Shop's parent company, where we are designing the next generation of mobile apps for Stop & Shop, Giant Food, and Martin's.

I have been working really hard to prove my worth over the past three months as a contractor:
  • Checking to see if we can used a favored automation tool used at this company, SerenityBDD, for mobile apps. (Answer? There really isn't that much documentation or support on it). 
  • Exploring Appium Desktop, using that as a teaching tool to explain setting up an Appium environment
  • Putting together the basic Appium framework 

My latest pet project is seeing if through one-on-one training I can get a manual testers to be able to design their first automated test, then add it to my basic test framework in a pull request... all in just ten half-hour sessions.

One roadblock, besides the fact I only started teaching myself Appium + Java in March when I was hired for this contract position? I can't shake this cold I caught two weeks ago, and I keep on losing my voice. My voice is a bullfrog croak and end up shouting in my trainee's ear too loudly as I try to excitedly improvise a lesson for my first test subject, a former SysAdmin who now is a manual tester on our project. He has some tinkering in Java, but not much.

"Okay -- brace yourself! This next code block will contain a lot of typing..."

Rule Number One: Trainees have to type everything out. Every semi-colon. Every statement. Every code block.

Rule Number Two: No wizards except the first one to set up the initial project. I walked him through creating a new Gradle + Java project with IntelliJ IDEA with the New Project Wizard. How to create a new directory off of the root directory called "src". How to create a sub-directory called "test". And another one called "java".

Rule Number Three: Trainees have to do their own searching. Do we need to import a new Java library into our project? I tell the trainee to Google the word "Maven Repository". I describe to him how to go to and search the Mvnrepository.com site for a third party dependency for Appium's Java-Client library. I tell him to copy-and-paste the code from the "Gradle" tab and back into our build.gradle file, in the 'Dependencies' code block.

Rule Number Four: I saw on Twitter recently this question by a beginning programmer:

  • Is it okay to copy-and-paste code? 
  • To copy? Yes. To paste? No.
... I am trying to have the trainee follow this dictum.

... Okay, enough about work. Let's get back to learning about DesiredCapabilities, where we match the Appium Server that is running with the Android emulator that is running. Three questions though...

May 18, 2017

New published article! "Coding is key to a test automation career: Are you prepared?"

I just published my fourth article for TechBeacon!

Coding is key to a test automation career: Are you prepared?

Want to know all the resources I have used to study automation development? Give it a read!

As Always .... Happy Testing!
-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

May 17, 2017

How to create and launch an Android emulator from Android Studio

Today, let's walk through how to create and launch an Android emulator with Android Studio.

What does an Android Emulator Consist Of?

"The Android Emulator runs a full Android system stack, down to the kernel level, that includes a set of preinstalled apps (such as the dialer) that you can access from your apps. You can choose which version of the Android system you want to run in the emulator when creating AVDs.

"The Android system images available through the AVD Manager contain code for the Android Linux kernel, the native libraries, the VM, and the various Android packages (such as the Android framework and preinstalled apps)". - Developer.Android.com, Run Apps on the Android Emulator

Do You Have Android Studio Downloaded?

First things first! Do you have Android Studio downloaded and installed?

May 16, 2017

Build a Basic Appium Framework: Design a Basic Test, Examining Mobile Elements

This is part two of of a seven-part blog series. Care to go back to the beginning

Last blog post, we reviewed installation and setup. For this one, let's write a simple mobile automation test, for this app, then investigate all mobile elements involved, with Appium Desktop.

The app we will be testing, ApiDemos-debug.apk

Our Basic Test:
  • Fire up the app, make sure the title on the first screen says “API Demos”
  • Select the “Text” button. Make sure that “LogTextBox” appears as an option.
  • Select “LogTextBox”. Make sure that the “Text/LogTextBox” header appears.
  • Select the ADD button.
  • Assert that the words “This is a test” appears in the panel.

May 15, 2017

Build a Basic Appium Framework: Review How to Inspect Mobile Apps with Appium Desktop

For this next blog series, we will be building a basic Appium Framework built on top of the information gathered in the Learning Appium Desktop series. This basic mobile automation test framework we will be constructing will consist of:

  • Appium Desktop: To investigate the app we will be writing automated tests for, and to start up an Appium server on our local machine. ( Official GitHub site
  • An Android Emulator: Created by Android Studio and the Android Virtual Machine (AVM) Manager, connected to your local computer with the Android Debug Bridge ("adb"). ( Official Developer.Android.Com site on emulators )
  • The code will be written in the Java Client version of Appium using a MacBook and IntelliJ as an Integrated Development Environment (IDE). Although Android Studio, IntelliJ, and Android .apk files run on both Windows and Macs, the next version of this project will consist of emulated iPhones, iPads and Apple .ipa files use XCode, which (as far as I know) doesn't work on Windows machines. ( Official Github for appium/java-client )
  • Tests will be kicked off using TestNG. I like the annotations and the built in ability to have parallel tests running. We won't explore it in this particular series, but we will later. ( See the official TestNG site ).  
  • It will use Hamcrest, a Java matcher program in order to create very readable tests. (Official site for http://hamcrest.org/ )
I will be attempting to finish code walkthrough over the next two weeks on a Monday / Wednesday / Friday publishing schedule.

May 14, 2017

Building a Basic Appium Mobile Testing Framework

This weekend, I finally wrote my first Appium framework! I've been wanting to do that for a while! It uses Appium + Java + TestNG + Hamcrest + the Page Object Pattern + Page Factories.

ApiDemos-debug.apk is the application under test. Check out the framework at my GitHub site!

I am halfway finished through a code walkthrough I can place on my blog, a series in five parts. It should be ready in a few weeks. The goal is to give manual testers a glimpse in how an Appium framework could be created. It'll have instructions on how to download and run the tests.

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

May 12, 2017

Tues, June 13th: Book Talk: Eran Kinsbruner, The Digital Quality Handbook: Guide for Achieving Continuous Quality in a DevOps Reality, in Cambridge, MA

Come hear Eran Kinsbruner, Mobile Technical Evangelist at Perfecto Mobile, talk about his just released book, The Digital Quality Handbook: Guide for Achieving Continuous Quality in a DevOps Reality, just released on April 28, 2017!

Perfecto Mobile will be sponsoring this Ministry of Testing - Boston event. Pizza and soda will be served.

About the The Digital Quality Handbook:

According to Amazon:
"As mobile and web technologies continue to expand and basically drives large organizational business in virtually every vertical or industry, it is critical to understand how to take existing release practices for mobile and web apps to the next level, including software development life cycle (SDLC), tools, quality, etc. Organizations which are already enjoying the power of digital are still struggling with various challenges that can be related to many factors, such as:

May 9, 2017

Tues. May 23rd: Moshe Milman, COO of Applitools will be giving a talk to Ministry of Testing - Boston:

We have a special guest on Tuesday, May 23rd coming to speak to the Ministry of Testing - Boston!

Moshe Milman, COO and co-creator of Applitools will be giving a talk to the Ministry of Testing - Boston about the Best Practices for Responsive Web Design. This event will be sponsored by Cengage Learning. Please RSVP only if you are planning to attend so we can get an accurate headcount when it comes to refreshments.

Where to Meet:

Cengage Learning is a twelve minute walk from South Station. It is at 20 Channel Center Street, Boston, MA.

May 7, 2017

What happens when you ask the Twitterverse: "How Can a Manual Tester Switch To Automation?"

So, I asked the Twitterverse a question... How Can Manual Testers Switch to Automation? ... Here are the responses I received from:
  • Rosie Sherry and Richard Bradshaw, Ministry of Testing UK
  • Jason Huggins, creator of the first version of Selenium, founder of Sauce Labs and Tapster.
  • Alan Richardson, EvilTester.com
  • Dave Haeffner, ElementalSelenium.com
  • Bas Dijkstra, author of the blog OnTestAutomation.com and a free open-source workshop on REST Assured.
  • Brendan Connolly, author of the blog AssertThis!
  • Angie Jones, from AngieJones.Tech.
  • Jim Holmes, from FrazzledDad.com.

May 4, 2017

Brian Jordan, Automated Testing for the Non (and FUTURE!) Coder: Notes from his talk with the Ministry of Testing - Boston

Do you have a cool software testing technique that you would like to share with an appreciative audience? Have you been working with a new automation tool, and want to share your thoughts about it? Come give a talk to the Ministry of Testing - Boston! We meet twice a month all around Cambridge and Boston, Massachusetts.

How can you become a speaker for our group? Just ask!

On Tuesday, May 2nd, Brian Jordan, formerly from Code.Org talked about techniques used to test the lesson plans he was generating to allow kids from Kindergarten to Sixth Grade to learn how to code.

Thank you, Moshe Milman, the COO of Applitools for sponsoring that night's food and beverage! I still need to confirm the location for your talk to us in two weeks.

And thank you so much, Tomer from iZotope for letting our group crash at your place once again! If it wasn't for you, we wouldn't have a Meetup!

April 25, 2017

Learning Appium Desktop: Inspecting an app with Appium Desktop

This is Part 6 of 6. Care to go back to the beginning?

So far we have:
  • Learned how to install Appium Desktop which bundles Appium with an inspector we can use to inspect our applications. 
  • Studied how to connect an Android device via USB through the Android Command Line tool adb.
  • Connected to the device via WiFi. 
  • Learned how to set up an emulated Android device.

With this blog post we are going to:
  • Inspect an app with Appium Desktop 
  • Before we need to do that, we need to find an app's Desired Capabilities!

April 21, 2017

Ministry of Testing - Atlanta and Llewellyn Falco crash a MoT-Boston Lean Coffee!

I really didn't expect many people to show up for this month's Lean Coffee at John Harvard's Brewery in Harvard Square. We've been getting around five to ten people showing up since I held the first one back in February. Six people signed up for the Meetup event, so I expected maybe it would be four people in attendance? Me plus three? This time, though, I was in for a surprise!

I've been reaching out to other Ministry of Testing chapters here in the United States. Philly. Portland, Oregon. Atlanta. Likewise, Claire Moss, co-organizer of MoT-Atlanta, while she was attending a Boston area Agile conference, has been looking to connect to other MoT groups. Her motto on her business card? "Using my evil powers for good!"

MoT-Boston meets MoT-Atlanta. It's T.J. and Claire!

April 20, 2017

Learning Appium Desktop: Find the Desired Capabilities: appPackage and appActivity. Bug in AAPT if giving just appName

This is Part 5 of 6. Care to go back to the beginning? Come back in a few days to see the next part!

When I tried running Appium Desktop, I came across some problems. I decided on using as a sample Android app, a pre-compiled one referred to me by an Introduction to Appium using Ruby by Dave Haeffner. https://github.com/appium/ruby_lib/blob/master/android_tests/api.apk
First, I started an emulator using Android Studio. Then, I started Appium Desktop.
Then I pressed: Select Start Server v1.6.4-beta -> Select Start New Session

For Desired Capabilities:
Appium can interact with a mobile app in a few ways:
  • Is the app already on the emulator or actual device? You need to find the package name and the activity name of the app you want to run, and pass that into the DesiredCapabilities.
  • You can pass the .apk into the app field of the DesiredCapabilities, and Appium will install the app itself. You won't have to:  adb -s emulator-5554 install myapp.apk
That didn't work. I noticed a few errors when looking at the Appium Server.

April 19, 2017

Learning Appium Desktop: Setting up Android Emulators with Android Virtual Device Manager, choosing Android operating system version

This is Part 4 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Ever since April 2009, when version 1.5 of the Android mobile operating system was released, the operating systems have been code-named some type of dessert.

Android supports, at the time of this blog post, Android 6.0 (Marshmallow, API Level 23, Released October 5, 2015), Android 7.0 (Nougat, API Level 24, Released August 22, 2016), and Android 7.1.1 (Nougat, API Level 25, October 4, 2016).

We'll be setting up using Android Virtual Device Manager two virtual Android devices.

April 18, 2017

Learning Appium Desktop: Setting up remote devices through WiFi

This is Part 3 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Yesterday, we talked all about connecting Android devices to your computer through a USB cable. What we didn’t cover was that you can also do this through WiFi!

Why is this important?

Imagine that you are an automation developer who wants to test on real devices. Instead of being forced to have each and every device attached to your computer with a tangle of USB cables, you can have all devices locked down in a lab.

With an automation test you can:
  • Install an Android app on a remote device.
  • Start up the Android app
  • Run your test.
  • Uninstall the app.

… All while connected to the WiFi!

April 17, 2017

Learning Appium Desktop: How to Connect To Your Android Device Using the Android SDK, the Android Command Line Tools, and the Android Debug Bridge (adb)

This is Part 2 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Last blog post, we went over how to get the Appium Server running on a Macbook, through the new Appium Desktop product.

With this post, we are going to cover:
  • Installing the Android SDK (Software Development Kit) on our Macbook through installing Android Studio
  • How to use some command line tools such as adb 
  • How to connect your Android devices wirelessly to your system

What is the Android SDK?

From Webotopia: "A software development kit that enables developers to create applications for the Android platform. The Android SDK includes sample projects with source code, development tools, an emulator, and required libraries to build Android applications. Applications are written using the Java programming language and run on Dalvik, a custom virtual machine designed for embedded use which runs on top of a Linux kernel".

April 12, 2017

SeConf2017: Microsoft is automating Windows, Intuit automating Mac Apps, all with Appium!

Star-Driver: Appium for the Future with Dan Cuellar
Joe Colantonio, Test Talks


"Apr 7, 2017 - It was an honor to finally meet one of my open-source heroes Dan Cuellar the creator of Appium. We had a quick chat at Selenium Conference 2017 in Austin and Dan was cool enough to let me interview him on camera. Dan shared his vision for Appium to be able to automate all things not just mobile apps".

April 11, 2017

Adventures in Automation: Two hundred posts!

Wow, this is the 200th post I have written here on Adventures in Automation.

This blog started a bit over two years ago. As I was trying to do one last big push to try to (somehow) make the shift from manual to automated testing, my wife suggested that instead of the many research notes I was compiling stay on my computer, why not share them in a blog?

It was a fabulous suggestion! Not only did I have a code portfolio on GitHub that employers could view, they also received a walkthrough of the code with my blog from automating a simple site covering basic concepts, to building a test framework for more complex sites and testing more complex APIs.

SeConf2017: Security Testing, Dockerizing Tests using Appium, Notes

Security Test Driven Development (STDD) Using Selenium / Appium
Surendran Ethiraj


Although many people think security testing is important, there aren’t many people implementing it sprint-by-sprint. To really help, security tests need to be easy to run, and they should be automated. Security should be in the DEV cycle, not waiting until the product is done.

Slides: https://www.slideshare.net/ATASlides/atagtr2017-security-test-driven-development-stdd

Surendran is using OWASP ZAP (the Zed Attack Proxy Project ) , FindBugs, and TestNG to set up security tests in the video.

April 10, 2017

SeConf2017: Rethinking Appium’s Code Base, Notes

Even though I have been a QA Engineer for quite some time, I am rather new to automation development. Lately, I've been trying to figure out how to use the open-source tool, Appium, to automate the mobile test suites that I have been writing for my current position.

Luckily for me, the Selenium Conference 2017 has just posted videos of its lectures online, with many of them providing good background information about Appium.

Rethinking Appium’s Code Base
Moiz Virani – Software Engineer, Sauce Labs


Moiz Virani ( @moizjv ), a Software Engineer at Saucelabs and contributor on the Appium project talked a bit about how the Appium project was re-written and restructured. They took a monolithic project and converted it into smaller microservices.

More that iOS and Android, they now have drivers for Windows and UIDriver, helping you automate your TV apps. It still uses the JSON Wire protocol, still iis open source and always will be open source.

April 8, 2017

Couldn't attend last week's Selenium Conference? No problems: The videos are now online!

Couldn't attend the Selenium Conference Talks in Austin, Texas on April 3rd through the 5th of 2017?

No problem! The videos are now online! https://www.youtube.com/user/seleniumconf/videos .

Thank you, Angie Jones for letting me know!

April 7, 2017

Things are heating up at the Ministry of Testing - Boston!

Things are heating up at the Ministry of Testing - Boston! Are you a Software Tester, QA Engineer, Manual Tester, Automation Developer, Software Engineer in Test, SDET, SET, QA Manager, or a Directory of Quality? Come check us out! Our calendar is jam packed!

April 2017

NASA International Space Apps Challenge 2017 Info Session! 
Brooklyn Boulders Somerville
Saturday, April 8, 2017
3:00 PM to 5:00 PM
  • Anne Fraysse says, "Are you fascinated by the hidden mysteries of the universe? Intrigued by NASA? Love to build and design? Want to be a part of the largest hackathon in the world? Come join me this Saturday at Brooklyn Boulders Somerville, where the International Space Apps Challenge, lead by NASA, is hosting a info session dedicated to their upcoming hackathon - the NASA International Space Apps Challenge (ISAC) 2017 - where teams across the world compete over a 48-hour period to design new and relevant technology!" 
  • Sign up here!

James Venetsanakos talks about Performance Testing @ iZotope, Cambridge
iZotope, 60 Hampshire St, Cambridge, MA
Wednesday, April 12, 2017
6:00 PM to 8:30 PM
  • James Venetsanakos ( LinkedIn ), a seasoned performance engineer will be giving a talk about Performance Testing. Come bring any questions you may have!
  • Sign up here!

Software Testing Lean Coffee @ Harvard Square, Cambridge, MA
John Harvard's Brew House
Tuesday, April 18, 2017
6:30 PM
  • A dinner-and-discussion group run like an end-of-sprint Agile retrosprective to break the ice. What QA related topic do you want to discuss tonight?
  • Sign up here!

April 6, 2017

Learning Appium Desktop: What is Appium Server and How Do You Start It Through Appium Desktop?

I've been asking myself the same question for the past two years, ever since I first became an automation developer: I wonder if I can figure out how to create a mobile automation framework? We already started covering the basic building blocks:

Learning Serenity BDD:
  • Serenity BDD with Cucumber, formats tests to be in an easy-to-read Given / Then / When format for feature files, which then can be broken down into step definitions. This is what is already being used in my current position.
  • Serenity BDD gives us easy-to-read reports. Picture the Given / Then / When as a summary report where you can drill down in the report to see the individual steps, how they were carried out, and which individual step passed or failed.
This section will cover Learning Appium Desktop:
  • Appium Desktop is a web based product used to inspect iOS and Android apps so I can determine a locator strategy for my eventual mobile automation test framework I will be building.
  • Appium Desktop comes with Appium 1.6.4-beta as the server. We will be covering basic concepts of Appium, and doing a deep-dive on many of the Android Command Line Tools used.
  • Although Appium allows both OS and Android, I was going to focus just on Android first, configuring both physical and virtual devices. 
  • I will be setting everything up on my Macbook. Why? I really like the Mac Terminal as a Command Line Interface. 

If you need more of an introduction to what is Appiun, Appium Docker, and what Appium can do, view:


What is Appium?

From Appium History: http://appium.io/history.html

"Dan Cuellar was the Test Manager at Zoosk in 2011, when he encountered a problem. The length of the test passes on the iOS product was getting out of hand. Less testing was an option, but would come with additional risk, especially with it taking several days to get patches through the iOS App Store Review process. He thought back to his days working on websites and realized automation was the answer.

"Dan surveyed the existing landscape of tools, only to find that all of them hand major drawbacks. The tool supplied by Apple, UIAutomation, required tests to be written in JavaScript, and did not allow for real time debugging or interpretation. It also had to be executed inside the Xcode profiling tool, Instruments. Other 3rd-party tools used private APIs and required SDKs and HTTP Servers to be embedded into the application. This seemed highly undesirable.

April 5, 2017

Become an Event Organizer for the Ministry of Testing - Boston Meetup!

Going to an Event or Conference?

Planning to go to a software testing conference? Do you dislike flying solo? Don't wait for someone else to put it on the calendar. Become an event organizer and gather together a ragtag group of Ministry of Testing - Boston members to sign up to the event with you!

Want to Host a Lean Coffee?

Do you wish there were more Ministry of Testing - Boston Lean Coffees in your area? Become an Event Organizer, find a spot to meet, and host your own! (Note, for this event, it is best to cap attendance at 15 people, or the discussion may be unwieldy).

Want to Host an Event at Your Company?

Are you a QA Manager or know someone who is? Do you have a new automated toolset to demo? Want to host a roundtable discussion on testing tools and techniques? Become an Event Organizer and host your very own Ministry of Testing - Boston Meetup event at your workplace! It's a great way to get the word out about open job reqs, testing tools you are creating, or simply a way donate your wisdom to the testing community.

Write to Conrad Holloman or T.J. Maher to become an Event Organizer for the Ministry of Testing - Boston group.

Once you become an Event Organizer, you will have access to the Ministry of Testing - Boston calendar. Create a new event!
- T.J. Maher, @tjmaher1­, Co-Organizer, Ministry of Testing - Boston

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

April 3, 2017

Webinar Notes: An Introduction to Appium Desktop by Johnathan Lipps

About the Webinar

From Sauce Labs: An Introduction to Appium Desktop (3/29/2017):

"Appium Desktop is a new graphical interface for starting an Appium server and inspecting your app's structure via Appium. It's recently been developed by the Appium contributors at Sauce Labs and is currently in open beta. In this webinar we will take you on a tour of Appium Desktop and show how it can be used with the goal of making it easier to write tests for your apps.

"Join Jonathan Lipps, Appium project lead committer and Director of Open Source for Sauce Labs, as he covers the following topics:
  • "Finding, downloading, and installing Appium Desktop for your platform
  • "Starting an Appium server with both simple and advanced options
  • "Interpreting Appium server log output
  • "Starting an inspector session using just Appium Desktop's server
  • "Starting an inspector session using Sauce Labs or TestObject
  • "Troubleshooting and where to go for help
  • "Appium Desktop's roadmap"
Video: An Introduction to Appium Desktop (55 minutes) https://youtu.be/IOSUBda2-g4

Slides: Introducing Appium Desktop:

Who is Johnathan Lipps?

According to his LinkedIn profile, as Director of Open Source at Sauce Labs: "I lead open source strategy, developer advocacy, and technological evangelism for the world's largest test cloud. In addition to ensuring the open source testing ecosystem is healthy, I lead development of Appium, the mobile automation framework, enabling a whole new level of automated testing for mobile devices and beyond".

...also according to LinkedIn: "I'm a software developer, a passionate leader, a linguist, a philosopher, a writer, a musician, a rock climber, and a runner".

Project Lead for the Appium project.

Johnathan Lipps metioned that Appium is Sauce Labs' biggest project, and have been working on it for quite some time. "Appium is the cross-platform solution for native, web, and hybrid mobile automation." With it, you can automate tests the apps you care about.

The main product in the Appium toolset is the Appium server, an invisible piece of software, installed by the Node Package Manager (npm) that sits on top of Node.JS.

The Appium server runs as a server on your machine, remotely, or in the cloud, such as with Sauce Labs, "enabling you to send requests for sessions, and within those sessions to send automation commands to [...] tell your device to do things".

The Appium Desktop (new!) is a point-and-click interface for using appium and inspecting your app's structure. The inspector helps you find web elements such as a button so you can then make an assertion. It is basicallt a wrapper for Appium.

Launch the desktop app, which provides a GUI for the exact same messages you would see in the command line. There used to be appium.app and appium.exe where you could also use Appium with a point-and-click interface. They were good applications that fell by the wayside. Appium Desktop is a "spiritual successor".

Differences Between Appium Command Line Interface (CLI) and Appium Desktop:

From Introducing Appium Desktop

Download and Install Appium Desktop

Appium Desktop is at https://github.com/appium/appium-desktop/releases

You can choose to adjust the port or the host if you need, and you start a simple server.
You can also select "Advanced" and play around with various options.

Make sure to read the documentation, since this will change how Appium behaves.

You can run from:

  • Automatic Server
  • Custom Servers
  • Sauce Labs
  • Test Object

You can also set Desired Capabilities such as platformName : iOS.

What are Desired Capabilities?

"Desired capabilities are a set of keys and values (i.e., a map or hash) sent to the Appium server to tell the server what kind of automation session we’re interested in starting up. There are also various capabilities which can modify the behavior of the server during automation. For example, we might set the platformName capability to iOS to tell Appium that we want an iOS session, rather than an Android or Windows one. Or we might set the safariAllowPopups capability to true in order to ensure that, during a Safari automation session, we’re allowed to use JavaScript to open up new windows". - Appium API Reference

If you have a huge set of Desired Capabilities you can edit in the GUI of Appium Desktop and see it in the JSON object that the Appium will interpret.

Appium server capabilities

automationNameWhich automation engine to useAppium (default) or Selendroid
platformNameWhich mobile OS platform to useiOSAndroid, or FirefoxOS
platformVersionMobile OS versione.g., 7.14.4
deviceNameThe kind of mobile device or emulator to useiPhone SimulatoriPad SimulatoriPhone Retina 4-inchAndroid EmulatorGalaxy S4, etc…. On iOS, this should be one of the valid devices returned by instruments with instruments -s devices. On Android this capability is currently ignored, though it remains required.
appThe absolute local path or remote http URL to an .ipa or .apk file, or a .zip containing one of these. Appium will attempt to install this app binary on the appropriate device first. Note that this capability is not required for Android if you specify appPackage and appActivity capabilities (see below). Incompatible with browserName./abs/path/to/my.apk or http://myapp.com/app.ipa
browserNameName of mobile web browser to automate. Should be an empty string if automating an app instead.'Safari’ for iOS and 'Chrome’, 'Chromium’, or 'Browser’ for Android
newCommandTimeoutHow long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the sessione.g. 60
language(Sim/Emu-only) Language to set for the simulator / emulatore.g. fr
locale(Sim/Emu-only) Locale to set for the simulator / emulatore.g. fr_CA
udidUnique device identifier of the connected physical devicee.g. 1ae203187fc012g
orientation(Sim/Emu-only) start in a certain orientationLANDSCAPE or PORTRAIT
autoWebviewMove directly into Webview context. Default falsetruefalse
Go to https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/caps.md to see more Android and iOS specific Desired Capabilities!

How to Use The Inspector:

Start the Session. You need to install Android SDKs of emulators on your local machine. After seeing the simulator, you can see the inspector window. What is going on with different elements?

You can navigate the XML source, the response tree. This is how you can find the id or CSS Selector that you then can call.

... It is like the Firefox plugin, Firebug and Firepath used to determine strategies to find locators.

You can perform actions directly from Appium Desktop, such as sending keys to a textbox.

Note, if you input your Sauce Labs credentials you can use their virtual emulators of Android or IOS devices.

See any issues? Are the issues with:

  • Appium? https://github.com/appium/appium/issues
  • Appium Desktop? https://github.com/appium/appium-desktop/issues

Want to See More Webinars from Sauce L?

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

April 2, 2017

Learning Serenity: The Screenplay Tutorial

We've covered a lot with Serenity BDD over the past few weeks!

This has all prepared us for John Ferguson Smart's tutorial on the Serenity BDD Screenplay Pattern. See http://serenity-bdd.info/docs/articles/screenplay-tutorial.html

Why Replace the Page Object Pattern with The Screenplay Pattern? 

Personally, I am trying to learn Serenity BDD and the Screenplay pattern because it is already the toolset and techniques we are already using where I am assigned as a contractor. I just started my career in automated testing two years ago.

Why does John Ferguson Smart and his group recommend this pattern they created? From Page Objects Refactored: SOLID steps to the Screenplay / Journey Pattern (2/11/2016):
"PageObjects have been the staple of automated web testing for over seven years. Simon Stewart wrote the original Selenium PageObject wiki entry in 2009. The concept was introduced to help test-developers reduce maintenance issues that, for many, resulted in flaky tests. Some mistook this as issues with Selenium, rather than with their own coding practices. While PageObjects can act as training wheels and be a good place to start, in this article we show you issues inherent to PageObjects and the benefits of refactoring them using SOLID OO principles. We then introduce the Screenplay Pattern, an alternative approach that could save you the trouble".
... I also heard that the Screenplay pattern has Appium built in as a way to kick off mobile automation. I'll be investigating that, next!

How to Get Started

Following the instructions on Writing automated acceptance tests using Serenity and the Screenplay Pattern:

"The easiest way to create a project skeleton for a Serenity Screenplay project is to use the Maven Archetype Plugin. To do this, run the mvn archetype:generate command (with a filter to reduce the number of artifacts Maven proposes) as shown here:"
$ mvn archetype:generate -Dfilter=screenplay
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: remote -> net.serenity-bdd:serenity-junit-screenplay-archetype (Serenity automated acceptance testing project using Screenplay, Selenium 2 and JUnit)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): :
"[...] You will then be prompted to enter a groupId, artifactId, and version for your project, and a root package for your classes [...]".

Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Define value for property 'groupId': : net.serenitybdd.tutorials
Define value for property 'artifactId': : todomvctests
Define value for property 'version':  1.0-SNAPSHOT: : 1.0.0-SNAPSHOT
Define value for property 'package':  net.serenitybdd.tutorials: :
Confirm properties configuration:
groupId: net.serenitybdd.tutorials
artifactId: todomvctests
version: 1.0.0-SNAPSHOT
package: net.serenitybdd.tutorials
 Y: : Y
"You will find your new project in the screenplay-tutorial directory".

$ cd screenplay-tutorial
$ mvn clean verify
... And the tests should run!

Explore the tutorial to learn more about all the moving parts as shown below:


And thank you for Learning Serenity with me! Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 31, 2017

Learning Serenity: Scaffolding a new project with Maven Archetypes

Imagine that you are starting a new job, where you encounter yet another new automated toolset you haven't heard of before. It means that you have:

This is why I blog: The only way to keep my head on straight is to take copious amounts of research notes. This blog is where I store them.

Am I struggling with a concept that I just can't get? I've befriended a few experts in my very short time as an automation developer (Hi, Alan, Dave, Joe, Jim, James M, Joon, Lark, Martin, James V, Bas, and Angie!) who really enjoy helping out beginners like me out. If I blog about a problem I am having, and point to the post on Twitter, I get excellent feedback from the automation community. Thank you all so much!

March 29, 2017

March 29, 2017: Talks on Performance Testing and REST Assured at the Ministry of Testing - Boston Meetup

The Ministry of Testing - Boston Meetup held it's March event on Monday, March 29th, once again at iZotope in the Kendall Square, MIT, Cambridge, MA area. Thank you so much Tomer for letting us meet there!

Before the Meetup

We had two special guests!

James Venetsanakos, Performance and Automation Testing Architect, gave a ten minute teaser for his upcoming talk on Performance Testing for the Ministry of Testing - Boston Meetup that will be held on on Wednesday, April 12, 2017. 
James giving his 10 Minute Lightning Talk on Performance Testing

You can watch the introduction to the 10 Minute Lightning Talk he gave below!


Bas Dijkstra, a Consultant in Test automation and  Service virtualization submitted a 99 Second Talk to the Ministry of Testing, Boston. Thank you so much, Bas!

After the talks, we held a roundtable discussion, talking about everything from how to organize your page objects, pluses and minuses on Open Source vs Paid load testing tools, the Agile Testing Quadrant and the Testing Pyramid, and best practices for automation.

I cannot wait for James' talk in April!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 27, 2017

Learning Serenity: Studying BDD using The Cucumber Book and BDD in Action

It’s been a productive two weeks here at Ahold in Quincy, MA. They are the the holding company for Stop & Shop, Giant Foods, and Martin’s, where I am testing mobile applications for the iPhone, iPad and Android.

Although I’m mostly focused on manual testing as I am attempting to get to know the business, I’ve been trying to get myself up to speed with the automation framework they are using, Serenity BDD. I have a long term goal of combining their current Serenity BDD test suite with Appium, so when we start developing a regression test suite, I can possibly automate it.

As I Tweeted a few days ago… Oh, this is going to take a while!

Hrm… thank you, Serenity BDD! I need to look into that.

Oh, and thank you very much, John Smart, for your words of encouragement on this blog series:

For now, since I am studying how to implement Serenity using a testing framework invented by Aslak Hellesoy called "Cucumber".

BDD and The Cucumber Book

For those who were wondering, the “BDD” in Serenity BDD stands for “Behavior Driven Development”. I first encountered BDD when I was supervising an offshore testing team attempting to translate my manual mobile test scripts into Cucumber / Ruby automation using Calabash-IOS. That is when I first encountered Matt Wynne’s “The Cucumber Book: Behaviour-Driven Development for Testers and Developers” (2012), which was quite helpful describing BDD.
Matt Wynne and Aslak Hellesøy [the creator of Cucumber] show you how to express your customers’ wild ideas as a set of clear, executable specifications that everyone on the team can read. You’ll learn how to feed those examples into Cucumber and let it guide your development. You’ll build just the right code to keep your customers happy, and not a line more. Although it was born in the Ruby community, you can use Cucumber to test almost any system, from a simple shell script or Perl script, to web applications written in PHP, Java, or any platform”. - The Cucumber Book, The Pragmatic Bookshelf 

About the Authors

Note: The Cucumber Book now a second edition, released on February 27, 2017.

Matt Wynne is a long-standing member of the Cucumber core team, fascinated by the challenge of helping tech and business to understand one another. He's one of the co-founders of Cucumber Ltd., the company behind Cucumber. He lives on the west coast of Scotland on an old farm with his family, two cats, their dog, and some ducks. Matt tweets from @mattwynne and @cucumberbdd.

Aslak Hellesoy is the creator of Cucumber. During his career Aslak has worked with both small and large organizations in industries such as telecom, trading, insurance, car manufacturing, education, and government. Aslak is a co-founder of Cucumber Ltd, the company behind Cucumber. He tweets from @aslak_hellesoy and @cucumberbdd.

"Steve Tooke is a programmer, trainer, and coach who is dedicated to improving his craft and helping others improve theirs. He's a co-founder of Cucumber Ltd. Steve tweets from @tooky and @cucumberbdd”.

Oh, and Why Is It Called "Cucumber"?

Why the name "Cucumber"? Read Aslak Hellesoy's answer on Quora, how he asked his fiancee, Patty for naming suggestions:

"Patty, I need a name for this new tool I just started hacking on. I want it to have a catchy, non-geeky sounding name.

"She paused for a few seconds, then said: Cucumber!

"And I thought: Cucumber? Really? Well, it's a lot better than Stories - so I'll go with that for now. I'll rename it again when I come up with something better".

What is an Acceptance Test for a Feature?

Here’s an example Matt Wynne lists in The Cucumber Book:

Feature: Sign up
     Sign up should be quick and friendly.
Scenario: Successful sign up
New users should get a confirmation email and be greeted personally by the site once signed in.
Given I have chosen to sign up
When I sign up with valid details
Then I should receive a confirmation email
And I should see a personalized greeting message

… This is an example of what is called an acceptance test. “Instead of a business stakeholder passing requirements to the development team without much opportunity for feedback, the developer and stakeholder collaborate to write automated tests that express the outcome that the stakeholder wants. We call them acceptance tests because they express what the software needs to do in order for the stakeholder to find it acceptable. The test fails at the time of writing, because no code has been written yet, but it captures what the stakeholder cares about and gives everyone a clear signal as to what it will take to be done”.

The Product Owner, a business analyst representing the customer’s wants and needs on an Agile Software Development project can express a client’s wishes and expectations in an easy-to-read format. They can be expressed in terms of behavior wanted in the finished product.

As Matt Wynne mentions, “Acceptance tests written in this style become more than just tests; they are executable specifications […]
“For many teams, they become the definitive source of truth as to what the system does. Having a single place to go for this information saves a lot of time that is often wasted trying to keep requirements documents, tests, and code all in sync. It also helps to build trust within the team, because different parts of the team no longer have their own personal versions of the truth”. - The Cucumber Book

How Do Features Get Turned Into Steps and Step Definitions?  

But how does it work?
“When you run [Cucumber], it reads in your specifications from plain-language text files called features, examines them for scenarios to test, and runs the scenarios against your system. Each scenario is a list of steps for Cucumber to work through. So that Cucumber can understand these feature files, they must follow some basic syntax rules. The name for this set of rules is Gherkin. Along with the features, you give Cucumber a set of step definitions, which map the business-readable language of each step into Ruby code to carry out whatever action is being described by the step. In a mature test suite, the step definition itself will probably just be one or two lines of Ruby that delegate to a library of support code, specific to the domain of your application, that knows how to carry out common tasks on the system. Normally that will involve using an automation library, like the browser automation library Capybara, to interact with the system itself”. - The Cucumber Book

Does Serenity BDD Still Use Features Files and Step Definitions? 

If you are using Serenity BDD with Cucumber, it will follow the same exact steps.

  • Narratives describe the story you want to tell.
  • Feature files describe the features of the product, breaking it down into scenarios outlining how the feature will be used.
  • Step definitions walk through step-by-step how to execute the code.

“Step definitions sit right on the boundary between the business’s domain and the programmer’s domain. […] Their responsibility is to translate each plain-language step in your Gherkin scenarios into concrete actions in […] code”.

What is the difference between Steps and Step Definitions?

“Each Gherkin scenario is made up of a series of steps, written in plain language. On its own, a step is just documentation; it needs a step definition to bring it to life. A step definition is a piece of Ruby code that says to Cucumber, ‘If you see a step that looks like this…, then here’s what I want you to do….’ When Cucumber tries to execute each step, it looks for a matching step definition to execute. So, how does Cucumber match a step definition to a step"
“Gherkin steps are expressed in plain text. Cucumber scans the text of each step for patterns that it recognizes, which you define using a regular expression. If you haven’t used regular expressions before, then just think of them like a slightly more sophisticated version of the wildcards you’d use to search for a file”. - The Cucumber Book

What is the Difference Between BDD and ATDD?

To answer this questions, let's jump to a sidebar in “BDD in Action” written by John Ferguson Smart, the creator of Serenity BDD.

BDD by any other name

“Many of the ideas around BDD are not new and have been practiced for many years under a number of different names. Some of the more common terms used for these practices include Acceptance-Test-Driven Development, Acceptance Test-Driven Planning, and Specification by Example. To avoid confusion, let’s clarify a few of these terms in relation to BDD.

Specification by Example describes the set of practices that have emerged around using examples and conversation to discover and describe requirements. In his seminal book of the same name,14 Gojko Adzic chose this term as the most representative name to refer to these practices. Using conversation and examples to specify how you expect a system to behave is a core part of BDD, and we’ll discuss it at length in the first half of this book.

Acceptance-Test-Driven Development (ATDD) is now a widely used synonym for Specification by Example, but the practice has existed in various forms since at least the late 1990s. Kent Beck and Martin Fowler mentioned the concept in 2000,15 though they observed that it was difficult to implement acceptance criteria in the form of conventional unit tests at the start of a project. But unit tests aren’t the only way to write automated acceptance tests, and since at least the early 2000s, innovative teams have asked users to contribute to executable acceptance tests and have reaped the benefits

Acceptance-Test-Driven Planning is the idea that defining acceptance criteria for a feature leads to better estimates than doing a task breakdown”.

What is the Goal of BDD?

“One of the key goals of BDD is to ensure that everyone has a clear understanding of what a project is trying to deliver, and of the underlying business objectives of the project. This, in itself, goes a long way toward ensuring that the application actually meets these objectives.
“You can achieve this by working with users and other stakeholders to define or clarify a set of high-level business goals for the application. These goals should provide a concise vision of what you need to build. Business goals are about delivering value, so it’s common to see them expressed in terms of increasing or protecting revenue, or of decreasing costs”. - BDD in Action

Now that the research has been done, we can start assembling a new Serenity BDD project..

Until then, Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"