April 25, 2017

Learning Appium Desktop: Inspecting an app with Appium Desktop

This is Part 6 of 6. Care to go back to the beginning?

So far we have:
  • Learned how to install Appium Desktop which bundles Appium with an inspector we can use to inspect our applications. 
  • Studied how to connect an Android device via USB through the Android Command Line tool adb.
  • Connected to the device via WiFi. 
  • Learned how to set up an emulated Android device.

With this blog post we are going to:
  • Inspect an app with Appium Desktop 
  • Before we need to do that, we need to find an app's Desired Capabilities!

April 21, 2017

Ministry of Testing - Atlanta and Llewellyn Falco crash a MoT-Boston Lean Coffee!

I really didn't expect many people to show up for this month's Lean Coffee at John Harvard's Brewery in Harvard Square. We've been getting around five to ten people showing up since I held the first one back in February. Six people signed up for the Meetup event, so I expected maybe it would be four people in attendance? Me plus three? This time, though, I was in for a surprise!

I've been reaching out to other Ministry of Testing chapters here in the United States. Philly. Portland, Oregon. Atlanta. Likewise, Claire Moss, co-organizer of MoT-Atlanta, while she was attending a Boston area Agile conference, has been looking to connect to other MoT groups. Her motto on her business card? "Using my evil powers for good!"

MoT-Boston meets MoT-Atlanta. It's T.J. and Claire!

April 20, 2017

Learning Appium Desktop: Find the Desired Capabilities: appPackage and appActivity. Bug in AAPT if giving just appName

This is Part 5 of 6. Care to go back to the beginning? Come back in a few days to see the next part!

When I tried running Appium Desktop, I came across some problems. I decided on using as a sample Android app, a pre-compiled one referred to me by an Introduction to Appium using Ruby by Dave Haeffner. https://github.com/appium/ruby_lib/blob/master/android_tests/api.apk
First, I started an emulator using Android Studio. Then, I started Appium Desktop.
Then I pressed: Select Start Server v1.6.4-beta -> Select Start New Session

For Desired Capabilities:
Appium can interact with a mobile app in a few ways:
  • Is the app already on the emulator or actual device? You need to find the package name and the activity name of the app you want to run, and pass that into the DesiredCapabilities.
  • You can pass the .apk into the app field of the DesiredCapabilities, and Appium will install the app itself. You won't have to:  adb -s emulator-5554 install myapp.apk
That didn't work. I noticed a few errors when looking at the Appium Server.

April 19, 2017

Learning Appium Desktop: Setting up Android Emulators with Android Virtual Device Manager, choosing Android operating system version

This is Part 4 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Ever since April 2009, when version 1.5 of the Android mobile operating system was released, the operating systems have been code-named some type of dessert.

Android supports, at the time of this blog post, Android 6.0 (Marshmallow, API Level 23, Released October 5, 2015), Android 7.0 (Nougat, API Level 24, Released August 22, 2016), and Android 7.1.1 (Nougat, API Level 25, October 4, 2016).

We'll be setting up using Android Virtual Device Manager two virtual Android devices.

But what devices should we set up? For that, we can go to the Android Dashboards at https://developer.android.com/about/dashboards/index.html

2.3.3 -
4.0.3 -
Ice Cream Sandwich150.9%
4.1.xJelly Bean163.5%
Data collected during a 7-day period ending on April 3, 2017.
Any versions with less than 0.1% distribution are not shown.

... And it looks like with screen sizes, most people seem to be using a Normal screen, hdpi or xhdpi.

Let's remember that when we create two emulated devices.

Android Virtual Device Manager

Dependencies and prerequisites
  • Android Studio 2.0 or higher
  • SDK Tools 25.0.10 or higher
  • adb integration enabled through Android Studio > Tools > Android > Enable ADB Integration

The easiest way to kick this off is through Android Studio:

  • Launch Android Studio
  • Go to Tools -> Android -> AVD Manager

Once we are at the "Your Virtual Devices" homescreen, let's select "Create Virtual Devices".

Randomly picking two emulators, I chose:

  • Nexus S, with 480 x 800 resolution, to capture the hdpi screen type. 
  • Galaxy Nexus, with 720 x 1280 resolution, to capture the xhdpi screen type. 

When selecting Operating Systems, let's choose the updated version of Nougat, and the updated version of Marshmallow to start. Later on, it would be wise to also test on Lollipop, since so many seem to be still using that Android operating system version.

I needed to download the older versions of Marshmallow, and Lollipop. They were installed by default in ~/Library/Android/sdk.

Nexus S, I paired with Marshmallow, Android 6.0, and gave it an AVD Name of Nexus S Marshmallow.
Galaxy Nexus, I paired with Nougat, Android 7.1.1, and dubbed it Galaxy Nexus Nougat.

Need more help? See Developer.Android's section on Create and Manage Virtual Devices at https://developer.android.com/studio/run/managing-avds.html

If you go to the Mac Terminal, you will see if you type: emulator -list-avds


Now, we can go back to the AVD Manager, and launch the Nexus_S_Marshmallow.

Going to the Mac Terminal and running: adb devices

List of devices attached
emulator-5554   device

So, now we have two emulators set up for testing, with one of them running!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

April 18, 2017

Learning Appium Desktop: Setting up remote devices through WiFi

This is Part 3 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Yesterday, we talked all about connecting Android devices to your computer through a USB cable. What we didn’t cover was that you can also do this through WiFi!

Why is this important?

Imagine that you are an automation developer who wants to test on real devices. Instead of being forced to have each and every device attached to your computer with a tangle of USB cables, you can have all devices locked down in a lab.

With an automation test you can:
  • Install an Android app on a remote device.
  • Start up the Android app
  • Run your test.
  • Uninstall the app.

… All while connected to the WiFi!

April 17, 2017

Learning Appium Desktop: How to Connect To Your Android Device Using the Android SDK, the Android Command Line Tools, and the Android Debug Bridge (adb)

This is Part 2 of 6. Care to go back to the beginning? Tune in tomorrow to see the next part!

Last blog post, we went over how to get the Appium Server running on a Macbook, through the new Appium Desktop product.

With this post, we are going to cover:
  • Installing the Android SDK (Software Development Kit) on our Macbook through installing Android Studio
  • How to use some command line tools such as adb 
  • How to connect your Android devices wirelessly to your system

What is the Android SDK?

From Webotopia: "A software development kit that enables developers to create applications for the Android platform. The Android SDK includes sample projects with source code, development tools, an emulator, and required libraries to build Android applications. Applications are written using the Java programming language and run on Dalvik, a custom virtual machine designed for embedded use which runs on top of a Linux kernel".

April 12, 2017

SeConf2017: Microsoft is automating Windows, Intuit automating Mac Apps, all with Appium!

Star-Driver: Appium for the Future with Dan Cuellar
Joe Colantonio, Test Talks


"Apr 7, 2017 - It was an honor to finally meet one of my open-source heroes Dan Cuellar the creator of Appium. We had a quick chat at Selenium Conference 2017 in Austin and Dan was cool enough to let me interview him on camera. Dan shared his vision for Appium to be able to automate all things not just mobile apps".

April 11, 2017

Adventures in Automation: Two hundred posts!

Wow, this is the 200th post I have written here on Adventures in Automation.

This blog started a bit over two years ago. As I was trying to do one last big push to try to (somehow) make the shift from manual to automated testing, my wife suggested that instead of the many research notes I was compiling stay on my computer, why not share them in a blog?

It was a fabulous suggestion! Not only did I have a code portfolio on GitHub that employers could view, they also received a walkthrough of the code with my blog from automating a simple site covering basic concepts, to building a test framework for more complex sites and testing more complex APIs.

SeConf2017: Security Testing, Dockerizing Tests using Appium, Notes

Security Test Driven Development (STDD) Using Selenium / Appium
Surendran Ethiraj


Although many people think security testing is important, there aren’t many people implementing it sprint-by-sprint. To really help, security tests need to be easy to run, and they should be automated. Security should be in the DEV cycle, not waiting until the product is done.

Slides: https://www.slideshare.net/ATASlides/atagtr2017-security-test-driven-development-stdd

Surendran is using OWASP ZAP (the Zed Attack Proxy Project ) , FindBugs, and TestNG to set up security tests in the video.

April 10, 2017

SeConf2017: Rethinking Appium’s Code Base, Notes

Even though I have been a QA Engineer for quite some time, I am rather new to automation development. Lately, I've been trying to figure out how to use the open-source tool, Appium, to automate the mobile test suites that I have been writing for my current position.

Luckily for me, the Selenium Conference 2017 has just posted videos of its lectures online, with many of them providing good background information about Appium.

Rethinking Appium’s Code Base
Moiz Virani – Software Engineer, Sauce Labs


Moiz Virani ( @moizjv ), a Software Engineer at Saucelabs and contributor on the Appium project talked a bit about how the Appium project was re-written and restructured. They took a monolithic project and converted it into smaller microservices.

More that iOS and Android, they now have drivers for Windows and UIDriver, helping you automate your TV apps. It still uses the JSON Wire protocol, still iis open source and always will be open source.

April 8, 2017

Couldn't attend last week's Selenium Conference? No problems: The videos are now online!

Couldn't attend the Selenium Conference Talks in Austin, Texas on April 3rd through the 5th of 2017?

No problem! The videos are now online! https://www.youtube.com/user/seleniumconf/videos .

Thank you, Angie Jones for letting me know!

April 7, 2017

Things are heating up at the Ministry of Testing - Boston!

Things are heating up at the Ministry of Testing - Boston! Are you a Software Tester, QA Engineer, Manual Tester, Automation Developer, Software Engineer in Test, SDET, SET, QA Manager, or a Directory of Quality? Come check us out! Our calendar is jam packed!

April 2017

NASA International Space Apps Challenge 2017 Info Session! 
Brooklyn Boulders Somerville
Saturday, April 8, 2017
3:00 PM to 5:00 PM
  • Anne Fraysse says, "Are you fascinated by the hidden mysteries of the universe? Intrigued by NASA? Love to build and design? Want to be a part of the largest hackathon in the world? Come join me this Saturday at Brooklyn Boulders Somerville, where the International Space Apps Challenge, lead by NASA, is hosting a info session dedicated to their upcoming hackathon - the NASA International Space Apps Challenge (ISAC) 2017 - where teams across the world compete over a 48-hour period to design new and relevant technology!" 
  • Sign up here!

James Venetsanakos talks about Performance Testing @ iZotope, Cambridge
iZotope, 60 Hampshire St, Cambridge, MA
Wednesday, April 12, 2017
6:00 PM to 8:30 PM
  • James Venetsanakos ( LinkedIn ), a seasoned performance engineer will be giving a talk about Performance Testing. Come bring any questions you may have!
  • Sign up here!

Software Testing Lean Coffee @ Harvard Square, Cambridge, MA
John Harvard's Brew House
Tuesday, April 18, 2017
6:30 PM
  • A dinner-and-discussion group run like an end-of-sprint Agile retrosprective to break the ice. What QA related topic do you want to discuss tonight?
  • Sign up here!

April 6, 2017

Learning Appium Desktop: What is Appium Server and How Do You Start It Through Appium Desktop?

I've been asking myself the same question for the past two years, ever since I first became an automation developer: I wonder if I can figure out how to create a mobile automation framework? We already started covering the basic building blocks:

Learning Serenity BDD:
  • Serenity BDD with Cucumber, formats tests to be in an easy-to-read Given / Then / When format for feature files, which then can be broken down into step definitions. This is what is already being used in my current position.
  • Serenity BDD gives us easy-to-read reports. Picture the Given / Then / When as a summary report where you can drill down in the report to see the individual steps, how they were carried out, and which individual step passed or failed.
This section will cover Learning Appium Desktop:
  • Appium Desktop is a web based product used to inspect iOS and Android apps so I can determine a locator strategy for my eventual mobile automation test framework I will be building.
  • Appium Desktop comes with Appium 1.6.4-beta as the server. We will be covering basic concepts of Appium, and doing a deep-dive on many of the Android Command Line Tools used.
  • Although Appium allows both OS and Android, I was going to focus just on Android first, configuring both physical and virtual devices. 
  • I will be setting everything up on my Macbook. Why? I really like the Mac Terminal as a Command Line Interface. 

If you need more of an introduction to what is Appiun, Appium Docker, and what Appium can do, view:


What is Appium?

From Appium History: http://appium.io/history.html

"Dan Cuellar was the Test Manager at Zoosk in 2011, when he encountered a problem. The length of the test passes on the iOS product was getting out of hand. Less testing was an option, but would come with additional risk, especially with it taking several days to get patches through the iOS App Store Review process. He thought back to his days working on websites and realized automation was the answer.

"Dan surveyed the existing landscape of tools, only to find that all of them hand major drawbacks. The tool supplied by Apple, UIAutomation, required tests to be written in JavaScript, and did not allow for real time debugging or interpretation. It also had to be executed inside the Xcode profiling tool, Instruments. Other 3rd-party tools used private APIs and required SDKs and HTTP Servers to be embedded into the application. This seemed highly undesirable.

April 5, 2017

Become an Event Organizer for the Ministry of Testing - Boston Meetup!

Going to an Event or Conference?

Planning to go to a software testing conference? Do you dislike flying solo? Don't wait for someone else to put it on the calendar. Become an event organizer and gather together a ragtag group of Ministry of Testing - Boston members to sign up to the event with you!

Want to Host a Lean Coffee?

Do you wish there were more Ministry of Testing - Boston Lean Coffees in your area? Become an Event Organizer, find a spot to meet, and host your own! (Note, for this event, it is best to cap attendance at 15 people, or the discussion may be unwieldy).

Want to Host an Event at Your Company?

Are you a QA Manager or know someone who is? Do you have a new automated toolset to demo? Want to host a roundtable discussion on testing tools and techniques? Become an Event Organizer and host your very own Ministry of Testing - Boston Meetup event at your workplace! It's a great way to get the word out about open job reqs, testing tools you are creating, or simply a way donate your wisdom to the testing community.

Write to Conrad Holloman or T.J. Maher to become an Event Organizer for the Ministry of Testing - Boston group.

Once you become an Event Organizer, you will have access to the Ministry of Testing - Boston calendar. Create a new event!
- T.J. Maher, @tjmaher1­, Co-Organizer, Ministry of Testing - Boston

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

April 3, 2017

Webinar Notes: An Introduction to Appium Desktop by Johnathan Lipps

About the Webinar

From Sauce Labs: An Introduction to Appium Desktop (3/29/2017):

"Appium Desktop is a new graphical interface for starting an Appium server and inspecting your app's structure via Appium. It's recently been developed by the Appium contributors at Sauce Labs and is currently in open beta. In this webinar we will take you on a tour of Appium Desktop and show how it can be used with the goal of making it easier to write tests for your apps.

"Join Jonathan Lipps, Appium project lead committer and Director of Open Source for Sauce Labs, as he covers the following topics:
  • "Finding, downloading, and installing Appium Desktop for your platform
  • "Starting an Appium server with both simple and advanced options
  • "Interpreting Appium server log output
  • "Starting an inspector session using just Appium Desktop's server
  • "Starting an inspector session using Sauce Labs or TestObject
  • "Troubleshooting and where to go for help
  • "Appium Desktop's roadmap"
Video: An Introduction to Appium Desktop (55 minutes) https://youtu.be/IOSUBda2-g4

Slides: Introducing Appium Desktop:

Who is Johnathan Lipps?

According to his LinkedIn profile, as Director of Open Source at Sauce Labs: "I lead open source strategy, developer advocacy, and technological evangelism for the world's largest test cloud. In addition to ensuring the open source testing ecosystem is healthy, I lead development of Appium, the mobile automation framework, enabling a whole new level of automated testing for mobile devices and beyond".

...also according to LinkedIn: "I'm a software developer, a passionate leader, a linguist, a philosopher, a writer, a musician, a rock climber, and a runner".

Project Lead for the Appium project.

Johnathan Lipps metioned that Appium is Sauce Labs' biggest project, and have been working on it for quite some time. "Appium is the cross-platform solution for native, web, and hybrid mobile automation." With it, you can automate tests the apps you care about.

The main product in the Appium toolset is the Appium server, an invisible piece of software, installed by the Node Package Manager (npm) that sits on top of Node.JS.

The Appium server runs as a server on your machine, remotely, or in the cloud, such as with Sauce Labs, "enabling you to send requests for sessions, and within those sessions to send automation commands to [...] tell your device to do things".

The Appium Desktop (new!) is a point-and-click interface for using appium and inspecting your app's structure. The inspector helps you find web elements such as a button so you can then make an assertion. It is basicallt a wrapper for Appium.

Launch the desktop app, which provides a GUI for the exact same messages you would see in the command line. There used to be appium.app and appium.exe where you could also use Appium with a point-and-click interface. They were good applications that fell by the wayside. Appium Desktop is a "spiritual successor".

Differences Between Appium Command Line Interface (CLI) and Appium Desktop:

From Introducing Appium Desktop

Download and Install Appium Desktop

Appium Desktop is at https://github.com/appium/appium-desktop/releases

You can choose to adjust the port or the host if you need, and you start a simple server.
You can also select "Advanced" and play around with various options.

Make sure to read the documentation, since this will change how Appium behaves.

You can run from:

  • Automatic Server
  • Custom Servers
  • Sauce Labs
  • Test Object

You can also set Desired Capabilities such as platformName : iOS.

What are Desired Capabilities?

"Desired capabilities are a set of keys and values (i.e., a map or hash) sent to the Appium server to tell the server what kind of automation session we’re interested in starting up. There are also various capabilities which can modify the behavior of the server during automation. For example, we might set the platformName capability to iOS to tell Appium that we want an iOS session, rather than an Android or Windows one. Or we might set the safariAllowPopups capability to true in order to ensure that, during a Safari automation session, we’re allowed to use JavaScript to open up new windows". - Appium API Reference

If you have a huge set of Desired Capabilities you can edit in the GUI of Appium Desktop and see it in the JSON object that the Appium will interpret.

Appium server capabilities

automationNameWhich automation engine to useAppium (default) or Selendroid
platformNameWhich mobile OS platform to useiOSAndroid, or FirefoxOS
platformVersionMobile OS versione.g., 7.14.4
deviceNameThe kind of mobile device or emulator to useiPhone SimulatoriPad SimulatoriPhone Retina 4-inchAndroid EmulatorGalaxy S4, etc…. On iOS, this should be one of the valid devices returned by instruments with instruments -s devices. On Android this capability is currently ignored, though it remains required.
appThe absolute local path or remote http URL to an .ipa or .apk file, or a .zip containing one of these. Appium will attempt to install this app binary on the appropriate device first. Note that this capability is not required for Android if you specify appPackage and appActivity capabilities (see below). Incompatible with browserName./abs/path/to/my.apk or http://myapp.com/app.ipa
browserNameName of mobile web browser to automate. Should be an empty string if automating an app instead.'Safari’ for iOS and 'Chrome’, 'Chromium’, or 'Browser’ for Android
newCommandTimeoutHow long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the sessione.g. 60
language(Sim/Emu-only) Language to set for the simulator / emulatore.g. fr
locale(Sim/Emu-only) Locale to set for the simulator / emulatore.g. fr_CA
udidUnique device identifier of the connected physical devicee.g. 1ae203187fc012g
orientation(Sim/Emu-only) start in a certain orientationLANDSCAPE or PORTRAIT
autoWebviewMove directly into Webview context. Default falsetruefalse
Go to https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/caps.md to see more Android and iOS specific Desired Capabilities!

How to Use The Inspector:

Start the Session. You need to install Android SDKs of emulators on your local machine. After seeing the simulator, you can see the inspector window. What is going on with different elements?

You can navigate the XML source, the response tree. This is how you can find the id or CSS Selector that you then can call.

... It is like the Firefox plugin, Firebug and Firepath used to determine strategies to find locators.

You can perform actions directly from Appium Desktop, such as sending keys to a textbox.

Note, if you input your Sauce Labs credentials you can use their virtual emulators of Android or IOS devices.

See any issues? Are the issues with:

  • Appium? https://github.com/appium/appium/issues
  • Appium Desktop? https://github.com/appium/appium-desktop/issues

Want to See More Webinars from Sauce L?

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

April 2, 2017

Learning Serenity: The Screenplay Tutorial

We've covered a lot with Serenity BDD over the past few weeks!

This has all prepared us for John Ferguson Smart's tutorial on the Serenity BDD Screenplay Pattern. See http://serenity-bdd.info/docs/articles/screenplay-tutorial.html

Why Replace the Page Object Pattern with The Screenplay Pattern? 

Personally, I am trying to learn Serenity BDD and the Screenplay pattern because it is already the toolset and techniques we are already using where I am assigned as a contractor. I just started my career in automated testing two years ago.

Why does John Ferguson Smart and his group recommend this pattern they created? From Page Objects Refactored: SOLID steps to the Screenplay / Journey Pattern (2/11/2016):
"PageObjects have been the staple of automated web testing for over seven years. Simon Stewart wrote the original Selenium PageObject wiki entry in 2009. The concept was introduced to help test-developers reduce maintenance issues that, for many, resulted in flaky tests. Some mistook this as issues with Selenium, rather than with their own coding practices. While PageObjects can act as training wheels and be a good place to start, in this article we show you issues inherent to PageObjects and the benefits of refactoring them using SOLID OO principles. We then introduce the Screenplay Pattern, an alternative approach that could save you the trouble".
... I also heard that the Screenplay pattern has Appium built in as a way to kick off mobile automation. I'll be investigating that, next!

How to Get Started

Following the instructions on Writing automated acceptance tests using Serenity and the Screenplay Pattern:

"The easiest way to create a project skeleton for a Serenity Screenplay project is to use the Maven Archetype Plugin. To do this, run the mvn archetype:generate command (with a filter to reduce the number of artifacts Maven proposes) as shown here:"
$ mvn archetype:generate -Dfilter=screenplay
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: remote -> net.serenity-bdd:serenity-junit-screenplay-archetype (Serenity automated acceptance testing project using Screenplay, Selenium 2 and JUnit)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): :
"[...] You will then be prompted to enter a groupId, artifactId, and version for your project, and a root package for your classes [...]".

Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Define value for property 'groupId': : net.serenitybdd.tutorials
Define value for property 'artifactId': : todomvctests
Define value for property 'version':  1.0-SNAPSHOT: : 1.0.0-SNAPSHOT
Define value for property 'package':  net.serenitybdd.tutorials: :
Confirm properties configuration:
groupId: net.serenitybdd.tutorials
artifactId: todomvctests
version: 1.0.0-SNAPSHOT
package: net.serenitybdd.tutorials
 Y: : Y
"You will find your new project in the screenplay-tutorial directory".

$ cd screenplay-tutorial
$ mvn clean verify
... And the tests should run!

Explore the tutorial to learn more about all the moving parts as shown below:


And thank you for Learning Serenity with me! Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 31, 2017

Learning Serenity: Scaffolding a new project with Maven Archetypes

Imagine that you are starting a new job, where you encounter yet another new automated toolset you haven't heard of before. It means that you have:

This is why I blog: The only way to keep my head on straight is to take copious amounts of research notes. This blog is where I store them.

Am I struggling with a concept that I just can't get? I've befriended a few experts in my very short time as an automation developer (Hi, Alan, Dave, Joe, Jim, James M, Joon, Lark, Martin, James V, Bas, and Angie!) who really enjoy helping out beginners like me out. If I blog about a problem I am having, and point to the post on Twitter, I get excellent feedback from the automation community. Thank you all so much!

March 29, 2017

March 29, 2017: Talks on Performance Testing and REST Assured at the Ministry of Testing - Boston Meetup

The Ministry of Testing - Boston Meetup held it's March event on Monday, March 29th, once again at iZotope in the Kendall Square, MIT, Cambridge, MA area. Thank you so much Tomer for letting us meet there!

Before the Meetup

We had two special guests!

James Venetsanakos, Performance and Automation Testing Architect, gave a ten minute teaser for his upcoming talk on Performance Testing for the Ministry of Testing - Boston Meetup that will be held on on Wednesday, April 12, 2017. 
James giving his 10 Minute Lightning Talk on Performance Testing

You can watch the introduction to the 10 Minute Lightning Talk he gave below!


Bas Dijkstra, a Consultant in Test automation and  Service virtualization submitted a 99 Second Talk to the Ministry of Testing, Boston. Thank you so much, Bas!

After the talks, we held a roundtable discussion, talking about everything from how to organize your page objects, pluses and minuses on Open Source vs Paid load testing tools, the Agile Testing Quadrant and the Testing Pyramid, and best practices for automation.

I cannot wait for James' talk in April!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 27, 2017

Learning Serenity: Studying BDD using The Cucumber Book and BDD in Action

It’s been a productive two weeks here at Ahold in Quincy, MA. They are the the holding company for Stop & Shop, Giant Foods, and Martin’s, where I am testing mobile applications for the iPhone, iPad and Android.

Although I’m mostly focused on manual testing as I am attempting to get to know the business, I’ve been trying to get myself up to speed with the automation framework they are using, Serenity BDD. I have a long term goal of combining their current Serenity BDD test suite with Appium, so when we start developing a regression test suite, I can possibly automate it.

As I Tweeted a few days ago… Oh, this is going to take a while!

Hrm… thank you, Serenity BDD! I need to look into that.

Oh, and thank you very much, John Smart, for your words of encouragement on this blog series:

For now, since I am studying how to implement Serenity using a testing framework invented by Aslak Hellesoy called "Cucumber".

BDD and The Cucumber Book

For those who were wondering, the “BDD” in Serenity BDD stands for “Behavior Driven Development”. I first encountered BDD when I was supervising an offshore testing team attempting to translate my manual mobile test scripts into Cucumber / Ruby automation using Calabash-IOS. That is when I first encountered Matt Wynne’s “The Cucumber Book: Behaviour-Driven Development for Testers and Developers” (2012), which was quite helpful describing BDD.
Matt Wynne and Aslak Hellesøy [the creator of Cucumber] show you how to express your customers’ wild ideas as a set of clear, executable specifications that everyone on the team can read. You’ll learn how to feed those examples into Cucumber and let it guide your development. You’ll build just the right code to keep your customers happy, and not a line more. Although it was born in the Ruby community, you can use Cucumber to test almost any system, from a simple shell script or Perl script, to web applications written in PHP, Java, or any platform”. - The Cucumber Book, The Pragmatic Bookshelf 

About the Authors

Note: The Cucumber Book now a second edition, released on February 27, 2017.

Matt Wynne is a long-standing member of the Cucumber core team, fascinated by the challenge of helping tech and business to understand one another. He's one of the co-founders of Cucumber Ltd., the company behind Cucumber. He lives on the west coast of Scotland on an old farm with his family, two cats, their dog, and some ducks. Matt tweets from @mattwynne and @cucumberbdd.

Aslak Hellesoy is the creator of Cucumber. During his career Aslak has worked with both small and large organizations in industries such as telecom, trading, insurance, car manufacturing, education, and government. Aslak is a co-founder of Cucumber Ltd, the company behind Cucumber. He tweets from @aslak_hellesoy and @cucumberbdd.

"Steve Tooke is a programmer, trainer, and coach who is dedicated to improving his craft and helping others improve theirs. He's a co-founder of Cucumber Ltd. Steve tweets from @tooky and @cucumberbdd”.

Oh, and Why Is It Called "Cucumber"?

Why the name "Cucumber"? Read Aslak Hellesoy's answer on Quora, how he asked his fiancee, Patty for naming suggestions:

"Patty, I need a name for this new tool I just started hacking on. I want it to have a catchy, non-geeky sounding name.

"She paused for a few seconds, then said: Cucumber!

"And I thought: Cucumber? Really? Well, it's a lot better than Stories - so I'll go with that for now. I'll rename it again when I come up with something better".

What is an Acceptance Test for a Feature?

Here’s an example Matt Wynne lists in The Cucumber Book:

Feature: Sign up
     Sign up should be quick and friendly.
Scenario: Successful sign up
New users should get a confirmation email and be greeted personally by the site once signed in.
Given I have chosen to sign up
When I sign up with valid details
Then I should receive a confirmation email
And I should see a personalized greeting message

… This is an example of what is called an acceptance test. “Instead of a business stakeholder passing requirements to the development team without much opportunity for feedback, the developer and stakeholder collaborate to write automated tests that express the outcome that the stakeholder wants. We call them acceptance tests because they express what the software needs to do in order for the stakeholder to find it acceptable. The test fails at the time of writing, because no code has been written yet, but it captures what the stakeholder cares about and gives everyone a clear signal as to what it will take to be done”.

The Product Owner, a business analyst representing the customer’s wants and needs on an Agile Software Development project can express a client’s wishes and expectations in an easy-to-read format. They can be expressed in terms of behavior wanted in the finished product.

As Matt Wynne mentions, “Acceptance tests written in this style become more than just tests; they are executable specifications […]
“For many teams, they become the definitive source of truth as to what the system does. Having a single place to go for this information saves a lot of time that is often wasted trying to keep requirements documents, tests, and code all in sync. It also helps to build trust within the team, because different parts of the team no longer have their own personal versions of the truth”. - The Cucumber Book

How Do Features Get Turned Into Steps and Step Definitions?  

But how does it work?
“When you run [Cucumber], it reads in your specifications from plain-language text files called features, examines them for scenarios to test, and runs the scenarios against your system. Each scenario is a list of steps for Cucumber to work through. So that Cucumber can understand these feature files, they must follow some basic syntax rules. The name for this set of rules is Gherkin. Along with the features, you give Cucumber a set of step definitions, which map the business-readable language of each step into Ruby code to carry out whatever action is being described by the step. In a mature test suite, the step definition itself will probably just be one or two lines of Ruby that delegate to a library of support code, specific to the domain of your application, that knows how to carry out common tasks on the system. Normally that will involve using an automation library, like the browser automation library Capybara, to interact with the system itself”. - The Cucumber Book

Does Serenity BDD Still Use Features Files and Step Definitions? 

If you are using Serenity BDD with Cucumber, it will follow the same exact steps.

  • Narratives describe the story you want to tell.
  • Feature files describe the features of the product, breaking it down into scenarios outlining how the feature will be used.
  • Step definitions walk through step-by-step how to execute the code.

“Step definitions sit right on the boundary between the business’s domain and the programmer’s domain. […] Their responsibility is to translate each plain-language step in your Gherkin scenarios into concrete actions in […] code”.

What is the difference between Steps and Step Definitions?

“Each Gherkin scenario is made up of a series of steps, written in plain language. On its own, a step is just documentation; it needs a step definition to bring it to life. A step definition is a piece of Ruby code that says to Cucumber, ‘If you see a step that looks like this…, then here’s what I want you to do….’ When Cucumber tries to execute each step, it looks for a matching step definition to execute. So, how does Cucumber match a step definition to a step"
“Gherkin steps are expressed in plain text. Cucumber scans the text of each step for patterns that it recognizes, which you define using a regular expression. If you haven’t used regular expressions before, then just think of them like a slightly more sophisticated version of the wildcards you’d use to search for a file”. - The Cucumber Book

What is the Difference Between BDD and ATDD?

To answer this questions, let's jump to a sidebar in “BDD in Action” written by John Ferguson Smart, the creator of Serenity BDD.

BDD by any other name

“Many of the ideas around BDD are not new and have been practiced for many years under a number of different names. Some of the more common terms used for these practices include Acceptance-Test-Driven Development, Acceptance Test-Driven Planning, and Specification by Example. To avoid confusion, let’s clarify a few of these terms in relation to BDD.

Specification by Example describes the set of practices that have emerged around using examples and conversation to discover and describe requirements. In his seminal book of the same name,14 Gojko Adzic chose this term as the most representative name to refer to these practices. Using conversation and examples to specify how you expect a system to behave is a core part of BDD, and we’ll discuss it at length in the first half of this book.

Acceptance-Test-Driven Development (ATDD) is now a widely used synonym for Specification by Example, but the practice has existed in various forms since at least the late 1990s. Kent Beck and Martin Fowler mentioned the concept in 2000,15 though they observed that it was difficult to implement acceptance criteria in the form of conventional unit tests at the start of a project. But unit tests aren’t the only way to write automated acceptance tests, and since at least the early 2000s, innovative teams have asked users to contribute to executable acceptance tests and have reaped the benefits

Acceptance-Test-Driven Planning is the idea that defining acceptance criteria for a feature leads to better estimates than doing a task breakdown”.

What is the Goal of BDD?

“One of the key goals of BDD is to ensure that everyone has a clear understanding of what a project is trying to deliver, and of the underlying business objectives of the project. This, in itself, goes a long way toward ensuring that the application actually meets these objectives.
“You can achieve this by working with users and other stakeholders to define or clarify a set of high-level business goals for the application. These goals should provide a concise vision of what you need to build. Business goals are about delivering value, so it’s common to see them expressed in terms of increasing or protecting revenue, or of decreasing costs”. - BDD in Action

Now that the research has been done, we can start assembling a new Serenity BDD project..

Until then, Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

March 26, 2017

New automation framework released to test APIs: Introducing Karate, by Intuit India's Peter Thomas

After spending the past month attempting to finish blogging about the API testing framework, REST Assured, I came across today the following item that had been retweeted:

According to Peter Thomas' LinkedIn profile, he has been an architect in Intuit, working for Intuit India ( @Intuit.In ) since 2012 and lives in the Bengaluru Area, India.

Peter even has Google Doc he posted about on March 20, 2017 comparing and contrasting Karate vs REST Assured.

*sigh* Sometimes it is so hard to keep up.

Here is some information from around the web about this new automation framework. During my next lull in work, I need to tinker with this!

Karate: Web-Services Testing Made Simple

From Peter Thomas' Medium article, Karate: Web-Services Testing Made Simple, posted on BLUEPrint by Intuit's Medium.com account:

"Intuit has open-sourced ‘Karate’, a framework that makes the tall claim that the business of testing web-APIs can actually be — fun.

"I know what you must be thinking. There’s no way that making HTTP requests and navigating the forest of data that is returned could be fun.

"But really, that’s what developers who tried Karate had to say. It actually didn’t surprise us. Because Karate was born out of a strong dis-satisfaction with the current state of solutions that exist. And a lot of thought went into Karate to keep it simple and elegant, to allow the user to focus on the functionality instead of boiler-plate, and to keep things concise.

"Karate strives to reduce the entry barrier to writing a test and more importantly — reduces the friction to maintain a test, because of how readable tests become [ Read More ]".

Karate's GitHub Site

From https://github.com/intuit/karate:

"Karate enables you to script a sequence of calls to any kind of web-service and assert that the responses are as expected. It makes it really easy to build complex request payloads, traverse data within the responses, and chain data from responses into the next request. Karate's payload validation engine can perform a 'smart compare' of two JSON or XML documents without being affected by white-space or the order in which data-elements actually appear, and you can opt to ignore fields that you choose.

"Since Karate is built on top of Cucumber-JVM, you can run tests and generate reports like any standard Java project. But instead of Java - you write tests in a language designed to make dealing with HTTP, JSON or XML - simple".

Peter's Hello World Example

Feature: karate 'hello world' example
Scenario: create and retrieve a cat

Given url 'http://myhost.com/v1/cats'
And request { name: 'Billie' }
When method post
Then status 201
And match response == { id: '#notnull', name: 'Billie' }

Given path response.id
When method get
Then status 200

On the site, Peter talks about how to get started, variables and expressions, data types, primary HTTP keywords, tips, and examples such as data driven tests, and how to use it with *.feature files.

Over ten demos are on the Karate Demo Page: https://github.com/intuit/karate/tree/master/karate-demo

Watch Joe Colantonio Talk About "Karate a Rest Test Tool – Basic API Testing"

Joe Colantonio, creator of the Test Talks podcast, just published on March 23rd an article, Karate a Rest Test Tool – Basic API Testing, which includes a mini tutorial:


You can get more instruction from https://www.joecolantonio.com/2017/03/23/rest-test-tool-karate-api-testing/

Follow Karate DSL on Twitter!

And, yes, you can follow Karate on Twitter:

Watch a Webinar from the Creator, Peter Thomas

TechGig published a webinar, "Karate: DSL for writing web service API acceptance tests, BDD":

"In this session, Peter will walk through the features of Karate, demonstrate how it handles manners of HTTP aspects such as file-uploads and cookies, and also explain how it can be extended via custom-functions in either JavaScript or Java. Karate's innovative plug-in mechanism for HTTP-header management makes integrating any kind of authentication and sign-in flow extremely simple. You get to hear straight [from] the creator of ‘Karate’ the motivations for creating this framework in the first place, how it differs the competition, and how it helps accelerate the development and quality-assurance of web-services of kinds, be it REST, SOAP or GraphQL.

"Key points of discussion:
  • "Why Karate was created, and what problems it solves well.
  • "Examples and demos of real-life web-service API tests using Karate.
  • "The architecture of Karate and how it combines technologies such as Cucumber, JsonPath and Nashorn.
  • "Insights into what goes into open-sourcing and releasing a public Java project".


Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"