10 Automated Testing Tools That Threat Stack Uses — and Why

Previously published on the Threat Stack Blog, September 19, 2019, co-author Laura Haiduck. 

All software development projects, whether they’re large or small, can benefit from well-planned and well-executed testing. It’s your way to ensure that the software you’re developing performs as expected and delivers value to the customer. More important — given the nature of our current cyber landscape — well-executed testing is your way to ensure that your software doesn’t ship with errors or vulnerabilities that could compromise its integrity. In a word, good testing lets you pass on performance value to customers — while also providing them with underlying security.

At Threat Stack, we have a complex, full stack development environment that requires a great deal of planning and decision making within the Test Engineering Team to make sure that all the functional areas of our platform are being tested effectively and efficiently. This includes decisions about what testing tools to use, since the right ones allow us to introduce automation as well as speed, repeatability, consistency, and accuracy of results.

Are you sure the buses are still listed? Interacting with APIs with Ruby + NET::HTTP + Gauge

This is Part 2 of 2 of a blog series. Care to go back to the beginning? 

There are many different Ruby libraries that allow you to interact with an API:

• Net/Http: The HTTP client built into Ruby standard library. 
• Httparty: Built on top of Net/Http by John Nunemaker, you can GET the HTTP Response HTTP Code, Response Message, and the HTTP Headers with one call. The Google Group was last active in 2017. 
• Faraday: Also allows you to get the status, headers, and body, allowing you a bit more to customize the HTTP request. 
• Rest-Client: a "simple HTTP and REST client for Ruby, inspired by the Sinatra’s microframework style of specifying actions: get, put, post, delete".

For this sample project, where we are simply getting data from the MBTA API, we will use:

• Net/HTTP to get data from the API
• The JSON library to parse the data
• The test/unit library to assert that the expected values and the actual values match up
• ThoughtWorks Gauge as the test framework. 

Are you sure the buses are still listed? Setting up data-driven API tests with Ruby + NET::HTTP + Gauge

The Massachusetts Bay Transit Authority (MBTA) operates a series of bus lines, whose information is accessible through their API, https://api-v3.mbta.com/. How could you test that a sampling of these bus lines are still listed?

• Route 210  | Quincy Center - Fields Corner           
• Route 212  | Quincy Center - North Quincy           
• Route 220  | Hingham Depot - Quincy Center         
• Route 222  | East Weymouth - Quincy Center         
• Route 230  | Montello Commuter Rail Station - Quincy   
• Route 236  | South Shore Plaza - Quincy Center

Sure, you could go to the MBTA website and search for the bus route at https://www.mbta.com/schedules/bus but, as we saw in our project Are You Sure The Bus Line Is Still Listed?, what is best for humans is not best when creating an automated test. 

With this project, we are going to use the built-in Ruby Library NET::HTTP to interact with the API, and Thoughtworks Gauge to set up the test framework, and we are going to make the tests data-driven, putting the information we need to verify in a table.

Related Documentation: 

• Documentation for the MBTA API  V3 is at https://www.mbta.com/developers/v3-api
• MBTA V3 API Developer Portal: https://api-v3.mbta.com

Come see Angie Jones, fresh from her world tour at the Ministry of Testing - Boston Meetup on September 23, 2019

Angie Jones, International Keynote Speaker, blogger at angiejones.tech, Director of TestAutomationU.com is on a world tour for her latest talk Visual Validation for Test Automation:

• August 12-14: CAST, Cocoa Beach, Florida
• September 4-6: Full Stack Fest, Barcelona
• September 23: STPCon: Angie is giving the Keynote: A Tale of Testing the Untestable
• September 23: Ministry of Testing - Boston Meetup
• September 30: StarWest, Anaheim, California
• October 14-16: PNSQC, Portland, Oregon
• October 24: TestBash Australia, a Ministry of Testing conference, Sydney, Australia
• November 20-22: ConTEST, NYC

Live or work in Boston? Come meet Angie Jones in person on Monday, September 23rd, 2019!

Angie Jones

Sign up, here!

• https://www.meetup.com/ministry-of-testing-boston/events/263810657/

Thank you, http://Applitools.com for having Angie speak to us!
Thank you, http://SmartBear.com for hosting!


Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

My free course "Introduction to Capybara" is now online!

I remember being so impressed when I first heard about Test Automation University, the latest project by Angie Jones. A free online university, sponsored by Applitools, where subject matter experts in the software testing field created tutorials? Yes, please! Finding up-to-date information on automated testing can be an exercise in frustration.

Imagine my surprise when a few weeks after TAU was launched, Angie Jones got in touch with me back in February: Did I want to create a course for Test Automation University?

I decided my topic would be on Capybara, the Selenium WebDriver wrapper I used to put together the UI Automation framework at work last year. It took a lot of time, effort, and sleepless nights, but after a month of effort, I was able to beat the deadline.

I hope you like it!

Introduction to Capybara
 https://youtu.be/hZjsvTYJIL8

• Register for the course: https://testautomationu.applitools.com/capybara-ruby/

Thank you so much, Angie, for giving me this opportunity! 

I cannot believe all the effort Angie Jones puts into reaching out to the software testing community. She is always on the go, giving lectures, running webinars, and speaking at conferences. 

Angie has been a guest speaker at my Ministry of Testing - Boston Meetup twice:

•  Q & A on "The Build That Cried Broken" lecture, 2017 
•  Stop Compromising Quality: Setting Automation Up for Success, 2018

... I can't wait to see what Angie has in store for the Meetup this fall!



Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

New to coding? Learn Java through Angie Jones' TestAutomationU

Are you a software tester who is new to coding? Don't know where to start?

Angie Jones has released a new Java Programming course on her Test Automation University.

https://youtu.be/Fo2U0o64oVI

• Java Programming: https://testautomationu.applitools.com/java-programming-course/

The Java course uses the OpenJDK General Availability release of Java 11.

The OpenJDK, the Open Java Development Kit is a free and open-source implementation of the Java Platform, Standard Edition (Java SE). (Get the OpenJDK FAQ).

Since I am using Windows, how I installed it:

• Download from JDK.Java.Net/Archive/ Java version 11.0.2 LTS version.
• Extract the Zip File to C:\Program Files\Java, creating a jdk-11 folder. 
• Set the PATH and JAVA_HOME according to this StackOverflow answer. 
• Downloaded and Installed IntelliJ IDEA Community Edition, setting it up with Java 11.0.2. 

I am loving the course! Angie sprinkled her fun and her energy throughout the course.

Here are a few tips taking the course:

• Expand the course to cover the left side of your monitor, with your IDE covering the right hand side of your monitor. 
• When Angie goes over code, type what she types, and execute when she executes.
• Do the outside exercises Angie recommends. 

Passively watching Angie type code will not allow you to learn. Only through you practicing the material will allow things to sink in. 

Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

Five Tips How Software Testers Can Collaborate With Software Developers

Whenever I join a new team, my first task is fostering and nurturing a good working relationship with the developers. Why? If there is good chemistry between testers and developers, the quality of work increases as the quality of communication increases.

The relationship between tester and developer shouldn’t be one of artist and art critic. Rather, it should be one between writer and copy-editor, each contributing to the quality of the product.

Developing a good working relationship with developers can be tricky. Here are five tips for nurturing and developing relationships with your developer teammates.

Notes: Selenium 4 With Simon Stewart

Simon Stewart, the Selenium product lead, the creator of WebDriver gave a brief preview on what is coming up with Selenium 4.

Selenium, Simon says, is a family: Selenium WebDriver, Selenium Grid, and now Selenium IDE.


https://youtu.be/6iHdvOYdJk8

Notes: The Ministry of Testing's Introduction to HTTP by Mark Winteringham

Even though I have been a software tester for the past twenty years, I still like taking introductory courses where I can practice going back to the basics.

Right now, I am working through Mark Winteringham's Introduction to HTTP, accessible when you purchase the Ministry of Testing's Professional Membership.

Notes: Amber Race, Exploring Service APIs Through Test Automation using Postman

Want to learn about testing REST APIs? Make sure to check out Amber Race's Exploring Service APIs Through Test Automation, part of Angie Jones' free Test Automation University. Amber is a Software Development Engineer in Test at Big Fish Games and blogs about her work at AmberTests.com.

My Threat Stack interview: What are the current trends in automation?

Threat Stack, where I work, has started a campaign to showcase on its company blog not just its developers and security analysts, but also its software testers.

I was interviewed for Threat Stack's article, Current Trends in Automated Software Testing: 10 Testing Pros Share Insights Into the Most Interesting Current Trends in Automated Software Testing.
My TLDR version:

When @ThreatStack inquired about the biggest trend in automation, I responded, "Free Education for all!"
* @TechGirl1908 with https://t.co/A97yVpOwpW
* @LisaCrispin with DevTestOps https://t.co/XFIoasi9rJ
* @FriendlyTester with https://t.co/2afqq8c8pz @ministryoftest

— T.J. Maher (@tjmaher1) May 30, 2019

... Make sure to read what my co-worker, Carl Robinson has been doing with Docker!

Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

New to Automation? Take a Foundation course at Test Automation University

New to Automated testing? Need to figure out what tests to automate? Who should do the automation work? How to get buy-in from the development team?

These and many other questions are answered in Angie Jones' Test Automation University course, Setting a Foundation for Test Automation.

 

The video course contains over 45 minutes of material covering topics such as:

• What is your goal for starting a test automation initiative and what is it that you want to accomplish?
• Who do you envision participating in your test automation initiative and in what capacity?
• How do you plan for the execution of this strategy?
• How to get people on board and help them understand their place in the automation strategy
• How to scale your automation

Registration is free for this course, the current fifteen courses available, and the five courses listed as "Coming Soon".  Also coming soon, ... a course I designed for Applitools Test Automation University: Introduction to Capybara.



Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

Long time no see!

Long time no see! How have you been?

I've been good. My nine-month old son is utterly adorable. My wife and I just love getting on the floor and playing with him. He's now crawling around so fast, we can barely keep up with him! He's also standing, pulling himself up on anything he can get a hold off. Kitchen cabinets. Walls. Shelving units. His Dad's hair. His Dad's ear. And somehow he is sprouting a third and fourth tooth!

Work has been amazing! I cannot believe how supportive everyone has been for this new father! Since we last spoke, I am on a new development team, focused more on the back-end, testing Threat Stack's many microservices that run the SecOps product.

Last year, I was creating a UI Automation framework from scratch, using Capybara + Ruby as a wrapper for Selenium WebDriver, and ThoughtWorks Gauge as a test framework.

To deepen my knowledge of Capybara and Gauge, I created a few demo projects:

• Gauge-Selenium-Ruby-first-draft (February 2018)
• Basic Capybara - Gauge (December 2018) 

Last September, I even volunteered writing a chapter of a software testing book "How to Gauge What Your Tests Are Doing" for Eran Kinsbruner's Continuous Testing for DevOps Professionals.

This year? I wrote an Introduction to Capybara course! It took a few months, and a lot of sleepless nights, but it was worth it. It is still being finished up. Watch this space. More details to come! 

Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

Check out what the Ministry of Testing, Applitools + Test Automation University, and Mabl + Lisa Crispin have brewing!

There is so much going on in the software testing community with the Ministry of Testing, Applitools' Test Automation University, and Mabl and Lisa Crispin's webinar series, Modeling Your Test Automation Strategy!

Four Years of Adventures in Automation!

When I started this blog, it was before I ever found my first automation development position. This blog was a way for me to keep up the good fight, journaling my experience.

Four years and 321 blog posts later, I have been on teams building testing frameworks in Selenium WebDriver + Java, NightwatchJS + NodeJS,  Capybara + Ruby, and now ScalaTest and Scala. The Programming Projects section of this site captures my experiments with many others.

This blog acted as a springboard for me to start publishing articles, contributing to software testing books, and giving talks at online conferences. You can read more about it in the Media section of this blog.

I don't get as much time to write on this blog as I used to. The main reason?

T.J. Maher and his son, Tommy.

... After work, I seem to be otherwise occupied in baby giggles, baby cuddles, and baby snuggles.

I wanted to thank you, dear reader, for all your help. This blog has been a magnificent conversation started with many a hero in the software testing field. Thank you so much, Alan Richardson, Bas Dijkstra, Jim Hazan, Matt Hutchinson - Testing Curator, Joe Colantonio, Lisa Crispin, and most importantly Angie Jones.

So far, in four years this blog has received 730,089 pageviews, with 70% of the visitors going "Software Testing? Automation? The hell?? This isn't what I Googled!" and promptly going back to wherever they came from.

If you want to drop a line to say hello, I am on Twitter, and I am always online!

Happy Testing!

-T.J. Maher
Sr. QA Engineer, Software Engineer in Test
Meetup Organizer, Ministry of Testing - Boston

Twitter | YouTube | LinkedIn | Articles

Basic Capybara-Gauge: Review the Ruby code with Rubocop

This is the seventh part of a series of blog posts. Care to go back to the beginning? 

Tinkering with Ruby only for the past year, Rubocop, a "Ruby static code analyzer and formatter, based on the community Ruby style guide" according to their Github site is both a blessing and a curse for someone new to the language. It is blessing that it is so detailed with output. The curse is that if you are new, it is going to take a lot of random Googling to figure out what all the errors thrown actual mean!

There is a lot of content in the Ruby Style Guide...

What is the Ruby Style Guide?

From the Ruby Style Guide README: "This Ruby style guide recommends best practices so that real-world Ruby programmers can write code that can be maintained by other real-world Ruby programmers. A style guide that reflects real-world usage gets used, while a style guide that holds to an ideal that has been rejected by the people it is supposed to help risks not getting used at all—no matter how good it is. [...]

"There are some areas in which there is no clear consensus in the Ruby community regarding a particular style (like string literal quoting, spacing inside hash literals, dot position in multi-line method chaining, etc.). In such scenarios all popular styles are acknowledged and it's up to you to pick one and apply it consistently".

Basic Capybara-Gauge: Add Chrome and ChromeDriver logging capabilities

This is the sixth part of a series of blog posts. Care to go back to the beginning? 

Want to view JavaScript errors in Chrome? Or see if your Selenium WebDriver instance is throwing any errors? Check out the Logging Preference Capabilities from the Chromium project.

With this article, we will demonstrate how to get these logs set up in your Capybara-Ruby project, and what to add in the spec_helper.rb file we created.

Basic Capybara-Gauge: Setting Up Specs and Step Implementations

This is the fifth part of a series of blog posts. Care to go back to the beginning? 

So far, we have written test specifications to test Dave Haeffner's The-Internet, and we have set up the Gauge-Ruby environment. We managed to get Chrome to run a simple Capybara test, and figured out how to make the test run in Headless Chrome. Our test we created last time...

specs/Navigation/Navigation.spec
# Navigation: Go to The-Internet Login Page

## NAVIGATION: Visit The-Internet
* LOGIN: Visit the Login Page
* LOGIN: Verify the heading is "Login Page"

... For this entry, we will be building upon that, figuring out how to get our Capybara-Gauge framework to execute the following:

Authorization: Validate logging into and out of The-Internet

• Scenario #1: Authorization: Successfully log into and out of the secure area 

• Scenario #2: Authorization: Verify alert error is shown when entering a blank user name and password.

Basic Capybara-Gauge: Set Up Headless Chrome

This is the fourth part of a series of blog posts. Care to go back to the beginning? 

We've come up with test specs for Dave Haeffner's The-Internet. We've set up a Ruby environment to run the tests, and configured Chrome to open the Login page.

With this entry, we will be activating Chrome in Headless Mode, i.e. running Chrome but without a visual interface, allowing the tests to run faster.

About Headless Chrome

According to Eric Bidelman's article, Getting Started with Headless Chrome, the new option of running Chrome in Headless mode started with Google Chrome version 59, released in May 2017.

Before, you had to use something such as PhantomJS to emulate the Chrome browser if you wanted to run Chrome in a place where the processing power to bring up a visual window would be a waste, such as running it with an automated test script running in a continuous integration environment. Now, it is built right in.

Basic Capybara-Gauge: Setting Up Capybara to Open a Chrome Browser

This is the third part of a series of blog posts. Care to go back to the beginning? 

So far, we have written test specifications to test Dave Haeffner's The-Internet, and we have set up the Gauge-Ruby environment.

Now, we need to set Capybara up to be integrated with the Gauge test framework. 

Basic Capybara-Gauge: Setup Ruby Environments for The-Internet

This is the second part of a series of blog posts. Care to go back to the beginning? 

With the last entry, we analyzed the App Under Test, Dave Haeffner's The-Internet and came up with some test specifications. With this entry, we will be setting up the Ruby environment that will be running the Capybara-Gauge tests.

Although I am developing this project on my Macbook with this example, it is possible to install Ruby, Gauge on a Windows Machine.  See my article, Intro to setting up a BDD framework with VS Code + Gauge + Ruby.