January 26, 2017

Test Automation and Specification by Example with Greg Tutunjian: Notes on Ministry of Testing - Boston Meetup

That. Was. Wonderful! The Ministry of Testing - Boston (formerly the Greater Boston QA and Testing Group) kicked off the first Meetup of 2017. I acted as co-host for the first time, meeting-and-greeting people for the first time since I was made co-organizer of the group!

I haven't run a Meetup since I was last hyperactive with the Nerd Fun - Boston Meetup over five years ago. Back then, I lead a herd of nerds to crash other people's events: lectures historical, technological, and scientific all over Boston and Cambridge. Now, I was the person setting up for the event! Luckily, Conrad, the owner of the Meetup group, found a speaker and the location; all I needed to do was set up and meet-and-greet.

That reminds me: I need to find speakers and places to host upcoming Meetups for Ministry of Testing - Boston or it is going to be merely Dinner and QA Discussion events (In the style of "Lean Coffees") on the official event calendar as far as the eye can see. Are you a QA Manager in the Cambridge / Boston area who loves talking about your own personal testing philosophy? Come up with a topic you want to share, and talk to Conrad about hosting one of our Meetups. Who knows? Maybe in the audience of your talk will be your next new hire!

More About the Meetup Topic

From the Meetup Event

Subject: Test Automation In The Hands Of “The Business”: How Specification By Example Is Changing How We Plan and Test Our Solutions

Date: Wednesday, January 25, 2016 @ 6:00 pm

Location:  Izotope, Cambridge, MA

Description: "We’re seeing an increase in the adoption of Specification By Example (SBE) in the forms of Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD): Tools and techniques designed to complement and accelerate these adoptions especially at the enterprise solution level (where even a small reduction in solution delivery time can have a very significant return on investment.) Quality Assurance (QA) professionals are uniquely prepared to lead the adoption (and prove the effectiveness) of these solution verification and validation methods due to QA’s need and desire to understand detailed and general solution behavior, architecture, usability and more. This talk will include a review of these methods including their origin, several use cases, and tools, techniques and resources for personal investigation".

About the Talk

Greg Tutunjian has been an Application Developer, a Systems Engineer, a Team Builder and Agile Delivery Lead, and currently is an Enterprise Agile Coach. As he puts it, "Experience Informs". He took us through snapshots of his entire career, from testing, validating, and leading his teams.

Greg Tutunjian ( LinkedIn ), ACP, CDA, CSM, CSP, SAFe SPC

As Greg put it in the summary of his talk, "Quality Assurance (QA) professionals are uniquely prepared to lead the adoption (and prove the effectiveness) of these solution verification and validation methods".

Why QA Professionals? QA are already better equipped. They already do:
  • Outside-In Testing. White box, black box testing. 
  • Solution centric vs development centric 
  • Incrememnting Value 
  • More Multidisciplicinary 
  • Direction Setting, examining metrics. 
  • Forming a Community of Practice: Centers of Excellence: Defines the standards and expectations

Sidenote: What is a Community of Practice? 

According to Wenger-Trayner, a "community of practice is a group of people who share a concern or a passion for something they do, and learn how to do it better as they interact regularly.

"This definition reflects the fundamentally social nature of human learning. It is very broad. It applies to a street gang, whose members learn how to survive in a hostile world, as well as a group of engineers who learn how to design better devices or a group of civil servants who seek to improve service to citizens".

Where Does Greg Come Into Play As An Agile Coach?

To answer the question when Greg comes into play as an Agile Coach, he mentioned the Satir Change Model. "Virginia Satir, a pioneer of family therapy, developed a model of how individuals experience change. The Satir Change Model says that as we cope with unexpected or significant change, we predictably move through four stages: Late Status Quo, Chaos, Practice and Integration, and New Status Quo". 

From https://stevenmsmith.com/ar-satir-change-model/
Greg comes aboard just after the Chaos, working with the team to come up with Practice and Integration.

How Business Validation Has Been Performed

Greg walked us through the many different models of business validation he has come across since the 1970s:
  • Back in 1975: It started with technical specifications, or specs.
  • 1985: Business requirement documents: BRD. ( See "BRD, a High Level Review")
  • 1990s: Rapid prototyping
  • 2000: Model behavior and predictive knowledge
  • 2005: Agile, and the small team movement
What Greg likes to use is a Specification by Example: (SBE): A collaboration method for specifying requirements and tests, Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD). 

Sidenote: What is Specification by Example?

From Martin Fowler's blog SpecificationByExample dated March 2004, 
"I was attending a workshop at XP/Agile Universe in 2002 when the phrase 'Specification By Example' struck me as a way to describe one of roles of testing in XP [...]
"Specification By Example isn't the way most of us have been brought up to think of specifications. Specifications are supposed to be general, to cover all cases. Examples only highlight a few points, you have to infer the generalizations yourself. This does mean that Specification By Example can't be the only requirements technique you use, but it doesn't mean that it can't take a leading role.
"[...] One of the great things about specification by example is that examples are usually much easier to come up with, particularly for the non-nerds who we write the software for".

Sidenote: What is Acceptance Test Driven Development?

From Elisabeth Hendrickson's blog, Test Obsessed dated December 2008, 
"Acceptance Test Driven Development (ATDD) is a practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. It’s the best way I know to ensure that we all have the same shared understanding of what it is we’re actually building. It’s also the best way I know to ensure we have a shared definition of Done.
"Obviously I think this is an important Agile development practice. In fact, it’s one of the core pieces of my Agile Testing class. Yet somehow I have neglected to write about it much on this blog. Time to rectify that.
"So, for your reading pleasure, I now present a PDF document with a detailed example of the whole ATDD flow".

Where Should You Start?

Greg advises, don't start out with a small unimportant area to start automating … start with a high-risk solution area, a bigger piece of work when you are pitching Agile. It is better for a manager to sell it. Work in Pairs. Facilitate Workshops. 

Greg could not stress the importance of putting together a Community of Practice: 
"A lot of great success happens in silence”. 
Put your success on a blog. Write about it. Talk about it. Share it. 

How does an Agile Project get started?

Greg says the first step is how he tries to find a good business partner and project. Then:
  • He adopts basic business scenarios.
  • He writes the step definitions.
  • Establishes a technology - facing flow.
  • Coaches, mentors, and Trains the Business.
  • Scale to Maximize the ROI 

Thank you very much, Greg for speaking at the Ministry of Testing - Meetup!

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

January 19, 2017

Notes on Sauce Labs presentation, "Future Proof Your Automation" by QASource

From the Sauce Labs Webinar section:

"If you’re finding it difficult to automate tests for new features, and/or a significant number of the bugs your team finds are false positives, you should consider future-proofing your automation. Join us for our upcoming webinar, where we'll feature presenters from QASource as they present 'Managers, Future Proof Your Automation.'

"The webinar will focus on a 'future proof' automation suite that:
• "Has an adaptive framework
• "Efficiently automates new features
• "Can be easily maintained
• "Reduces false positives
• "Is long-lasting

"In this webinar, Rajeev Rai, CEO of QASource, and Rick Rampton, Head of Client Success at QASource, will share techniques, strategies and processes that will make your automation future proof".

Presentation given by QASource, an Outsourcing Service for QA

Seven Steps to Future Proof Automation

If you wish to future proof your automation, there are seven strategies you must implement:

  • Automation Scope Review
  • Evaluate Best Suited Tools & Techniques
  • Automation Design Strategy
  • Script Creation Strategy
  • Execution Strategy, Reporting Strategy
  • Maintenance Strategy

Automation Scope Review

Automation should not be developed to use just once and then sit on a shelf. It should be used again and again.


  • What can and cannot be automated?
  • What should or should not be automated? You cannot automated everything.
  • Would it be quicker and more valuable to manually test something?
  • Would this test be run regularly? Good candidate to automate. Run four times a year? Not a good candidate.

Evaluate Best Suited Tools & Techniques

There needs to be collaboration between DEV and QA to find the best tool. Pair it with what the developers are using. If automating an AngularJS front-end, Protractor and JavaScript may be a better tool than Selenium WebDriver / Java.

Need Mobile applications in the future? Know that Selenium / Java may not be the best solution since it does not do Mobile.

Automation Design Strategy

You want to decide what you want to use for:
  • Test Data
  • What type of Global Configs and Test Driver
  • What Page Objects to feed into Test?
  • What is the scheduling?
  • What are the reports going to look like?

Note, you want your framework to be:

  • Loosely coupled
  • Customized Actions
  • Global Configurations,
  • Application Independent Utilities
  • Test Data Parameterization: Data independence

How do you want to identify the web elements you are interacting with?

  • ID? Class? Link? Name? Tag? CSS? Xpath?


  • What is the framework development process?
  • What is your strategy for writing the framework?
  • Building an automation framework is building a software project: How to code commit, how to do code reviews, how to develop the framework?

Create a living document showing your framework development process.

Script Creation Strategy

Remember: A Record and playback is never enough.
  • Script Mapping strategy.
  • Script Pattern Strategy
  • Script Commenting Strategy
  • Independent tests
  • Unique Test Data
  • Exception Handling
  • Controlled States
  • Tear Down
  • Soft and Hard Assertions

Execution Strategy

  • You need to take a test suite, break it up into different groups, and run these groups of tests in Parallel.
  • You also need a way to re-execute any errors.

... What are you capturing in Reports?

Reporting Strategy

Reporting is very important. What type of reporting works in your organization? Who needs to be informed? How do you frame the report to answer questions about what is happening with the testing? Are they looking for a Dashboard, to Drill down as results, to have Analysis? Should the results be emailable?

Note: You cannot improve what you cannot measure. You need good automation strategies, and find good metrics to work with.

Maintenance Strategy

How can you maintain your scripts? How do you deal with flaky tests? How do you know it is not a flaky test, but is instead an error that sporadically keeps on happening?

Flaky tests: Fails inconsistently wasting developers and QA engineer’s time. It damages the reputation for automation.

Don’t turn them off, Isolate the flaky tests into a separate group.

Checklist: Is Your Automation Future Proof?

Want more Webinars from QASource?

See https://www.qasource.com/webinars

  • Clean Up Your High Maintenance Test Automation Framework
  • Measuring Your Way To Successful Automation Webinar With Experts
  • Reducing False Positives Webinar With QASource’s Webinar

QASource: Reducing False Positives in Automated Testing

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

January 18, 2017

Boston area Software testing recruiting firms: Share your industry experience. Host a Software Testing Meetup Event!

Are You a Recruiting Company or Placement Firm in the Boston or Cambridge, MA area? Are you having trouble finding software testing candidates to meet the needs of your clients? Have you ever thought about hosting a Meetup event at your place of business, educating software testers on how to keep up with the ever changing software industry and sharing your knowledge ?

A newly rebranded Meetup group of software testers, The Ministry of Testing - Boston, has just been formed, with the Greater Boston QA and Testing Meetup moving under the umbrella of the Ministry of Testing, a UK-based software testing community. We are looking to place new events on our event calendar for our almost four hundred person strong Meetup.

Formerly the Greater Boston QA and Testing Meetup
Ministry of Testing - Boston website: https://www.meetup.com/ministry-of-testing-boston/
Ministry of Testing: http://www.ministryoftesting.com/

Imagine an event where your recruiters could give the "inside scoop" on how software testers adjust their skill set to meet the needs of today's job market, navigating these uncertain times.

Imagine establishing a relationship with thirty or so future candidates for software testing positions as you answer difficult questions such as:

  • Is there hope for traditional manual tester without a coding background to get placed in a perm or contract position?
  • What tools and technologies does your company forecast to be in hot demand in 2017?
  • What is the best strategy for potential candidates to get themselves ready for automation development positions?
  • What do you look for in a resume?
  • How do you pitch candidates to clients who may not be an exact match?
  • Do coding bootcamps even help QA Engineers? If so, which ones? 
  • What are the difference qualifications needed between a mid-level QA Engineer, a Senior QA Engineer, and a Software Engineer in Test?
  • What qualifications does it take to be a QA Lead or a QA Manager?

This could be all addressed in one Meetup event or a series of Meetup events.

... And, of course, members would be encouraged to bring their resume.

Interested? Share your event idea, what you would like as a discussion topic, and the location of the Meetup Event with Conrad Holloman, or T.J. Maher.

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

January 17, 2017

Looking for ways to improve your software testing? Purchase a TestSphere Deck!

The Ministry of Testing just announced a new product to help software testers improve their craft: They have distilled their collective wisdom down into the size of a deck of cards.

Pre-order now! Shipping in March 2017.

Taken from the Ministry of Testing Store site:

"A hundred cards featuring a hundred testing concepts: Heuristics, Techniques, Feelings, Quality Aspects and Patterns. 

"Each card explains the concept with a slogan and three examples that were designed to have you think about them in different ways.

"Testers brainstorm, coach, share knowledge and share experiences by using these cards as a conversation starter, a story telling game, a tool that facilitates workshops and as a guide for their testing sessions.

"Whether you’re a new tester looking for a fun way to learn about testing or one with many years of experience behind you, TestSphere aims to unlock barriers that you didn’t know were there".

The five categories for the cards:

  • Heuristics: Possible ways of tackling a problem.
  • Techniques: Clever activities we use in our testing to find possible problems.
  • Feelings: Every feeling that was triggered by your testing should be handled as a fact.
  • Quality Aspects: Possible aspects of your application that may be of interest.
  • Patterns: Patterns in our testing, but also patterns that work against us, while testing such as Biases.

The Ministry of Testing is a software testing group that is heavily active in the United Kingdom, making inroads in the United States in the past year, and most recently with the Ministry of Testing - Boston Meetup group, where I just became co-organizer.

Buy your pack now! They will be shipping in March 2017.

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

January 9, 2017

Know any Coding IDEs for a Chromebook?

Two weeks ago, the hard drive on my home computer started clicking. That is when the blue screens happened. DTC_WATCHDOG_ERRORS and stopcodes thrown in Windows 10. First once a day, then once an hour, and this weekend, every time my Desktop loads up.

While I am trying to figure out what PC to buy next (Dell XPS? Dell Inspiron? All-in-one? Regular desktop? I surf the web, use IntelliJ, and not much else on it), I was going to pick up a cheap Chromebook to handle my daily requirement of blogging and web surfing at home, or blogging about various Meetups I attend.

Let's say I want to tinker with Java or JavaScript. Is there an IDE you like to use? I was tinkering with blogging about REST Assured next, but I didn't think doing any Java development, no matter how small the project, would work on a Chromebook, but I wanted to check.

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

January 3, 2017

New Ministry of Testing: Boston Meetup: Software Testing Lean Coffee, 2/07/2017 6:30 pm @ John Harvard's Brewpub, Cambridge, MA

Any QA Engineers in the Boston area? Come join the new Ministry of Testing: Boston group as we hold a roundtable discussion at an actual round table, at John Harvard's Brewpub in Cambridge, MA. I'll be next door before the Meetup if anyone wants to chat beforehand.
"T.J. Maher is 5 foot 7, with short brown hair, blue eyes, and a red MEETUP sign attached to his black messenger bag. He will be waiting in The Garage Food Court, Lower Level [Tuesday, February 7th] from 6:00 pm to 6:20 pm sitting at a table with a red MEETUP table tent".
Looking for directions? Want to RSVP? Click here.

Happy Testing!

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"

Ever run a Lean Coffee for Software Testing Meetups before? Looking for pointers!

The Agile End-of-Sprint Retrospective

My favorite part of the Agile Software Development methodology is the End-Of-Sprint Retrospective. Every two weeks, the software development gets to come together as a whole and reflect on just how much they accomplished. During the meeting, they discuss what went well, what didn't go well, what the team needs to work on.

Everyone gets a voice during the discussion: Not just the product owner. Not just the developers. Not just the outspoken. Although it may take some coaxing from a skilled scrum master playing various Agile games, each individual team member, even the introverted ones, gets a chance to write down what the group should talk about, after grouping up similar topics vote on what issues should be discussed, and speak about them.


  • My favorite Agile game is the Sad, Glad, or Mad game, mentioned in Esther Derby's Agile Retrospectives: Making Good Teams Great (2006) (Write on a Post-It Note what made you mad, what made you glad, and what made you mad during this past sprint).
  • Make sure to begin the retro with the Retrospective Prime Directive from Norm Kerth's, Project Retrospectives: A Handbook for Team Review (2001)  "Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand."

... I've often wondered if the same methods would work in a round-table discussion -- write discussion topics on Post-It Notes, vote on topics to discuss to get the priority, "timebox" them at ten minutes, then go to the next topic -- only to find recently that Jim Benson and Jeremy Lightsmith beat me by six or seven years.

The Start of "Lean Coffee"

"Lean Coffee started in Seattle in 2009. Jim Benson and Jeremy Lightsmith wanted to start a group that would discuss Lean techniques in knowledge work – but didn’t want to start a whole new cumbersome organization with steering committees, speakers, and such. They wanted a group that did not rely on anything other than people showing up and wanting to learn or create". - LeanCoffee.Org

"The Lean Coffee format is both easy to follow and effective at facilitating learning and collaboration through group discussions. Although the name combines ‘Lean’ (eg. Lean Thinking, Lean Startup, etc.) and ‘Coffee’ (implying casual morning sessions), neither the topics nor the meeting times need be so rigid. For instance, I’ve attended Lean Coffee meetups held in mornings, afternoons and evenings. You can gather at a local coffee house, a pub or at your office. Most successful Lean Coffee groups maintain a reliable cadence, meeting at the same time and place each week or two". - AgileCoffee.com

Taken from How to Run a Lean Coffee by AgileWeekly:

How About A Software Testing Lean Coffee in Boston?

Stealing the concept from the Ministry of Testing - London Meetup, I would like to run one or two Lean Coffees for the newly re-branded Ministry of Testing - Boston and see how it goes.

What I was thinking of:

  • Find a restaurant that seats 10 or 15 (smaller group to start) in Cambridge or Boston
  • Bring along a few pads of Post-It Notes and Sharpies to pass around to attendees. 
  • Have guests fill out Post-It Notes loosely related to Software Testing or QA
  • Place collected Post-It Notes in a pile saying "To Discuss"
  • Read out the topics once to let people hear them, then a second time for people to vote on them -- everyone gets two votes. The Post-It Notes are sorted according to priority. 
  • People vote on how long to initially talk about the subject: 5 minutes or 10 minutes.
  • The discussion begins! Place the topic in a "Discussing" pile. Start the timer on the phone. People then can vote if they want to extend it another five or ten minutes. 
  • Discussion over? Place the topic in the "Discussed" pile, and it is on to the next topic! Take the next one from the "To Discuss" pile and repeat the process. 
... Have you ever run a Lean Coffee before? What do you think? Any pointers?

If you are from the Ministry of Testing - London, I'd especially love to hear from you! 

Thank you for your time! 

-T.J. Maher
Twitter | LinkedIn | GitHub

// Sr. QA Engineer, Software Engineer in Test, Software Tester since 1996.
// Contributing Writer for TechBeacon.
// "Looking to move away from manual QA? Follow Adventures in Automation on Facebook!"