Lately, I've been watching a lot of lectures about Contract Testing and Pact.io, trying to prepare for an upcoming job interview. When diving into a new toolset I can never simply jump into the code. I need to know: Why was this toolset created? What problem did it solve? How was this tool created? How did this toolset evolve?
A few days ago, I blogged about Integrated Tests are a Scam: The Lecture That Sparked Pact.io talking about J. B. Rainsberger's 2013 lecture. Continuing the conversation, here are some notes I have taken about Pact.
The History of Pact.io
According to Pact Docs / History:
"Pact was originally written by a development team (including Ron Holshausen from DiUS) at realestate.com.au in 2013. They were trying to solve the problem of how to do integration testing for their new Ruby microservices architecture, and they threw together a codebase that was to become the first Pact implementation. As the team was writing Ruby microservices, the serialization of the "matching logic" (e.g. regular expressions) was done using Ruby specific code. The team was under the pump to finish their actual customer deliverable, so they got their Pact MVP working, and left it at that.
"A few months later Beth Skurrie from DiUS joined the one of the teams that was working with the Pact authors' team. She had recently seen a talk by J.B.Rainsberger entitled 'Integration tests are a scam', which promoted the concept of 'collaboration' and 'contract' tests, so she was immediately interested when she was introduced to Pact. After trying (as most people do) to convince the authors that the provider should be the contract writer, she was soon convinced by Brent Snook, one of the original authors, of the value of consumer driven contracts. At this stage, she realised that what was missing in the current implementation was the ability to run the same request under different conditions, and "provider states" were born.
"[...] Pact spread around the codebases in the wider program of work at realestate.com.au, until it hit its first Java microservice. realestate.com.au had many JVM projects, so a group of DiUS consultants (including Ron Holshausen again) started the pact-jvm project on a hack day. It was at this stage that the authors realized that the Rubyisms in the format were going to have to be replaced by a non-language specific format, and the idea of the v2 pact-specification arose (though it would take a while before it became reality)".
SmartBear Acquires Pact.io
In April 2022, DiUS sold PactFlow to SmartBear, the company behind Swagger, ReadyAPI, and other API toolsets. SmartBear has then been the steward for Pact, providing a lot of information about the toolset:
Docs.Pact.io has a basic Introduction https://docs.pact.io/
SmartBear University at https://smartbear.com/academy/ offers simple courses in Introduction to Contract Testing https://smartbear.com/academy/pactflow/introduction-to-contract-testing
Swagger Contract Testing University: https://support.smartbear.com/swagger/contract-testing/docs/en/swagger-contract-testing-university/overview.html offers videos and workshops on introductions to Pact, CI / CD, and Creating a Pact Plugin, along with advanced workshops.
But What Is Contract Testing, according to SmartBear Academy?
Let's say you have a server that provides data, such as a list of all the users on the system, sharing it through an API to a mobile app. The mobile app would be the Consumer, and the server would be the Provider.
If you wanted to test that things were working correctly, according to SmartBear's course, "Introduction to Pact", you can test things in isolation, breaking the test into two parts:
- The Consumer talks to a mock provider, which simulates sending the data back to the consumer.
- Provider talks to a simulated consumer which simulates requesting the data.
Example: You don't trigger a smoke alarm by setting your house on fire, and seeing if it responds. You simulate a fire by pressing the button to send the signal, and you test that the smoke alarm is triggered.
The Consumer can talk to a mock provider and the Provider can talk to a simulated consumer.
Contract testing ensures that a provider is compatible with the consumer expectations of it. These needs are captured in a Contract.
Example: An http system would check that the provider accepts the expected requests, and that it returns the expected responses. The contract is verified against the Provider. This is Contract Testing.
Consumer Driven Contract Testing
When using consumer driven contract testing, the consumers expectations are serialized to a contract file during the execution of its own automated tests, and then verified against a provider during the providers automated tests. The contract is generated in such a way that ties the content to the consumer code, either by use of a "test double" that records requests and expected responses and writes them to a contract. An HTTP contract would consist of request / response pairs.
A Consumer would send a response, such as GET /orders/1234 an HTTP request, and the mock would give an http response back, such as id: 1234, and list items. This is recorded, and matched with something such as PactFlow,
If the actual responses match the expected responses, the contract has been successfully verified. Any mismatches show an incompatibility between Consumer and Provider.
Provider Verification
Provider Verification Step: runs locally where the requests in the Contract file are played back against a REAL instance of the provider, to make sure it captures all expectations in the consumer contract. The provider team stubs out all the dependencies keeping the focus on the consumers expectations. When you include these tests in the providers own build, running on the dev build, it provides fast feedback and helps ensure that breaking changes are not committed.
The Pact Broker
A Pact Broker:
- Open source application sharing consumer driven contacts.
- Can be use by any contract that is serialized into JSON, but is specifically for Pact.
- Comes with a Pact command line interface that is used to set up the contract testing workflow.
- See Sharing the Pact With Pact Broker (SmartBear Docs)
Contracts and verification results are sent to the Pact broker from the CI Builds on the consumer and provider. Consumer CI build generates a Pact during the run of its isolated tests, and publishes the generated Pact Broker. The Provider CI retrieves the Pact, performs the verification, publishes the results locally, back to the Broker. Consumer and Provider CI Builds check with the broker before deploying, to ensure the application version they are about to deploy are compatible of the versions already in the environment.
The Pact Broker: No integration tests needed. Services can be deployed independently. You can share contracts and verification results between consumer and provider projects. It can tell you which versions of the application can be deployed safely together. It versions your contracts, and ensures backward compatibility, between multiple consumer and provider versions. It also provides API documentation of your application that is guaranteed to be up to date. It also shows you how your services really interact. It also allows you to visualize your relationship between your services.
The Pact Broker can allow you to self host, or use PactFlow a SaaS package.
CI / CD
Contract Testing can be used in CI / CD platforms, such as Jenkins. Consumer driven contract testing: Consumer pipeline, you run tests from the consumer's codebase, configure a test to publish the contract to PactFlow. Then you configure a step in the provider CI / CD pipeline that takes the PactFlow that verifies the contract, publishes the results to Pactflow. Optionally, you have the step, "can-i-deploy". Best way to use these steps is the Pact Broker Client command line interface. It allows you to create, update, and delete and query resources in PactFlow. It is available as a Docker image and a standalone executable.
-T.J. Maher
Software Engineer in Test
BlueSky | YouTube | LinkedIn | Articles
No comments:
Post a Comment