The internet is a harsh mistress. Sites go down, change without notice, or even just disappear entirely. The web ―are you sitting down? ―is not 100 percent reliable. This means that testing a project that has external dependencies, things can fail that aren’t even your bugs. What’s a software testing engineer to do?!
Make your own internet, that’s what. Or at least a network layer mocking system to take care of that outbound traffic, so there are no third party downtime network issues or other constraints that break your test. The software engineering team at the LinkedIn social networking service announced in a blog post Friday that they have done just that, building a new internet mocking tool called Flashback to remove that uncontrolled variable from the testing equation.
A variety of internet mocking tools is already available, of course. These work by intercepting HTTP connections initiated by a web application, and then later “replaying” them as needed. In other words, a mocking tool replaces any HTTP interaction with its previously recorded instance which can be reliably served as many times as necessary in testing.
The LinkedIn testing team, in fact, started out using Betamax , an open-source test automation tool that did just that. Shangshang Feng , a LinkedIn software engineer, said that there were many things the team liked about Betamax ―but that there were some capabilities they needed that it did not provide, particularly for service level integration testing. And so Feng, with colleagues Yabin Kang and Dan Vinegrad , drew upon Betamax’s open source roots to build a new network layer mocking system called Flashback.
Flashback is designed to mock HTTP and HTTPS resources, like web services and REST APIs, for testing purposes. It records HTTP/HTTPS requests and plays back a previously recorded HTTP transaction ― which in Flashback is called a “scene” ― so that no external internet calls are required during testing.
“Imagine a standard LinkedIn profile page,” said Feng. “It requires more than 100 downstream dependency services to render. If you are testing only one tiny specification on this page, you really don’t want to rely on all those dependencies. Especially when some of these dependencies actually are eternal services from the internet, which you have no way to control.”
The best way to ensure a consistent testing environment, said Feng, is to mock up the dependencies. Going back to our LinkedIn profile page example, these pages use something called LinkedIn Ingestion Service to grab external content ―a YouTube video, say ― to enrich your LinkedIn profile. Typical three-tier architecture has a gateway service in the middle, Feng explained. “This is where Flashback fits in ―this middle layer that is working as a proxy. Our opportunity is to mock in this layer to test our Ingestion Service result without relying on the external providers.”
Flashback handles HTTP/HTTPS traffic, intercepts the request and response and stores them for “replay” as necessary. This completely isolated testing environment is Flashback’s main area of innovation over Betamax, but there’s more. Flashback also handles authentication calls from third party websites. “None of the other open source tools was doing that,” said Yabin Kang, another of Flashback’s architects. “When talking to the internet you always need to handle the authentication, and often some parameter in the request is different. So we wanted to give the user the ability to match this request exactly.”
HTTP requests generally contain a URL, method, headers, and body. Flashback allows match rules to be defined for any combination of these components. Flashback also allows users to add whitelist or blacklist labels to URL query parameters, headers, and the body.
Kang and Feng pointed to the following hypothetical OAuth authorization flow, the request query parameters may look like the following:
oauth_consumer_key="jskdjfljsdklfjlsjdfs",oauth_nonce="ajskldfjalksjdflkajsdlfjasldfja;lsdkj",
oauth_signature="asdfjaklsdjflasjdflkajsdklf",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="1318622958",
oauth_token="asdjfkasjdlfajsdklfjalsdjfalksdjflajsdlfa",
oauth_version="1.0"
Many of these values will change with every request, since OAuth requires clients to generate a new value for oauth_nonce every time. “In our testing, we need to verify values of oauth_consumer_key, oauth_signature_method, and oauth_version while also making sure that oauth_nonce, oauth_signature, oauth_timestamp, and oauth_token exist in the request. Flashback gives us the ability to create our own match rules to achieve this goal,” the engineers wrote in the blog post announcing Flashback’s release. “This feature lets us test requests with time-varying data, signatures, tokens, etc. without any changes on the client side.”
The LinkedIn team is excited to offer Flashback as an open source tool for any test team that may find it of use. Flashback is written in Java but can support any environment using HTTP service. Feng first presented the proof of concept for Flashback at the 2015 Google Test Automation Conferenceand was approached afterward by multiple people asking if the project would be open source. “Initially I thought this would only be usefully internally at LinkedIn, but these audience members made me realize this is a testing scenario that many companies would have a use for,” said Feng.
The team refined Flashback over the course of 2016, culminating in today’s release. The ultimate question, of course, is “how does this make my life better as a developer?”
The LinkedIn team reports that Flashback now helps them release faster. The company has a longstanding commitment to making their testing environment as automated as possible, and “an aggressive testing philosophy” geared to rapid deployment of new code. “When we can do continuous integration within reliable test infrastructure, then we know if everything passed ― our service is doing fine, and we can quickly and confidently deploy that new version,” said Feng.
For instance, said Feng, “We used to get data, content, directly from, say, YouTube. Sometimes things would change from the source ―different title or other changes. And that would cause a failure that had nothing to do with our code.”
Flashback, he said, can be used just like a testing framework for. “After setup, everything you used to have to do, writing tests old style to get data from YouTube, is now handled by Flashback.”
With one significant added bonus, Feng added: “Now, in integration testing, you know your ‘external’ responses are going to be served reliably and consistently.”
Feature Image: (Left to right) Shangshang Feng, Daniel Vinegard, Yabin Kang (courtesy LinkedIn.)