Co-authors:Shangshang Feng,Yabin Kang, andDan Vinegrad
At LinkedIn, we often develop web applications that need to interact with third-party websites. We also employ automatic testing to ensure the quality of our software before it is shipped to production. However, a test is only as useful as it is reliable. With that in mind, it can be highly problematic for a test to have external dependencies, for instance on a third-party website. These external sites may change without notice, suffer from downtime, or otherwise become temporarily inaccessible, as the internet is not 100 percent reliable. If one of our tests relies on being able to communicate with a third-party website, the cause of any failures is hard to pinpoint. A failure could be due to an internal change at LinkedIn, an external change made by the maintainers of the third-party website, or an issue with the network infrastructure. As you can imagine, there are many reasons why interactions with a third-party website may fail. So, you may wonder, how will I deal with this problem? The good news is that there are many internet mocking tools that can help. One such tool is Betamax . It works by intercepting HTTP connections initiated by a web application, and then later replaying them. For a test, Betamax can be used to replace any interaction over HTTP with previously recorded responses, which can be served very reliably.
Initially, we chose to use Betamax in our test automation at LinkedIn. It worked quite well, but we ran into a few problems, such as:
For security reasons, our test environment does not have internet access; however, as with most proxies, Betamax requires an internet connection to function properly.
We have many use cases that require using authentication protocols, such as OAuth and OpenId. Some of these protocols require complex interactions over HTTP. In order to mock them, we needed a sophisticated model for capturing and replaying the requests.
To address these challenges, we decided to build upon ideas established by Betamax and create our own internet mocking tool, called Flashback. We are also proud to announce that Flashback is now open source.
What is Flashback?Flashback is designed to mock HTTP and HTTPS resources, like web services and REST APIs, for testing purposes. It records HTTP/HTTPS requests and plays back a previously recorded HTTP transaction―which we call a "scene"―so that no external connection to the internet is required in order to complete testing.
Flashback can also replay scenes based on partial matching of requests. It does so using “match rules.” A match rule associates an incoming request with a previously-recorded request, which is then used to generate a response. For example, the following code snippet implements a basic match rule, where the test method “matches” an incoming request via the URL.
HTTP requests generally contain a URL, method, headers, and body. Flashback allows match rules to be defined for any combination of these components. Flashback also allows users to add whitelist or blacklist labels to URL query parameters, headers, and the body.
For instance, in an OAuth authorization flow, the request query parameters may look like the following:
oauth_consumer_key="jskdjfljsdklfjlsjdfs",
oauth_nonce="ajskldfjalksjdflkajsdlfjasldfja;lsdkj",
oauth_signature="asdfjaklsdjflasjdflkajsdklf",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="1318622958",
oauth_token="asdjfkasjdlfajsdklfjalsdjfalksdjflajsdlfa",
oauth_version="1.0"Many of these values will change with every request, since OAuth requires clients to generate a new value for oauth_nonce every time. In our testing, we need to verify values of oauth_consumer_key, oauth_signature_method, and oauth_version while also making sure that oauth_nonce, oauth_signature, oauth_timestamp, and oauth_token exist in the request. Flashback gives us the ability to create our own match rules to achieve this goal. This feature lets us test requests with time-varying data, signatures, tokens, etc. without any changes on the client side.
This flexible matching and the ability to function without connecting to the internet are the attributes that separate Flashback from other mocking solutions. Some other notable features include:
Flashback is a cross-platform and cross-language solution, with the ability to test both JVM and non-JVM (C++, python, etc.) apps.
Flashback can generate SSL/TLS certificates on the fly to emulate secured channels for HTTPS requests.
How to record an HTTP transaction
Recording an HTTP transaction for later playback using Flashback is a relatively straightforward process. Before we dive into the procedure, let us first lay out some terminology:
A Scene stores previously-recorded HTTP transactions (in JSON format) that can be replayed later ( example ).
The Root Path is the file path of the directory that contains the Flashback scene data.
A Scene Name is the name of a given scene.
A Scene Mode is the mode in which the scene is being used―either “record” or “playback.”
A Match Rule is a rule that determines if the incoming client request matches the contents of a given scene.
Flashback Proxy is an HTTP proxy with two modes of operation, record and playback.
Host and port are the proxy host and port.
In order to record a scene, you must make a real, external request to the destination, and the HTTPS request and response will then be stored in the scene with the match rule that you have specified. When recording, Flashback behaves exactly like a typical “Man in the Middle” (MITM) proxy―it is only in playback mode that the connection flow and data flow are restricted to just between the client and the proxy.
To see Flashback in action, let us create a scene that captures an interaction with example.org by doing the following:
Step 1: Checkout Flashback source code: `git clone https://github.com/linkedin/flashback.git `
Step 2: Start the Flashback admin server: `./startAdminServer.sh -port 1234`
Step 3: Start Flashback Proxy:
Note the Flashback example above will be started in record mode on localhost, port 5555. The match rule requires an exact match (match HTTP body, headers, and URL). The scene will be stored under /tmp/test1.
Step 4: Flashback is now ready to record, so use it to proxy a request to example.org: `curl http://www.example.org -x localhost:5555 -X GET`
Step 5: Flashback can (optionally) record multiple requests in a single. To finish recording,shutdown Flashback: `curl "