Unit testing wai applications by Mark Seemann
One way to unit test a wai application with the API provided by Network.Wai.Test.
I'm currently developing a REST API in Haskell using Servant, and I'd like to test the HTTP API as well as the functions that I use to compose it. The Servant documentation, as well as the servant Stack template, uses hspec to drive the tests.
I tried to develop my code with hspec, but I found it confusing and inflexible. It's possible that I only found it inflexible because I didn't understand it well enough, but I don't think you can argue with my experience of finding it confusing.
I prefer a combination of HUnit and QuickCheck. It turns out that it's possible to test a wai application (including Servant) using only those test libraries.
Testable HTTP requests #
When testing against the HTTP API itself, you want something that can simulate the HTTP traffic. That capability is provided by Network.Wai.Test. At first, however, it wasn't entirely clear to me how that library works, but I could see that the Servant-recommended Test.Hspec.Wai is just a thin wrapper over Network.Wai.Test (notice how open source makes such research much easier).
It turns out that Network.Wai.Test enables you to run your tests in a Session monad. You can, for example, define a simple HTTP GET request like this:
import qualified Data.ByteString as BS import qualified Data.ByteString.Lazy as LBS import Network.HTTP.Types import Network.Wai import Network.Wai.Test get :: BS.ByteString -> Session SResponse get url = request $ setPath defaultRequest { requestMethod = methodGet } url
This get function takes a url and returns a Session SResponse. It uses the defaultRequest, so it doesn't set any specific HTTP headers.
For HTTP POST requests, I needed a function that'd POST a JSON document to a particular URL. For that purpose, I had to do a little more work:
postJSON :: BS.ByteString -> LBS.ByteString -> Session SResponse postJSON url json = srequest $ SRequest req json where req = setPath defaultRequest { requestMethod = methodPost , requestHeaders = [(hContentType, "application/json")]} url
This is a little more involved than the get function, because it also has to supply the Content-Type HTTP header. If you don't supply that header with the application/json value, your API is going to reject the request when you attempt to post a string with a JSON object.
Apart from that, it works the same way as the get function.
Running a test session #
The get and postJSON functions both return Session values, so a test must run in the Session monad. This is easily done with Haskell's do notation; you'll see an example of that later in the article.
First, however, you'll need a way to run a Session. Network.Wai.Test provides a function for that, called runSession. Besides a Session a value, though, it also requires an Application value.
In my test library, I already have an Application, although it's running in IO (for reasons that'll take another article to explain):
app :: IO Application
With this value, you can easily convert any Session a to IO a:
runSessionWithApp :: Session a -> IO a runSessionWithApp s = app >>= runSession s
The next step is to figure out how to turn an IO a into a test.
Running a property #
You can turn an IO a into a Property with either ioProperty or idempotentIOProperty. I admit that the documentation doesn't make the distinction between the two entirely clear, but ioProperty sounds like the safer choice, so that's what I went for here.
With ioProperty you now have a Property that you can turn into a Test using testProperty from Test.Framework.Providers.QuickCheck2:
appProperty :: (Functor f, Testable prop, Testable (f Property)) => TestName -> f (Session prop) -> Test appProperty name = testProperty name . fmap (ioProperty . runSessionWithApp)
The type of this function seems more cryptic than strictly necessary. What's that Functor f doing there?
The way I've written the tests, each property receives input from QuickCheck in the form of function arguments. I could have given the appProperty function a more restricted type, to make it clearer what's going on:
appProperty :: (Arbitrary a, Show a, Testable prop) => TestName -> (a -> Session prop) -> Test appProperty name = testProperty name . fmap (ioProperty . runSessionWithApp)
This is the same function, just with a more restricted type. It states that for any Arbitrary a, Show a, a test is a function that takes a as input and returns a Session prop. This restricts tests to take a single input value, which means that you'll have to write all those properties in tupled, uncurried form. You could relax that requirement by introducing a newtype and a type class with an instance that recursively enables curried functions. That's what Test.Hspec.Wai.QuickCheck does. I decided not to add that extra level of indirection, and instead living with having to write all my properties in tupled form.
The Functor f in the above, relaxed type, then, is in actual use the Reader functor. You'll see some examples next.
Properties #
You can now define some properties. Here's a simple example:
appProperty "responds with 404 when no reservation exists" $ \rid -> do actual <- get $ "/reservations/" <> toASCIIBytes rid assertStatus 404 actual
This is an inlined property, similar to how I inline HUnit tests in test lists.
First, notice that the property is written as a lambda expression, which means that it fits the mould of a -> Session prop. The input value rid (reservationID) is a UUID value (for which an Arbitrary instance exists via quickcheck-instances).
While the test runs in the Session monad, the do notation makes actual an SResponse value that you can then assert with assertStatus (from Network.Wai.Test).
This property reproduces an interaction like this:
& curl -v http://localhost:8080/reservations/db38ac75-9ccd-43cc-864a-ce13e90a71d8 * Trying ::1:8080... * TCP_NODELAY set * Trying 127.0.0.1:8080... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 8080 (#0) > GET /reservations/db38ac75-9ccd-43cc-864a-ce13e90a71d8 HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.65.1 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 404 Not Found < Transfer-Encoding: chunked < Date: Tue, 02 Jul 2019 18:09:51 GMT < Server: Warp/3.2.27 < * Connection #0 to host localhost left intact
The important result is that the status code is 404 Not Found, which is also what the property asserts.
If you need more than one input value to your property, you have to write the property in tupled form:
appProperty "fails when reservation is POSTed with invalid quantity" $ \ (ValidReservation r, NonNegative q) -> do let invalid = r { reservationQuantity = negate q } actual <- postJSON "/reservations" $ encode invalid assertStatus 400 actual
This property still takes a single input, but that input is a tuple where the first element is a ValidReservation and the second element a NonNegative Int. The ValidReservation newtype wrapper ensures that r is a valid reservation record. This ensures that the property only exercises the path where the reservation quantity is zero or negative. It accomplishes this by negating q and replacing the reservationQuantity with that negative (or zero) number.
It then encodes (with aeson) the invalid reservation and posts it using the postJSON function.
Finally it asserts that the HTTP status code is 400 Bad Request.
Summary #
After having tried using Test.Hspec.Wai for some time, I decided to refactor my tests to QuickCheck and HUnit. Once I figured out how Network.Wai.Test works, the remaining work wasn't too difficult. While there's little written documentation for the modules, the types (as usual) act as documentation. Using the types, and looking a little at the underlying code, I was able to figure out how to use the test API.
You write tests against wai applications in the Session monad. You can then use runSession to turn the Session into an IO value.
