Testing asynchronous APIs

[stevana]

Work in progress.

Consider the following example, a simple server that when sent a number replies by sending the sum of all numbers it has been sent so far (asynchronously, say on a message bus).

data Action v :: * -> * where
  Add :: Int -> Action v ()
  Sum :: Action v Int

Model = Int

next m (Add i) = m + i
next m Sum     = m

post m (Add i) _ = True
post m Sum     i = i == m

So far so good, but what is the semantics for Sum?

sem (Add i) = apiCallToEndpoint i server
sem Sum     = ?

It doesn't really make sense, the Sum action is created by the environment (message bus).

Here's an idea, don't generate Sum actions. Instead let the user provide a function:

environmentEvents :: TChan History -> IO ()

That has access to the history channel, and can listen to the bus and add Sum events. It's important that the user adds a InvocationEvent for Sum immediately after it sees a ResponseEvent for Add on the channel -- otherwise linearisation might fail, e.g.:

|--- Add 1 ---| |--- Add 2 ---|
                                 |-- Sum 1 --|

The ResponseEvent for Sum should be added when the actual message appears on the message bus.

One clear downside of this approach is that it exposes an implementation detail, the history channel, to the user. I'm also not sure what to do about pids (we can't/shouldn't have multiple pending invocations on the same pid -- that would break sequentiality of the thread).

[stevana]

One way to hide the history channel from the user, could be to have the user provide a function that triggers invocation events:

triggers :: action Concrete resp -> Maybe (exists resp'. action Concrete resp')

And another function that creates response events:

environmentResponses :: Producer IO (Constructor, Dynamic)

In the above example the implementations of these would be something like:

triggers (Add i) = Just Sum
triggers Sum    = Nothing

environmentResponses = do
  msg <- readMessageFromBus
  case msg of
    'S' : 'u' : 'm' : ' ' : n : [] -> yield (Constructor "Sum", toDyn (read n :: Int))
    _ -> environmentResponses

[stevana]

I still think this is an interesting topic, but I'm not working on this actively.

If I would revisit it, then I would try to combine what I've learned from https://github.com/advancedtelematic/quickcheck-state-machine-distributed and the literature on behavioral programming.

[stevana]

https://web.archive.org/web/20160405163934/people.brunel.ac.uk/~csstrmh/research/asynchronous_testing.html