A little confusion about one image of the paper
The author's work is great!There's a question i want to ask: context:
` Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.
Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.
Thanks for your question, we have found a mistake in this picture. The orange line shoule be w/o batching and the blue one shoule be OTP batching.
`
Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.