Gangmuk Lim
Gangmuk Lim
thank you very much for quick answer. Can I ask a few more followed up questions? 1. Can we know how many and how large chunks of dataset TF brings...
Cross posting my comments here for future consideration --- We can make it pluggable. It should be system-wide variable which shouldn't be changed during the runtime. Otherwise, it will mess...
My previous proposal to move tokenizer to the gateway was wrong. I missed the fact that the tokenization is only being used in prefix aware routing. I measured the tokenizer...
What log do you mean? pod log? @Jeffwan I didn't know I was supposed to update that part in yaml. I was using what it has been running there. Will...
@Jeffwan It was resolved. I remember I was using the wrong unit. (0.0-1.0 v.s. 0-100)
@Jeffwan Hmm. I think it was triggered. I will see next time I push benchmarks
@Jeffwan you are right. I made another PR under benchmarks dir. It didn't trigger the CI test. this is the current `workflow/installation-tests.yml`. which it seems not including benchmarks path. Is...
@linjianshu did you run it with v0.3.0? and if so, can you let me know the setup and workload? I want to run a experiment with workload in similar cluster...
@Jeffwan @kerthcet Dynamo maintains consistent view using kind of pub-sub implementation [dynamo kv routing](https://github.com/ai-dynamo/dynamo/blob/main/docs/kv_cache_routing.md#dynamo-events)
@Jeffwan @DwyaneShi Could you check this issue? I have been just waiting for the engine pods to be restarted. But it is not supposed to be like that. This is...