Shubham Tagra

Macro based whitelisting of locations allowed to cache

2

This is useful particularly in case of partitioned tables. Today, whitelisting is a regex e.g. if user wants to whitelist two tables which appear in location `reviews` and `bookings` under...

Publish jmx metrics for connection pool

2

Adding following jmx metrics, per host level, should help in debugging leaked connections: - Total active connections in pool - Connections borrowed from pool

It can happen that right when the eviction of a file occurs a new request comes for the file. In such a case the new request may see a `NoSuchFileException`...

Need some experimentation to figure out a good value but current value of 200 is too big. With 'N' nodes and 2 servers, one node could be handling 2*200*N connections...

Handle Interrupts better

1

Today interruptions are thrown as RuntimeException which bubbles up as warning logs. Most of the times these interrupts are expected and not needed to be logged e.g. user cancel a...

When we allow on-disk cache to be reused across BKS restarts, we have following issues in space accounting: 1. Available space doesnt consider the space occupied by cached files. It...

We do an extra listStatus call for every FS.open() call, in CachingInputStream creation, to get the fileSize and lastModifiedDate. This increase the api calls to clouds and can mean additional...

Refer https://github.com/qubole/rubix/pull/425#discussion_r455572415

Noticed that CI tests somehow get stuck, one of the runs was stuck in `TestBookKeeperFactory` in `Waiting for MockBookKeeperServer to come up` when the `MockBookKeeperServer` had failed to come up...