phoenix icon indicating copy to clipboard operation
phoenix copied to clipboard

PHOENIX-7166 : Set default values of feature flags introduced by metadata caching redesign

Open palashc opened this issue 2 years ago • 0 comments

Set the following config values

  1. phoenix.metadata.invalidate.cache.enabled = true
  2. phoenix.ddl.timestamp.validation.enabled = true
  3. phoenix.default.update.cache.frequency = Long.MAX_VALUE (NEVER)

Changes

  • Some places in code assume default UCF is 0 or 0L is hardcoded - replace that with default UCF property.
  • Tests which use ConnectionlessQueryServicesImpl - BaseConnectionlessQueryTest, its extensions and ParallelPhoenixConnectionFailureTest
    • This CQS implementation is used where no connection to hbase cluster is necessary. For such tests, we will not be able to make an admin call to get a list of region servers. Hence, we will keep the feature off for such tests.
  • UpdateCacheIT and UpdateCacheConnectionLevelPropIT
    • These classes have tests to verify client side cache updates and UCF functionality. PHOENIX-6883 redesigns metadata caching for both client and server. Some tests needed change in expected number of RPC calls.
  • Some queries like SELECT NEXT VALUE FOR SEQUENCE do not need timestamp validation.
    • Filter out table types when validating last_ddl_timestamps - only do it for (system&user)tables/views/indexes.
  • Creating an index on a table with the same client leads to an update of only the timestamp (not last_ddl_timestamp) in client cache and not the PTable object.
    • Client is not able to refresh its cache on subsequent queries because the reference in its cache has the latest timestamp so server does not send the latest PTable object.
    • One way to fix this - when we get a StaleMetadataCacheException, remove the entity in client cache and then force update.
  • Creating a view on a table with the same client increases the sequence number of the table on the server but client does not see it. Altering the table with the same connection leads to a ConcurrentMutationException.
    • Update cache in retry when dropping column.
  • Inherited View Index does not exist in SYSCAT so validating last_ddl_ts for them leads to TNFE.
    • Parse out the index name from the modified index name.

palashc avatar Jan 02 '24 22:01 palashc