confluentinc
compression defaults
In: compression.type and ksql.streams.producer.compression.type it says: "The default is snappy" but the default seems to be 'none' (unless explicitly set in server configuration file or set for the session).
I peeked at this when @alonpr mentioned it in Slack. I believe that the default value is coming from Apache Kafka.
@alonpr Thank you for the heads up! @jnh5y Just to confirm: the default, as seen by ksqlDB, is "none", and not "snappy"? It's not listed by SHOW PROPERTIES, so I'd guess it's "none".
compression.type=snappy in the CP 7.2 distribution, set in the ksql-server.properties config file.
Yes, I do believe that's correct. I do not see anything in the ksql codebase which is overriding the default.
Further, (I think that) the AK default is set here: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java#L352
If CP sets compression.type and/or ksql.streams.producer.compression.type, that certainly makes sense.
@colinhicks should we fix this as a docs bug or should we fix the code so that snappy is used by default? (Admittedly, we could backport a docs fix and fix the config going forward.)
@jnh5y @colinhicks I have a PR up if we decide we want to fix the docs first: https://github.com/confluentinc/ksql/pull/9208.
There is an existing ticket to unify these configs so that there is less confusion moving forward: https://github.com/confluentinc/ksql/issues/7351. I think we should not change any defaults until we tackle that.
I think another source of the confusion is that Confluent Cloud KSQL does in fact configure snappy (for both configs here), while other KSQL distributions do not specify the compression.