datafusion-comet

datafusion-comet copied to clipboard

feat: Add random row generator in data generator

1

## Which issue does this PR close? Closes #. ## Rationale for this change Follow up of #426, supports generating random rows for specified struct type. ## What changes are...

advancedxy

feat: add hex scalar function

1

## Which issue does this PR close? Related to https://github.com/apache/datafusion-comet/issues/341. ## Rationale for this change I recently added `unhex` so this PR adds `hex`. It's another scalar function that isn't...

tshauck

build: Add spark-4.0 profile and shims

7

## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/372 ## Rationale for this change To be ready for Spark 4.0 ## What changes are included in this PR? This...

kazuyukitanimura

Minor: Generate the supported Spark builtin expression list into MD file

1

## Which issue does this PR close? Follow up on #331 Closes #282 Related #240 ## Rationale for this change Generate the supported Spark builtin expression list into MD file...

comphead

PR build for Linux Java 11 with Spark 3.4 is not running

4

### Describe the bug The github action PR build for Linux Java 11 with Spark 3.4 is not running ### Steps to reproduce Looks like the variable substitution is not...

kazuyukitanimura

feat: correlation support

2

## Which issue does this PR close? Supports `CORR` The implementation mostly is the same as the DataFusion's implementation. The reason we have our own implementation is that DataFusion has...

huaxingao

feat: Add HashJoin support for BuildRight

1

## Which issue does this PR close? Closes #390. ## Rationale for this change ## What changes are included in this PR? ## How are these changes tested?

viirya

Comet doesn't support Spark BroadcastHashJoinExec if it is null-aware anti-join

1

### What is the problem the feature request solves? DataFusion HashJoin LeftAnti doesn't support null-aware anti join. See https://github.com/apache/datafusion/issues/10583 ### Describe the potential solution _No response_ ### Additional context _No...

viirya

build: bump spark version to 3.4.3

1

## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## How are these changes tested?

huaxingao

Cast String to Date ANSI Mode - Spark 3.2 - Mismatch between Spark and Comet Errors

2

### Describe the bug When a String which is an invalid date is cast to a Datetype **In spark 3.2 the error message is** ``` org.apache.spark.SparkException: Job aborted due to...

vidyasankarv