dbldatagen

dbldatagen copied to clipboard

Problem with column which is named "ID"

3

## Expected Behavior Generation of column with name "ID". ## Current Behavior Exception: `AnalysisException: Reference 'ID' is ambiguous, could be: ID, ID.` ## Steps to Reproduce (for bugs) ``` import...

SemyonSinchenko

Generating text with baseColumn not consistent

4

## Expected Behavior When generating any data, if `baseColumn` is set to a reference column in `withColumn` then the data generated for the new column should be the same when...

ericfeunekes

Error in creating ArrayType cols

2

## Expected Behavior When creating a column from existing schema or new that is of a composite type such as an Array of integers the expected behaviour is to have...

danielm-db

## Expected Behavior Time intervals can be specified as "12 minutes, 2 seconds". You can also specify "1 minute, 2 seconds". You should be able to specify "1 minute 1...

ronanstokes-db

## Expected Behavior ## Current Behavior ## Steps to Reproduce (for bugs) ## Context ## Your Environment * `dbldatagen` version used: * Databricks Runtime version: * Cloud environment used:

ronanstokes-db

Add support for Spark 3.2

1

## Expected Behavior Add explicit support for Spark 3.2 (included in Databricks runtime 9.1) ## Current Behavior The current versions of the framework work in Databricks 9.1 (which is based...

ronanstokes-db

Uninstalling and reinstalling wheel on cluster running DBR 8.3 may fail

1

If you have a named cluster specification in your Databricks environment and it had the current or a previous build of the datagenerator installed, when you uninstall the library and...

ronanstokes-db

Enhancement: Generate standard data sets

3

## Enhancement: Generate standard data sets It would be useful to be able generate standard data sets without having to define columns etc for quick demos and benchmarking of different...

ronanstokes-db

Generate data based on estimated Delta table size

2

As I have been using the data generator I have had to use trial and error to get a table size I require. Not sure if this is feasible but...

tfayyaz

Some users have requested the ability to generate 100 GB or 1 TB of data without specifying the number of rows

ronanstokes-db