databricks-sql-cli icon indicating copy to clipboard operation
databricks-sql-cli copied to clipboard

Project is abandoned, please warn users this is not supported.

Open mahiki opened this issue 11 months ago • 4 comments

README should have a warning that no issues have been addressed since 2023 and the project is abandoned.

There are other CLI options, please suggest some at the top of the README:

mahiki avatar Feb 19 '25 17:02 mahiki

You just saved me some time. Thanks for the links, too.

I wonder if the linkage/description from this page should be updated too: https://docs.databricks.com/aws/en/dev-tools/databricks-sql-cli

u35253 avatar Feb 19 '25 23:02 u35253

What are the best replacements for databricks-sql-cli, if it is truly abandoned? Maybe some good options can be mentioned in the comments here.

Regarding Harlequin: not a replacement for databricks-sql-cli

I would state that, in general, Harlequin is definitively not a suitable replacement for databricks-sql-cli at this time (and not for the forseeable future).

  • Current: Harlequin does not support STDIN and STDOUT (e.g., basic terminal based usage), but is instead a "TUI/IDE". In other words, there is no -e <statement> or -f <file> option, to submit statements and direct output to STDOUT, bypassing the whole "text IDE" experience, for normal CLI tool use cases (e.g., shell scripting).
  • When asked, the creator/maintainer of Harlequin stated that that basic input/output outside of the IDE interface is "a totally different set of features than what I'm focused on".

Regarding: usql: does it actually work with Databricks?

AFAICT, usql seems to not connect correctly with Databricks personal access tokens by using the databricks-sql-go DSN format that appears suggested.

  • The parsing seems quite mangled and just plain wrong in my testing.
  • There are also no real examples of a full usql config for Databricks on the usql or the related dburl Github pages.
  • The usql possibility seems promising/hopeful, but requires additional debugging and investigation from where I sit. (At least databricks-sql-cli and harlequin can connect to Databricks and are easily configured using hostname, personal access token, and HTTP path, unlike usql AFAICT so far -- despite what the README pages indicate.)

Again, I appreciate the tips, and I hope that solid options become well documented.

u35253 avatar Feb 20 '25 23:02 u35253

Following up: usql does seem to work with databricks after all

The usql config issue for Databricks mentioned in the immediately preceding comment seems to be a non-issue.

  • There was just some nuance to saving configuring the format correctly (it seems that the correct [array syntax](https://github.com/xo/usql/blob/2be774d5fd7c660d8f7477f647cec585a3b111bb/contrib/config.yaml#L32C1-L33C1 was already documented and appears to work).
  • Thus, regarding usql as a possible alternative tool, the connection string seems to work, and results are returned.
  • Details are in a separate (and resolved) usql issue.

u35253 avatar Feb 21 '25 03:02 u35253

Good news on usql.

Like you mentioned, the databricks connection works only while using a config file. I find that usql is actively maintained, and matches up well with psql (which I consider an industry standard-setter).

Here is a working example:

# ~/.usql_config.yaml
connections:
  mycorp:
    - databricks
    - token:[email protected]:443/sql/1.0/warehouses/8675309abcdefXYZ


# basic connection
usql --config ~/.usql_config.yaml mycorp

# my preferred data pager, and editor
PAGER=pspg EDITOR=vim usql --config ~/.usql_config.yaml mycorp

You can use the \edit metacommand in usql to edit the last command, and there are other useful REPL commands for saving output to file etc.

mahiki avatar Feb 22 '25 04:02 mahiki