PyAthena

PyAthena copied to clipboard

Add missing fields to AthenaQueryExecution model (fix #364)

1

laughingman7743

https://boto3.amazonaws.com/v1/documentation/api/1.24.51/reference/services/athena.html#Athena.Client.get_query_execution https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryExecution.html ``` { 'QueryExecution': { 'QueryExecutionId': 'string', 'Query': 'string', 'StatementType': 'DDL'|'DML'|'UTILITY', 'ResultConfiguration': { 'OutputLocation': 'string', 'EncryptionConfiguration': { 'EncryptionOption': 'SSE_S3'|'SSE_KMS'|'CSE_KMS', 'KmsKey': 'string' }, 'ExpectedBucketOwner': 'string', 'AclConfiguration': { 'S3AclOption': 'BUCKET_OWNER_FULL_CONTROL' }...

laughingman7743

laughingman7743

Can't cast to VARBINARY

2

Hi, first of all: great project. I already created quite complex queries and everything works great, but it seems like there is a problem with the `VARBINARY` type mapping. If...

takacsd

https://github.com/laughingman7743/PyAthena/issues/272#issuecomment-1200464052

laughingman7743

Impl s3fs cursor

1

A cursor implementation to read CSV files in S3 without using Pandas. It would be good to be able to use `awsathena+s3fs` in SQLAlchemy. https://github.com/fsspec/s3fs https://docs.python.org/3/library/csv.html

laughingman7743

how to get QueryExecutionId from cursor when using sqlalchemy + pyathena

8

I am using sqlalchemy + pyathena Here is an example code: from urllib.parse import quote_plus from sqlalchemy import create_engine, inspect conn_str = "awsathena+rest://@athena.{region_name}.amazonaws.com:443/{schema_name}?s3_staging_dir={s3_staging_dir}" engine = create_engine(conn_str.format(region_name="us-east-1", schema_name="default", s3_staging_dir=quote_plus("s3://aws-athena-query-results-bucket/"))) conn =...

mdeshmu

https://github.com/sqlalchemy/sqlalchemy/blob/main/README.dialects.rst

laughingman7743

Documenting/improving memory behavior

2

I'm writing a package using PyAthena that might or might not have extremely large results sets, so I've been interested in memory usage. It seems that the default cursor is...

wumpus

SQLAlchemy + Pandas very slow when compared to AWS Wrangler

17

Tested same query that returns more than 2 million lines. AWS Wrangler takes 1m34s to return a DataFrame. SQLAlchemy + PyAthena takes 16m37s to return a DataFrame. [See attached notebook...

avibrazil