PyHive icon indicating copy to clipboard operation
PyHive copied to clipboard

presto - timestamps and decimal(38,18) returned as strings?

Open tooptoop4 opened this issue 6 years ago • 2 comments

Why are presto timestamp/decimal(38,18) data types returned a string (enclosed in u'') instead of python datetime/numeric types?

#presto jdbc: select typeof(col1),typeof(col2),typeof(col3),typeof(col4),typeof(col5),typeof(col6) from hive.x.y #result is timestamp timestamp bigint decimal(38,18) varchar varchar

desc hive.x.y #result is for_dt timestamp NO NO NO NO 1 for_d timestamp NO NO NO NO 2 for_h bigint NO NO NO NO 3 value decimal(38,18) NO NO NO NO 4 metric varchar(2147483647) NO NO NO NO 5 lat_lon varchar(2147483647) NO NO NO NO 6

attempt 1 #python from sqlalchemy.engine import create_engine engine = create_engine('presto://u:p@host:port',connect_args={'protocol': 'https', 'requests_kwargs': {'verify': 'mypem'}}) result = engine.execute('select * from hive.x.y limit 1') print(result.fetchall()) #result is [(u'2010-02-18 03:00:00.000', u'2010-02-18 00:00:00.000', 3, u'-0.191912651062011660', u'hey', u'there')]

attempt 2 #python from pyhive import presto import requests from requests.auth import HTTPBasicAuth

req_kw = { 'verify': 'mypem', 'auth': HTTPBasicAuth('u', 'p') }

cursor = presto.connect( host='host', port=port, protocol='https', username='u', requests_kwargs=req_kw, ).cursor()

query = '''select * from x.y limit 1''' cursor.execute(query) print cursor.fetchall() #result is [(u'2010-02-18 03:00:00.000', u'2010-02-18 00:00:00.000', 3, u'-0.191912651062011660', u'hey', u'there')]

tooptoop4 avatar Jun 18 '19 11:06 tooptoop4

@tooptoop4 we are not actively working on the library at the moment, however contributions are welcome.

bkyryliuk avatar Apr 16 '20 22:04 bkyryliuk

I find this is a limitation to both pyhive and the official python-presto-client. I find the need to cast it afterwards.

RJ3 avatar May 26 '20 19:05 RJ3