ODBC error 50311 source encoding UTF-8 using ICU using TD ODBC 16.20

[gilessbrown]

Seeing this error with Teradata ODBC 16.20 but not 16.10. Not sure if this is a bug in the underlying library or maybe just an incompatibility?

File "download_from_teradata.py", line 71, in main
    for row in res:
  File "./lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 868, in __iter__
    row = self.fetchone()
  File "./lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1195, in fetchone
    self.cursor, self.context)
  File "./lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "./lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "./lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
    raise value.with_traceback(tb)
  File "./lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1186, in fetchone
    row = self._fetchone_impl()
  File "./lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1067, in _fetchone_impl
    return self.cursor.fetchone()
  File "./lib/python3.6/site-packages/teradata/util.py", line 121, in fetchone
    return next(self, None)
  File "./lib/python3.6/site-packages/teradata/util.py", line 163, in __next__
    values = next(self.iterator)
  File "./lib/python3.6/site-packages/teradata/tdodbc.py", line 1152, in rowIterator
    checkStatus(rc, hStmt=cursor.hStmt, method="SQLFetch")
  File "./lib/python3.6/site-packages/teradata/tdodbc.py", line 231, in checkStatus
    raise DatabaseError(i[2], u"[{}] {}".format(i[0], msg), i[0])
sqlalchemy.exc.DatabaseError: (teradata.api.DatabaseError) (50311, '[HY000] [Teradata][Support] (50311) Error converting invalid input with source encoding UTF-8 using ICU. Rejected bytes began with: 80.') (Background on this error at: http://sqlalche.me/e/4xp6)

[rina-shilo]

same error when using driver ODBC 16.10 and when using python driver teradatasql:

File "/usr/local/lib/python3.7/site-packages/teradata/udaexec.py", line 836, in fetchall return self.cursor.fetchall() File "/usr/local/lib/python3.7/site-packages/teradata/util.py", line 139, in fetchall for row in self: File "/usr/local/lib/python3.7/site-packages/teradata/util.py", line 163, in next values = next(self.iterator) File "/usr/local/lib/python3.7/site-packages/teradata/tdodbc.py", line 1152, in rowIterator checkStatus(rc, hStmt=cursor.hStmt, method="SQLFetch") File "/usr/local/lib/python3.7/site-packages/teradata/tdodbc.py", line 231, in checkStatus raise DatabaseError(i[2], u"[{}] {}".format(i[0], msg), i[0]) teradata.api.DatabaseError: (50311, '[HY000] [Teradata][Support] (50311) Error converting invalid input with source encoding UTF-8 using ICU. Rejected bytes began with: D64C.')

[sushmit86]

Has there been any resolution to this? I think this is an issue in Mac/Linux like environment

[yvan]

anyone working on or thinking about this? should i just downgrade my driver to 16.10? or 15.*?

my error:

Error: ('HY000', '[HY000] [Teradata][Support] (50311) Error converting invalid input with source encoding UTF-8 using ICU. Rejected bytes began with: E474. (50311) (SQLGetData)')

my environment:

centos 7 tdodbc1620-16.20.00.49-1.noarch.rpm (downloaded tdodbc1620__linux_indep.16.20.00.49-1.tar.gz) installed with yum.

[yvan]

One workaround is just to use teradatasql for python:

pip install teradatasql==16.20.0.39
conn = teradatasql.connect(None, host=host, user=user, password=password)

i do not have any encoding issue with this version of the driver.

[mylocalaichat]

Solved this by adding

Charset=UTF8

in odbc.ini file. I figured that, there is no need to translate for my application and the default was ASCII

[yvan]

can anyone confirm when/if this makes it to the next release? :)

[pberden]

I solved this by passing the charset option like so:

session = udaExec.connect(method="odbc", system="server", username="user", password="pass", charset="UTF8")

[KriAga]

@pberden thanks, worked for me!