Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto'

[mungewell]

Attempting to run the example given here (without extra bracket ;-): https://sqlite-utils.datasette.io/en/stable/python-api.html#reading-rows-from-a-file

from sqlite_utils.utils import rows_from_file
import io

rows, format = rows_from_file(io.StringIO("id,name\n1,Cleo"))
print(list(rows), format)
# Outputs [{'id': '1', 'name': 'Cleo'}] Format.CSV

Gives error

>"c:\Program Files\Python37\python.exe" test2.py
Traceback (most recent call last):
  File "test2.py", line 4, in <module>
    rows, format = rows_from_file(io.StringIO("id,name\n1,Cleo"))
  File "C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main\sqlite_utils\utils.py", line 300, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
AttributeError: '_io.StringIO' object has no attribute 'readinto'

I am running Python on Windows.

>"c:\Program Files\Python37\python.exe"
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

[mungewell]

Attempted to test on a machine with a new version of Python, but install failed with an error message for the 'click' package.

C:\WINDOWS\system32>"c:\Program Files\Python310\python.exe"
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()

C:\WINDOWS\system32>cd C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main

C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main>"c:\Program Files\Python310\python.exe" setup.py install
running install
running bdist_egg
running egg_info

...

Installed c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg
Searching for click
Downloading https://files.pythonhosted.org/packages/3d/da/f3bbf30f7e71d881585d598f67f4424b2cc4c68f39849542e81183218017/click-default-group-wheel-1.2.2.tar.gz#sha256=e90da42d92c03e88a12ed0c0b69c8a29afb5d36e3dc8d29c423ba4219e6d7747
Best match: click default-group-wheel-1.2.2
Processing click-default-group-wheel-1.2.2.tar.gz
Writing C:\Users\swood\AppData\Local\Temp\easy_install-aiaj0_eh\click-default-group-wheel-1.2.2\setup.cfg
Running click-default-group-wheel-1.2.2\setup.py -q bdist_egg --dist-dir C:\Users\swood\AppData\Local\Temp\easy_install-aiaj0_eh\click-default-group-wheel-1.2.2\egg-dist-tmp-z61a4h8n
zip_safe flag not set; analyzing archive contents...
removing 'c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg' (and everything under it)
Copying click_default_group_wheel-1.2.2-py3.10.egg to c:\program files\python310\lib\site-packages
click-default-group-wheel 1.2.2 is already the active version in easy-install.pth

Installed c:\program files\python310\lib\site-packages\click_default_group_wheel-1.2.2-py3.10.egg
error: The 'click' distribution was not found and is required by click-default-group-wheel, sqlite-utils

[mungewell]

log.txt

[simonw]

This is very strange. GitHub Actions CI here runs against Windows and installs OK.

Marking this as "Help wanted" to see if anyone can figure out what's going on here.

[mcarpenter]

The Windows aspect is a red herring: OP's sample above produces the same error on Linux. (Though I don't know what's going on with the CI).

The same error can also be obtained by passing an io from a file opened in non-binary mode ('r' as opposed to 'rb') to rows_from_file(). This is how I got here.

The fix for my case is easy: open the file in mode 'rb'. The analagous fix for OP's problem also works: use BytesIO in place of StringIO.

Minimal test case (derived from utils.py):

import io
from typing import cast

#fp = io.StringIO("id,name\n1,Cleo") # error
fp = io.BytesIO(bytes("id,name\n1,Cleo", encoding='utf-8')) # okay
reader = io.BufferedReader(cast(io.RawIOBase, fp))
reader.peek(1) # exception thrown here

I see the signature of rows_from_file() correctly has fp: BinaryIO but I guess you'd need either a runtime type check for that (not all ios have mode()), or to catch the AttributeError on peek() to produce a better error for users. Neither option is ideal.

Some thoughts on testing binary-ness of ios in this SO question: https://stackoverflow.com/questions/44584829/how-to-determine-if-file-is-opened-in-binary-or-text-mode

[simonw]

This is being handled in:

• #520