Unable to set Codec to Custom Type (pguint)

[ndbeals]

• asyncpg version: 0.20.0
• PostgreSQL version: 12.2
• Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: No, N/A, N/A
• Python version: 3.8.0
• Platform: Linux 64bit (Ubuntu 18.04)
• Do you use pgbouncer?: No
• Did you install asyncpg with pip?: Yes
• If you built asyncpg locally, which version of Cython did you use?: N/A
• Can the issue be reproduced under both asyncio and uvloop?: That's actually what I'm using.

I am using the user extension pguint (there's a pg11&12 branch) and it's installed fine. I'm trying to load ~100GB of article data onto my database, so I'm using the binary copy functions.

I created a binary encoder and decoder:

await con.set_type_codec( 'uint1', encoder=lambda x: struct.pack('!B',x), decoder=lambda x: struct.unpack('!B',x), format='binary')

But when I try to use it, I get:

Caught Insert Error.  UID:2419 EX: no binary input function available for type uint1

as my error. I tried delving into the source of this project but wasn't able to glean much. Does this error mean that the postgres extension itself is missing the required function on its end?

As a side note: I wrote a correct encoder/decoder (both text and binary) for the tsvector type, I want to submit a pull request but have no idea where I should put the new code, want to point me in the right direction?

also, should I be including the postgres varlena header in my binary encoding function outputs? or does your code do that?

[elprans]

no binary input function available for type uint1

This means that there is no binary I/O support for the type on the server side. Hence, you cannot use format='binary' and must use text I/O.

[elprans]

also, should I be including the postgres varlena header in my binary encoding function outputs? or does your code do that?

No, varlena is purely server-internal.

[elprans]

As a side note: I wrote a correct encoder/decoder (both text and binary) for the tsvector type, I want to submit a pull request but have no idea where I should put the new code, want to point me in the right direction?

Data codecs live in https://github.com/magicstack/py-pgproto, please open a PR there. Then, register the new codecs in init_tsearch_codecs() in asyncpg.

[saivarunk]

@elprans @ndbeals does asyncpg support tsvector type ?

[elprans]

There is I/O support, but only as text currently. The PR mentioned above did not make it to https://github.com/magicstack/py-pgproto yet.

[saivarunk]

@elprans I was trying to register a custom encoder/decoder for tsvector using set_type_codec. I'm facing this exception :

File "/Users/project/.venv/lib/python3.8/site-packages/asyncpg/connection.py", line 965, in set_type_codec raise ValueError('unknown type: {}.{}'.format(schema, typename)) ValueError: unknown type: public.tsvector

This is how I am trying to register the encoder/decoder pair : await con.set_type_codec( 'tsvector', encoder=encode_tsvector, decoder=decoe_tsvector, format='text' )

[elprans]

tsvector is a builtin type, so you want to pass a schema explicitly: schema='pg_catalog'

[saivarunk]

Thanks, @elprans, was able to fix the issue by changing the schema to the correct one.