walrus icon indicating copy to clipboard operation
walrus copied to clipboard

Listening to a bpchar column returns only the first character

Open w3b6x9 opened this issue 3 years ago • 4 comments

Describe the bug

When listening to a column of type bpchar, only the first character is returned.

To Reproduce Steps to reproduce the behavior:

  1. CREATE TABLE test ( id char(3) primary key );
  2. create publication some_pub for table test;
  3. INSERT INTO test (id) VALUES ('abc');
  4. Run polling query
  5. For record I'm getting back id of just a when I'm expecting abc.

Expected behavior

I'm expecting to get back the entire record instead of just the first character.

Additional context

  • Related issue: https://github.com/supabase/realtime/issues/283

w3b6x9 avatar Aug 23 '22 19:08 w3b6x9

walrus does not currently consider type modifiers

Example Screen Shot 2022-08-23 at 3 23 16 PM

In all 3 cases, the data type (as far as walrus knows) is char

Unfortunately, due to Postgres' "coerce via truncate" behavior Screen Shot 2022-08-23 at 3 23 35 PM

the result is to silently drop all but the first character of the string.


short term fix:

replace instances of char(n) with varchar(n) because an unmodified varchar is variable length by default


long term fix:

Include modifiers during type casting. That will require some research wrt how to get the modifier efficiently (not available through current access methods) and use it in data casts.

olirice avatar Aug 23 '22 20:08 olirice

Are you suggesting Walrus won't be modified to overcome the deficit? I chose char(n) because it's more efficient than varchar(n) and because the desired field is always fixed length. I'd have to add a length constraint on the varchar just to accommodate what comes free with char.

If using varchar is just a short term thing, it's no concern. I can easily use varchar in development for a time knowing the permanent fix is coming.

By the way, thank you for the quick response to my issue.

mlanza avatar Aug 23 '22 20:08 mlanza

Are you suggesting Walrus won't be modified to overcome the deficit

walrus will be updated to solve this issue, the short term fix is an option for users who are currently experiencing the issue and can't wait for an upstream fix

I chose char(n) because it's more efficient than varchar(n)

This is no longer the case in Postgres. Here's a discussion that talks through it

I'd have to add a length constraint on the varchar

varchar supports an optional maximum length argument just like char so you could use varchar(3) to avoid any check constraints

--

even though varchar(3) doesn't have any downsides, we'll still figure out a way to support char(n) so no one else has to trip over the weird behavior. thanks for reporting!

olirice avatar Aug 23 '22 20:08 olirice

This is very helpful. Thank you for taking the time to make such a thorough reply.

mlanza avatar Aug 23 '22 20:08 mlanza