cli icon indicating copy to clipboard operation
cli copied to clipboard
verified publisher icon supabase

`supabase db diff --linked` fails to spin up shadow db on MacOs

Open gkossakowski opened this issue 7 months ago • 4 comments

Describe the bug supabase db diff --linked is stuck spinning up shadow db when I run this command on MacOs (on MBP-M4)

To Reproduce Steps to reproduce the behavior:

  1. create a new project with https://vercel.com/templates/next.js/supabase
  2. spin up a new db on Supabase Cloud
  3. link that db to the project - there's no local db in this setup)
  4. try the db diff command

Expected behavior I get the actual diff

Screenshots Instead of a screenshot, log from a command line might be helpful:

❯ supabase db diff --debug --linked
Using connection pooler: postgresql://postgres.essfjhuxdhpcrhrdmfgm:[YOUR-PASSWORD]@aws-0-eu-central-1.pooler.supabase.com:6543/postgres
Supabase CLI 2.23.4
Connecting to remote database...
2025/06/11 10:01:08 PG Send: {"Type":"StartupMessage","ProtocolVersion":196608,"Parameters":{"database":"postgres","user":"postgres.essfjhuxdhpcrhrdmfgm"}}
2025/06/11 10:01:08 PG Recv: {"Type":"AuthenticationSASL","AuthMechanisms":["SCRAM-SHA-256"]}
2025/06/11 10:01:08 PG Send: {"Type":"SASLInitialResponse","AuthMechanism":"SCRAM-SHA-256","Data":"n,,n=,r=gWFSZaOOqO2bj9O6P+1PV648"}
2025/06/11 10:01:08 PG Recv: {"Type":"AuthenticationSASLContinue","Data":"r=gWFSZaOOqO2bj9O6P+1PV648RUF4eGVtbmxBeFRlNEh1UWxpeUtVSUlFY29KRWdnPT0=,s=Rer3M0Ct9AhQXa63wdrIDA==,i=4096"}
2025/06/11 10:01:08 PG Send: {"Type":"SASLResponse","Data":"c=biws,r=gWFSZaOOqO2bj9O6P+1PV648RUF4eGVtbmxBeFRlNEh1UWxpeUtVSUlFY29KRWdnPT0=,p=8w0S7rNQUN830vY8Zd+Skxsf9wtQOGXeiY7fzAWSj/w="}
2025/06/11 10:01:08 PG Recv: {"Type":"AuthenticationSASLFinal","Data":"v=E1kt5omXpbA4UcZMJ6xMdyIKFHqUyzAiKlktcJfGxWs="}
2025/06/11 10:01:08 PG Recv: {"Type":"AuthenticationOK"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"DateStyle","Value":"ISO, MDY"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"IntervalStyle","Value":"postgres"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"TimeZone","Value":"UTC"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"application_name","Value":"Supavisor"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"client_encoding","Value":"UTF8"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"default_transaction_read_only","Value":"off"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"in_hot_standby","Value":"off"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"integer_datetimes","Value":"on"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"is_superuser","Value":"off"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"server_encoding","Value":"UTF8"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"server_version","Value":"15.8"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"session_authorization","Value":"postgres"}
2025/06/11 10:01:08 PG Recv: {"Type":"ParameterStatus","Name":"standard_conforming_strings","Value":"on"}
2025/06/11 10:01:08 PG Recv: {"Type":"BackendKeyData","ProcessID":641327723,"SecretKey":1627363492}
2025/06/11 10:01:08 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
2025/06/11 10:01:08 PG Send: {"Type":"Parse","Name":"lrupsc_1_0","Query":"-- List user defined schemas, excluding\n--  Extension created schemas\n--  Supabase managed schemas\nselect pn.nspname\nfrom pg_namespace pn\nleft join pg_depend pd\n  on pd.objid = pn.oid\njoin pg_roles r \n  on pn.nspowner = r.oid\nwhere pd.deptype is null\n  and not pn.nspname like any($1)\n  and r.rolname != 'supabase_admin'\norder by pn.nspname\n","ParameterOIDs":null}
2025/06/11 10:01:08 PG Send: {"Type":"Describe","ObjectType":"S","Name":"lrupsc_1_0"}
2025/06/11 10:01:08 PG Send: {"Type":"Sync"}
2025/06/11 10:01:09 PG Recv: {"Type":"ParseComplete"}
2025/06/11 10:01:09 PG Recv: {"Type":"ParameterDescription","ParameterOIDs":[1009]}
2025/06/11 10:01:09 PG Recv: {"Type":"RowDescription","Fields":[{"Name":"nspname","TableOID":2615,"TableAttributeNumber":2,"DataTypeOID":19,"DataTypeSize":64,"TypeModifier":-1,"Format":0}]}
2025/06/11 10:01:09 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
2025/06/11 10:01:09 PG Send: {"Type":"Bind","DestinationPortal":"","PreparedStatement":"lrupsc_1_0","ParameterFormatCodes":[1],"Parameters":[{"binary":"0000000100000000000000190000000b0000000100000013696e666f726d6174696f6e5c5f736368656d610000000570675c5f250000000b5c5f616e616c79746963730000000a5c5f7265616c74696d650000000b5c5f737570617669736f72000000097067626f756e6365720000000470676d71000000087067736f6469756d000000057067746c650000001473757061626173655c5f6d6967726174696f6e73000000057661756c74"}],"ResultFormatCodes":[1]}
2025/06/11 10:01:09 PG Send: {"Type":"Describe","ObjectType":"P","Name":""}
2025/06/11 10:01:09 PG Send: {"Type":"Execute","Portal":"","MaxRows":0}
2025/06/11 10:01:09 PG Send: {"Type":"Sync"}
2025/06/11 10:01:09 PG Recv: {"Type":"BindComplete"}
2025/06/11 10:01:09 PG Recv: {"Type":"RowDescription","Fields":[{"Name":"nspname","TableOID":2615,"TableAttributeNumber":2,"DataTypeOID":19,"DataTypeSize":64,"TypeModifier":-1,"Format":1}]}
2025/06/11 10:01:09 PG Recv: {"Type":"DataRow","Values":[{"text":"extensions"}]}
2025/06/11 10:01:09 PG Recv: {"Type":"DataRow","Values":[{"text":"public"}]}
2025/06/11 10:01:09 PG Recv: {"Type":"CommandComplete","CommandTag":"SELECT 2"}
2025/06/11 10:01:09 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
Creating shadow database...
2025/06/11 10:01:09 PG Send: {"Type":"Terminate"}
663325bcbfd3d291f9d96ef84e270152ded3df2980d1886022f04bc68b7d4362 container logs:
failed to read docker logs: Error response from daemon: No such container: 663325bcbfd3d291f9d96ef84e270152ded3df2980d1886022f04bc68b7d4362
Failed to remove container: 663325bcbfd3d291f9d96ef84e270152ded3df2980d1886022f04bc68b7d4362 Error response from daemon: No such container: 663325bcbfd3d291f9d96ef84e270152ded3df2980d1886022f04bc68b7d4362
failed to inspect container health: Error response from daemon: No such container: 663325bcbfd3d291f9d96ef84e270152ded3df2980d1886022f04bc68b7d4362

System information Rerun the failing command with --create-ticket flag.

--create-ticket has no effect on my command:

❯ supabase db diff --debug --linked --create-ticket
Using connection pooler: postgresql://postgres.essfjhuxdhpcrhrdmfgm:[YOUR-PASSWORD]@aws-0-eu-central-1.pooler.supabase.com:6543/postgres
Supabase CLI 2.23.4
Connecting to remote database...
2025/06/11 10:10:23 PG Send: {"Type":"StartupMessage","ProtocolVersion":196608,"Parameters":{"database":"postgres","user":"postgres.essfjhuxdhpcrhrdmfgm"}}

Additional context

This is happening on fresh install of docker, with latest MacOS. I reproduced this on my machine and my friend's machine as well (MacOS, MBP M1).

gkossakowski avatar Jun 11 '25 08:06 gkossakowski

If anyone is looking at this ticket wondering about a workaround: I rolled a manual migration process that emulates "declarative schemas" from Supabase, but instead of using supabase diff --linked, it's handled by prompting LLMs in Cursor to handcraft raw SQL and execute it with psql. It might make you cry a little at first sight, but it works for us—for now.

gkossakowski avatar Jun 11 '25 08:06 gkossakowski

Also using a Mac M1 here. Started with a barebones app, ran supabase init, supabase start, added simple foobar.sql schema and then npx supabase db diff -f create_foobar_table --debug. Hangs with exactly the same output (different container hash though).

bvanderdrift avatar Jul 02 '25 13:07 bvanderdrift

Dumping on this.

When running supabase db pull --schema auth or supabase db diff --linked on a Mac with Apple Silicon (M3), the command fails during the "Creating shadow database..." step. The process hangs for a moment before exiting with a Error response from daemon: No such container error.

This bug appears to be specific to the interaction with the protected auth schema, as running supabase db pull on other schemas (like public) works correctly. The issue is reproducible with both brew and pnpm (and perhaps npm) installations of the CLI.

To Reproduce

Steps to reproduce the behavior:

  1. Environment:

    • Use a Mac with Apple Silicon M3 (though I guess from other commenters and OP this also happens on other versions)
    • Ensure Docker Desktop is installed and running.
    • Install the latest Supabase CLI (e.g., brew install supabase/tap/supabase or pnpm i -g supabase).

  2. Remote Project Setup:

    • Create a new project on the Supabase cloud dashboard.
    • In the remote project's SQL Editor, add a simple trigger to the auth.users table to ensure there are custom changes to pull. 
      -- Create a public.profiles table
create table public.profiles (
  id uuid references auth.users not null primary key,
  email text
);
-- Create a trigger function
create function public.handle_new_user()
returns trigger as $$
begin
  insert into public.profiles (id, email)
  values (new.id, new.email);
  return new;
end;
$$ language plpgsql security definer;
-- Create the trigger
create trigger on_auth_user_created
  after insert on auth.users
  for each row execute procedure public.handle_new_user();

  3. Local Project Setup:

    • On your local machine, create a new directory: mkdir test-project && cd test-project
    • Initialize a new Supabase project: supabase init
    • Link to the remote project: supabase link --project-ref <your-project-ref>

  4. Trigger the Bug:

    • Ensure the local Supabase stack is not running to avoid conflicts: supabase stop
    • Attempt to pull the schemas, including auth: supabase db pull --schema public,auth --debug

  5. Control Test (Shows Normal Behavior):

    • Run the same command without the auth schema: supabase db pull --schema public --debug. Observe that this command succeeds.

Expected Behavior

The CLI should successfully create the shadow database, diff the schemas, and generate a new migration file in supabase/migrations/ containing the custom trigger and function from the remote auth schema.

Actual Behavior

The CLI prints Creating shadow database... and hangs for approximately 30-60 seconds. It then fails and no migration file is created.

> pnpm supabase db pull --schema auth,storage --debug

Output:

Loading project ref from file: supabase/.temp/project-ref
Using connection pooler: postgresql://postgres.kkrirxnvxmubveetdwgt:[YOUR-PASSWORD]@aws-0-eu-central-1.pooler.supabase.com:6543/postgres
Supabase CLI 2.39.2
Connecting to remote database...
2025/09/05 17:19:34 PG Send: {"Type":"StartupMessage","ProtocolVersion":196608,"Parameters":{"database":"postgres","user":"postgres.kkrirxnvxmubveetdwgt"}}
2025/09/05 17:19:34 PG Recv: {"Type":"AuthenticationSASL","AuthMechanisms":["SCRAM-SHA-256"]}
2025/09/05 17:19:34 PG Send: {"Type":"SASLInitialResponse","AuthMechanism":"SCRAM-SHA-256","Data":"n,,n=,r=6GK117Wne2uo2Y0uVdjYrHpU"}
2025/09/05 17:19:34 PG Recv: {"Type":"AuthenticationSASLContinue","Data":"r=6GK117Wne2uo2Y0uVdjYrHpURU5DK0dveHpvRVNoRjQzU0l6dERiQ2tLRWhqMWd3PT0=,s=kXH8fh3huUOI0jXVpTUyfg==,i=4096"}
2025/09/05 17:19:34 PG Send: {"Type":"SASLResponse","Data":"c=biws,r=6GK117Wne2uo2Y0uVdjYrHpURU5DK0dveHpvRVNoRjQzU0l6dERiQ2tLRWhqMWd3PT0=,p=fkpvAetdgccUbEbQzEIBs2sNCztcMg+H/TbTDfCPm0g="}
2025/09/05 17:19:35 PG Recv: {"Type":"AuthenticationSASLFinal","Data":"v=yMjlGu4Wq3GJIS8pQcuPi1cALATGsxiAeIUpy8PcskE="}
2025/09/05 17:19:35 PG Recv: {"Type":"AuthenticationOK"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"DateStyle","Value":"ISO, MDY"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"IntervalStyle","Value":"postgres"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"TimeZone","Value":"UTC"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"application_name","Value":"Supavisor"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"client_encoding","Value":"UTF8"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"default_transaction_read_only","Value":"off"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"in_hot_standby","Value":"off"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"integer_datetimes","Value":"on"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"is_superuser","Value":"off"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"server_encoding","Value":"UTF8"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"server_version","Value":"15.8"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"session_authorization","Value":"postgres"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterStatus","Name":"standard_conforming_strings","Value":"on"}
2025/09/05 17:19:35 PG Recv: {"Type":"BackendKeyData","ProcessID":2512082609,"SecretKey":2504612688}
2025/09/05 17:19:35 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
2025/09/05 17:19:35 PG Send: {"Type":"Parse","Name":"lrupsc_1_0","Query":"SELECT version FROM supabase_migrations.schema_migrations ORDER BY version","ParameterOIDs":null}
2025/09/05 17:19:35 PG Send: {"Type":"Describe","ObjectType":"S","Name":"lrupsc_1_0"}
2025/09/05 17:19:35 PG Send: {"Type":"Sync"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParseComplete"}
2025/09/05 17:19:35 PG Recv: {"Type":"ParameterDescription","ParameterOIDs":[]}
2025/09/05 17:19:35 PG Recv: {"Type":"RowDescription","Fields":[{"Name":"version","TableOID":317681,"TableAttributeNumber":1,"DataTypeOID":25,"DataTypeSize":-1,"TypeModifier":-1,"Format":0}]}
2025/09/05 17:19:35 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
2025/09/05 17:19:35 PG Send: {"Type":"Bind","DestinationPortal":"","PreparedStatement":"lrupsc_1_0","ParameterFormatCodes":null,"Parameters":[],"ResultFormatCodes":[0]}
2025/09/05 17:19:35 PG Send: {"Type":"Describe","ObjectType":"P","Name":""}
2025/09/05 17:19:35 PG Send: {"Type":"Execute","Portal":"","MaxRows":0}
2025/09/05 17:19:35 PG Send: {"Type":"Sync"}
2025/09/05 17:19:35 PG Recv: {"Type":"BindComplete"}
2025/09/05 17:19:35 PG Recv: {"Type":"RowDescription","Fields":[{"Name":"version","TableOID":317681,"TableAttributeNumber":1,"DataTypeOID":25,"DataTypeSize":-1,"TypeModifier":-1,"Format":0}]}
2025/09/05 17:19:35 PG Recv: {"Type":"DataRow","Values":[{"text":"20250905114417"}]}
2025/09/05 17:19:35 PG Recv: {"Type":"CommandComplete","CommandTag":"SELECT 1"}
2025/09/05 17:19:35 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
Creating shadow database...
2025/09/05 17:20:35 PG Recv: {"Type":"ParameterStatus","Name":"application_name","Value":"Supavisor"}
2025/09/05 17:21:35 PG Recv: {"Type":"ParameterStatus","Name":"application_name","Value":"Supavisor"}
cfe069e675ef20adc8623926456abb66dd7ca79d28b94c44e9b50f054821d70d container logs:
failed to read docker logs: Error response from daemon: No such container: cfe069e675ef20adc8623926456abb66dd7ca79d28b94c44e9b50f054821d70d
Failed to remove container: cfe069e675ef20adc8623926456abb66dd7ca79d28b94c44e9b50f054821d70d Error response from daemon: No such container: cfe069e675ef20adc8623926456abb66dd7ca79d28b94c44e9b50f054821d70d
2025/09/05 17:21:36 PG Send: {"Type":"Terminate"}
failed to inspect container health: Error response from daemon: No such container: cfe069e675ef20adc8623926456abb66dd7ca79d28b94c44e9b50f054821d70d

System Information

  • Model: MacBook Air (13-inch, M3, 2024)
  • Chip: Apple M3
  • Memory: 16 GB
  • macOS: macOS Sequoia 15.5
  • Docker Desktop Version: Docker version 28.1.1, build 4eba377
  • Supabase CLI Version: 2.39.2
  • Installation Method: Brew and pnpm/npm

ViaAnthroposBenevolentia avatar Sep 05 '25 12:09 ViaAnthroposBenevolentia

Hi, I have tried this on Macbook M3 pro (16-inch, 2024, running on Sequoia) and its working fine for me, can you guys please try it again?

thealish avatar Nov 15 '25 18:11 thealish