Unable to deploy functions because referenced dependencies are not available

[alita-moore]

Describe the bug I can't share code between functions because the deploy script claims that it can't find files that are actually available locally. I assume it's some kind of docker context issue.

To Reproduce Steps to reproduce the behavior:

1. Create a supabase project
2. Create a folder at the same level as the supabase project
3. Create a function
4. Import from the folder you created earlier into the function

You can see an example file structure from this example

Expected behavior I expect the deploy to work and for the bundler to have access to the files referenced

Screenshots If applicable, add screenshots to help explain your problem.

System information Rerun the failing command with --create-ticket flag.

• Ticket ID: e0f757134382497ca76b840b2109ec03

• Version of OS: Ubuntu 23.10

• Version of CLI: 1.183.5

• Version of Docker: Docker version 27.0.3, build 7d4bcd8

• Versions of services: Using workdir /Users/alitamoore/workspaces/monorepo/infra/supabase-platform

   SERVICE IMAGE      │      LOCAL       │   LINKED
  

─────────────────────────┼──────────────────┼───────────── supabase/postgres │ 15.1.0.153 │ 15.1.0.153 supabase/gotrue │ v2.155.3 │ v2.155.3 postgrest/postgrest │ v12.0.1 │ v12.0.1 supabase/realtime │ v2.29.15 │ - supabase/storage-api │ v1.6.8 │ v1.6.8 supabase/edge-runtime │ v1.54.10 │ - supabase/studio │ 20240701-05dfbec │ - supabase/postgres-meta │ v0.83.2 │ - supabase/logflare │ 1.4.0 │ - supabase/supavisor │ 1.1.56 │ - darthsim/imgproxy │ v3.8.0 │ -

This is a major problem because it means I can't share code between functions which breaks a lot of use cases

[sweatybridge]

Thanks for creating a new issue. I will link the comment https://github.com/supabase/cli/issues/1584#issuecomment-2237923864 but let's continue the discussion here.

[alita-moore]

I feel like the ideal solution to this problem is to allow for workspace packages (i.e. using pnpm), but I think that this functionality is not yet available in deno. So I've been using this workaround instead (i.e. importing from a file that's outside of the function directory) in order to share code and standardize between projects.

[alita-moore]

Ah I just saw your comment, that's very exciting (using the import map instead). I will try it out!

Edit: I'm messing around with it, seems to be working! I'll close this ticket once I've refactored everything and re-deployed / verified it works.

[alita-moore]

@sweatybridge it seems that if you import files from files that are mapped in the import_map that those are not included.

For example, if I import a file example.ts with the following body:

// example.ts
import "./example2.ts"

Then my import_map.json looks like

{
"imports": {
  "example": "path/to/example.ts"
  }
}

where example2.ts is another file, then if I include the import_map.json to the example.ts file, it will give me an error claiming that example2.ts is not found. So it seems that including it in the import map includes on that single file and not its dependencies.

I haven't tried this when the import map maps to a .js file, though. Maybe that does include the imported files?

[sweatybridge]

it seems that if you import files from files that are mapped in the import_map that those are not included.

Yes, that's a limitation indeed. But you can workaround it by adding the whole directory to import_map.json. For example,

{
"imports": {
  "folder/": "../../some/folder",
  }
}

And in your code,

// example.ts
import "folder/example.ts"

[alita-moore]

but then I get an error like the following from the vscode deno:

Relative import path "@wovenlibs/deno-core/index.ts" not prefixed with / or ./ or ../ and not in import map from "file:///Users/alitamoore/workspaces/monorepo/infra/supabase-platform/supabase/functions/api-consumers/services/create-consumer.service.ts"deno(resolver-error)

Is it possible at all in the future to automatically detect the dependencies and include those files a well?

[alita-moore]

actually it seems that doing something like this works, but I admit it's not an ideal solution:

{
  "imports": {
    "@wovenlibs/deno-core": "../../../deno-core/src/index.ts",
    "__@wovenlibs/deno-core": "../../../deno-core/src"
}

with the expectation that __@... is not used

[alita-moore]

Okay, I can confirm that the above works as a workaround. However, without the ability for the script to pull referenced files it means that it won't be able to import from files that have dependencies. Is that by design or is that something you can add support for? Being unable to import other workspace packages from the imported files makes it harder to share code within a project.

[sweatybridge]

Hi @alita-moore, we have shipped an experimental flag for docker-less deploy to the beta release channel.

npx supabase@beta functions deploy --use-api

With this new approach, pulling referenced files from any code or import map should work. For eg.

my-repo/
├─ my-app/
│  ├─ supabase/
│  │  │  functions/
│  │  │  │  slug/
│  │  │  │  ├─ index.ts
│  │  │  │  ├─ deno.json
├─ my-lib/
│  ├─ src/
│  │  ├─ index.ts
├─ README.md

Given the directory layout above, you can import my-lib from either index.ts or deno.json.

[alita-moore]

Unfortunately it's still not working. Here's the output:

pnpx supabase@beta functions deploy --use-api        
WARN: no SMS provider is enabled. Disabling phone login
Uploading asset (sso): supabase/functions/sso/deno.json
Uploading asset (sso): supabase/functions/sso/index.ts
failed to read file: open ../../libs/core/dist/src/index.js: invalid argument
Try rerunning the command with --debug to troubleshoot the error.

and then my deno.json note that this configuration works with deno run

{
  "imports": {
    "@types/express": "npm:@types/express@^5.0.1",
    "axios": "npm:axios@^1.8.4",
    "express": "npm:express@^5.1.0",
    "@wovendomain/core": "../../../../../domain/core/dist/index.js",
    "@wovenlibs/core": "../../../../../libs/core/dist/src/index.js",
    "lodash": "npm:lodash@^4.17.21",
    "luxon": "npm:luxon@^3.6.1",
    "ramda": "npm:ramda@^0.30.1",
    "stacktrace-js": "npm:stacktrace-js@^2.0.2",
    "uuid": "npm:uuid@^11.1.0",
    "yup": "npm:yup@^1.6.1"
  }
}

by the way, you all should really add warnings to --debug that it outputs your authentication token. I uploaded the outputs of --debug so accidentally use made my supabase token public. I deleted the comment and invalidated the token but still, I think it's probably a bad idea to include sensitive information in --debug

[alita-moore]

note that pnpx supabase@beta functions deploy works, though :)

[sweatybridge]

Thanks @alita-moore for following up. I've reduced verbosity of debug logging and have a hunch of the issue with --use-api flag. I will try to fix it soon.

[alita-moore]

after further investigation on the updates to deno, it seems that they now support monorepos which is great news! However, it seems that supabase function deployments do not accurately resolve workspace dependencies. It works using deno run for example but supabase functions deploy fails. I think it's related to this issue, however, I've created a new issue to track it: https://github.com/supabase/supabase/issues/35021

I mention it here because using deno's built in monorepo support would be the optimal solution to sharing dependencies among projects. In this issue I manually input the path to the other dependencies, but that's cumbersome and causes problems when resolving non-deno packages / types. It would be really amazing if supabase functions could resolve and include these workspace packages!

[sweatybridge]

I've created a separate issue to track the --use-api deploy flag error when importing ../../libs. https://github.com/supabase/cli/issues/3467#issue-3007949643

It will probably take more time as the fix isn't trivial.