Allow emulator writeable `publicUrl` with `PUT`, like a writeable `getSignedURL`

[brianmhunt]

We use signed URLs to create writeable destinations for things like caching and intermediary steps. We would like to run this functionality in isolation so we can test it in CI and develop locally (and specifically without service account credentials). This does not seem to currently be possible with the Firebase Emulator.

We use the live production API to produce a PUT URL like this:

   async function generatePutUrl (bucketID: string, path: string) {
    const options = {
      version: 'v4',
      action: 'write',
      expires: Date.now() + 60 * 60 * 1000, // 1 hour
    } as const

    const [url] = await new Storage()
      .bucket(bucketID)
      .file(path)
      .getSignedUrl(options)
  } 

When using the Firestore Emulator we'd like to be able to also write to the emulator bucket. According #3400 this isn't currently supported, however the comment at the bottom that suggests using .publicUrl() is a workaround for readable URLs.

This issue would be solved if the return value from the .publicUrl() accepted a PUT by the emulator that uploaded the file. Currently the emulator returns "Not Implemented", but I suspect honouring PUT would be fairly straightforward.

To distinguish this from #3400, that issue would be resolved by supporting signed URLs. This issue would be resolved by using publicUrl instead of signed URLs (i.e. this issue feels much simpler).

[taeold]

@brianmhunt Thanks for a thorough writeup. We are currently very shortstaffed on the storage emulators, but I'll try to get the attention of folks who might be more informed than myself.

[brianmhunt]

Thanks @taeold, very much appreciated.

[yuchenshi]

Hi, thanks for filing this feature request! We are unable to promise any timeline for this, but if others also have this request, adding a +1 on this issue can help us prioritize adding this to the roadmap.

(Googler-only internal tracking bug: b/222161599)

[thenikso]

+1

[AchrafBn]

+1

[tonyjhuang]

Hi all thanks for the interest in this feature request. As this is part of the GCS API and not explicitly the Firebase Storage API, this likely won't be prioritized soon. However, we do welcome all PRs and the team would be happy to give reviews to anyone who would like to take a stab at this issue.

[xhuberdeau]

Hello,

I know this thread is 3 years old, but I stumbled upon the same problem and I think I have a working (and maybe hackish) solution. My context is:

• my nextJS backend need to generate 2 signed URLs:
  • one for writing an image (uploadUrl)
  • one for displaying this image once uploaded (publicUrl)
• my frontend would call the previous endpoint to
  • first upload an image (uploadUrl) in a form
  • then display the thumbnail of it (publicUrl)

Indeed the getSignedUrl function does not work with the emulator. When digging into this, I used the emulator UI to see if I could replicate what's behind the upload file button inside the storage emulator. To be able to create a file on a bucket you need to:

• do a POST request
• the URL should be http://127.0.0.1:9098/v0/b/your-bucket-name/o?yourFileName.extension
• you need to pass a hardcoded authorization header: Firebase owner
• pass your file as binary data

Curl snippet:

curl --location 'http://127.0.0.1:9098/v0/b/your-bucket-name/o?name=image.jpg' \
--header 'Authorization: Firebase owner' \
--header 'Content-Type: image/jpeg' \
--data-binary '@/path/to/your/image/image.jpg'

Here is a throwable snippet of what the backend code could look like:

  const {fileName, fileType} = controllerInput;
  const bucketName = 'your-bucket-name';
  const bucket = storage.bucket(bucketName);
  const file = bucket.file(fileName);

  if (isLocalEnv) {
    const searchParams = new URLSearchParams();
    searchParams.append('name', fileName);
    const uploadUrl = `http://127.0.0.1:9098/v0/b/${bucketName}/o?${searchParams.toString()}`;

    return {uploadUrl, uploadAuthorizationHeader: 'Firebase owner', publicUrl: file.publicUrl()};
  } else {
    const expirationDate = Date.now() + 15 * 60 * 1000; // 15 min expiry;
    const [uploadUrl] = await file.getSignedUrl({
      action: 'write',
      expires: expirationDate,
      contentType: fileType,
    });

    const [publicUrl] = await file.getSignedUrl({
      action: 'read',
      expires: expirationDate,
      contentType: fileType,
    });

    return {uploadUrl, publicUrl};
  }

Then my frontend (React) can do something like

const useUploadFile = () => {
  const {mutateAsync} = trpc.fileUploader.uploadFile.useMutation();

  const uploadFile = async (file: File) => {
    const {publicUrl, uploadUrl, uploadAuthorizationHeader} = await mutateAsync({
      fileName: file.name,
      fileType: file.type,
    });
    const headers: AxiosRequestConfig['headers'] = {
      'Content-Type': file.type,
    };
    if (uploadAuthorizationHeader !== undefined) {
      headers.Authorization = uploadAuthorizationHeader;
    }

    await axios.post(uploadUrl, file, {
      headers,
    });

    return publicUrl;
  };

  return uploadFile;
};

Hope this helps

[kwameopareasiedu]

@xhuberdeau You're a life-saver. Thanks for this...

[kwameopareasiedu]

@xhuberdeau The emulator upload works, but regardless of Content-Type header value, the uploaded file has a type of application/octet-stream. Did u come across this issue...??