`sign` fails with 404 after deleting and re-signing with the same extension ID

[cadorn]

Is this a feature request or a bug?

When using sign the package is built and uploaded, passes validation but when downloading again from AMO it gets a 404. There appears to be a bug in the re-packaging process?

What is the current behavior?

Gets a 404 when downloading signed XPI from AMO.

From https://github.com/mozilla/web-ext/issues/1111#issuecomment-407384817, here are steps to reproduce

1. Create a new extension
2. Use web-ext to sign it
3. Delete the extension from DevHub
4. Create a new version of the extension (with the same ID)
5. Try to sign it with web-ext.

What is the expected or desired behavior?

Should get 200 with XPI content.

Version information (for bug reports)

web-ext: 1.10.0

Here is an example:

Validating add-on [...........................................................................................................]
Validation results: https://addons.mozilla.org/en-US/developers/upload/165388899c0e4dea92247ef6eb9cdfef
Downloading signed files: ...
/dl/source/github.com~pinf-it~it.pinf.org.mozilla.web-ext/node_modules/sign-addon/dist/webpack:/src/amo-client.js:283
              throw new Error(
                    ^
Error: Got a 404 response when downloading https://addons.mozilla.org/api/v3/file/745574/firephp_for_firefox_devtools-0.1.0pre_1508200790-fx.xpi?src=api
    at Request.<anonymous> (/dl/source/github.com~pinf-it~it.pinf.org.mozilla.web-ext/node_modules/sign-addon/dist/webpack:/src/amo-client.js:283:21)

1.10.1 fails with:

Validating extension:

_const.MESSAGE_TYPES.includes is not a function

lint: TypeError: _const.MESSAGE_TYPES.includes is not a function
    at Message.set (/dl/source/github.com~pinf-it~it.pinf.org.mozilla.web-ext/node_modules/addons-linter/dist/webpack:/src/message.js:52:24)

[kumar303]

Sorry about running into this. I think this is a dupe of https://github.com/mozilla/web-ext/issues/680 . You could test it by waiting a couple minutes and then checking the URL to see if it's still a 404.

[cadorn]

@kumar303 Can web-ext download it for me after the fact or do you have some NodeJS code crafted that can do the api auth as an example?

[kumar303]

I believe the final URL to the file is not auth-protected so you can just download it

[cadorn]

@kumar303 It is protected:

$ curl https://addons.mozilla.org/api/v3/file/745574/firephp_for_firefox_devtools-0.1.0pre_1508200790-fx.xpi?src=api
{"detail":"Authentication credentials were not provided."}

[kumar303]

Can you consistently reproduce a 404? If so, can you attach a log with --verbose ?

[kumar303]

I'll re-open until we figure out if it's really a dupe or not.

[cadorn]

Reproducible 100% of the time. Verbose log: https://gist.github.com/cadorn/e7c8f32106d6a91834de860b11f241b5

[kumar303]

Thanks for the verbose log. I checked the validation results and it was successful so you can probably also find a working link to the file by signing into the DevHub. Sorry that this happens 100% of the time for you -- it must be due to the geographic region you're connecting from. From this evidence it definitely looks to be a dupe of https://github.com/mozilla/web-ext/issues/680

[cadorn]

@kumar303 The validation result shows but there is no release/download page I can find in the DevHub. If you can send me the download link I should get when logged into the DevHub I can try it. I think the signing process is stalling on the server and not completing. It is not just slow.

So at this point I am still at a loss. You say the download link should not be authenticated yet it is. I do not know how to sign a request to allow the download and I cannot access the downloads from the DevHub so self-publishing of extensions is broken for me.

I need this to start offering previews of new WebExtension to my users before 57 is released.

[kumar303]

there is no release/download page I can find in the DevHub

Oh really? I'll re-open based on that comment.

I don't have access to view your add-on but for one of my unlisted add-ons I can click Manage status & versions then click on the version number and I arrive at this screen which lets me download the signed add-on. Do you not see this? Does your add-on show up in My Submissions at all?

You could make an authorized request to the download link with a script like this. However, if you can't see your add-on in the DevHub then something else is wrong.

var jwt = require('jsonwebtoken');

var key = 'user:...';
var secret = '634...';

var issuedAt = Math.floor(Date.now() / 1000);
var payload = {
  iss: key,
  jti: Math.random().toString(),
  iat: issuedAt,
  exp: issuedAt + 60,
};

var token = jwt.sign(payload, secret, {
  algorithm: 'HS256',  // HMAC-SHA256 signing algorithm
});

process.stdout.write('JWT ' + token)

Then you could use it with:

curl -H "Authorization: $(node ~/path/to/make-api-jwt.js)" https://addons.mozilla.org/api/v3/...

[cadorn]

@kumar303 I may have found the issue. It appears that signing works for new extensions. Then when I delete the extension from the DevHub and re-sign it it errors out.

So it appears that the DevHub delete process is not thorough enough or the prior usage of an ID is not validated soon enough.

[kumar303]

Aha. Maybe we don't handle that case well enough. Are you also saying the original extension that you were trying to sign had been deleted in the DevHub?

[cadorn]

Are you also saying the original extension that you were trying to sign had been deleted in the DevHub?

Correct. I had signed it and then deleted it. Then tried to sign it again.

[gapan]

Same problem here.

Steps to reproduce:

1. Create a new extension
2. Use web-ext to sign it
3. Delete the extension from DevHub
4. Create a new version of the extension
5. Try to sign it with web-ext. It fails with the same error message as @cadorn

I worked around it by changing the extension applications/id in the manifest.json file slightly.

[kumar303]

Hi @gapan, sorry for the delayed reply and thanks for the detailed steps. I filed an API bug here: https://github.com/mozilla/addons/issues/5980

I elaborated on your steps with some details. Can you confirm that the steps are accurate?

[gapan]

Hi @kumar303 , yes they are accurate. Thanks!

[diox]

@gapan and @cadorn, 2 questions for you to help us debug this further (I don't have the permissions to check that for myself):

• Does your add-on has listed versions or only unlisted versions?
• Can you see in the devhub "Manage Status & Versions" page for your add-on the section "Listing visibility" at the top ? If so, does it say your add-on is visible, invisible or something else ?

Thanks.

[gapan]

Mine has only unlisted versions. To be exact, it only has one unlisted version.

There is no "Listing visibility" section in the "Manage Status & Versions" page that I can see. There is a question mark right next to the "Latest version" string, that displays a balloon help message with the text "This is the newest uploaded unlisted version".

[diox]

Can you still reproduce ? We've landed a change in prod that might affect this. If you can still reproduce I'll investigate further.

[cadorn]

I hope to get to checking this again in the near future. Thanks for the potential fix!

[gapan]

Unfortunately, I can still reproduce the problem.

[embbnux]

Got this error after delete extension from DevHub as @gapan . Any update about this?

[embbnux]

After using different applications.gecko.id, it is fixed