Always get "It seems that this CI build is not part of a merge request."

[lowtower]

Hallo,

following #22, I always get the same response from washingmachine:

Running after script... $ /root/washingmachine/washingmachine run -v --add-comments-in-commits It seems that this CI build is not part of a merge request

I have created some "merge requests" to generate a different answer without success.

I am using gitlab at gitlab.com. I guess it's gitlab in version 10.6 on their own servers.

[lowtower]

Hello David,

I guess the GITLAB_API_TOKEN was incorrect.

Nevertheless, now I Get the following error/notice:

It seems that this CI build is not part of a merge request.
Unable to find a previous build for this branch. Skipping adding comments inside the commit. Could not find finished job with build name "test:7.1" and stage "test" in pipeline "20816304"

I definitely merged a PR, so something else must be wrong. The tool doesn't find any finished job. What does that mean? Is there any requirement to have different stages, one with code-coverage and a different one with washingmachine?

[moufmouf]

Is there any requirement to have different stages, one with code-coverage and a different one with washingmachine?

No there is definitely no such thing.

For this to work, you need to create the pull request before the build is passing. My advice would be to start by creating the pull request, and then pushing your code to trigger the build. If you are still getting the same message, could you give me more information?

For instance: did you create the pull request in the same repository or is it part of a fork? (I'm doing most of my tests in the same repository, so if you have the issue in a fork, could you try to create a PR in the same repository and tell me if it works? That would help me pinpoint the problem)

[lowtower]

Thanks for Your reply. Did as You recommended - always in the same repo, but merge request from different branch. Now I get something like the following, even after third, fourth, ... commit to source branch:

$ /root/washingmachine/washingmachine run -v --add-comments-in-commits --open-issue
[warning] We could not find a build for commit 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715.
Unable to find a previous build for this branch. Skipping adding comments inside the commit. Could not find finished job with build name "test:7.1" and stage "test" in pipeline "20879313"

[moufmouf]

Ok, this is weird. There is most definitely a bug somewhere. The message "We could not find a build for commit XXX" should not be displayed on the second or third commit.

Could you increase verbosity and send me the output?

$ /root/washingmachine/washingmachine run -vvv --add-comments-in-commits --open-issue

[lowtower]

$ /root/washingmachine/washingmachine run -vvv --add-comments-in-commits --open-issue
[debug] Called API, got 20 merge requests
Pipeline current commit: f044cb54781a1d0cb7a447fa991c86c995a606f6
Target branch: test-ci
Target commit: 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715
Last common commit: 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715
[debug] Looking for latest pipeline for commit 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715
[debug] Analysing 118 pipelines to find pipeline for commit 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715
[debug] Found pipeline 20853938 for commit 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715
[warning] We could not find a build for commit 2f3006a1fe66f99b58c39f1dcb3b97b0c5481715.
[debug] Looking for latest pipeline for commit f044cb54781a1d0cb7a447fa991c86c995a606f6
[debug] Analysing 118 pipelines to find pipeline for commit f044cb54781a1d0cb7a447fa991c86c995a606f6
[debug] Found pipeline 20892498 for commit f044cb54781a1d0cb7a447fa991c86c995a606f6
Unable to find a previous build for this branch. Skipping adding comments inside the commit. Could not find finished job with build name "test:7.1" and stage "test" in pipeline "20892498"

[lowtower]

Looks okay as far as I can tell.

[lowtower]

I took a different branch as target. Now, I get something Like:

$ /root/washingmachine/washingmachine run -vvv --open-issue --add-comments-in-commits
[debug] Called API, got 20 merge requests
Pipeline current commit: 9497a23d74e016318a42b0586f0694ea99f45cc1
Target branch: test_washingmachine
Target commit: 3d73070d08d238ea9f541335b6cd8b652d203bc8
Last common commit: 3d73070d08d238ea9f541335b6cd8b652d203bc8
[debug] Looking for latest pipeline for commit 3d73070d08d238ea9f541335b6cd8b652d203bc8
[debug] Analysing 260 pipelines to find pipeline for commit 3d73070d08d238ea9f541335b6cd8b652d203bc8
[debug] Found pipeline 20816210 for commit 3d73070d08d238ea9f541335b6cd8b652d203bc8

In RunCommand.php line 322:

[RuntimeException] 
Invalid ZIP archive /tmp/artgBgbbg.zip 


Exception trace:
TheCodingMachine\WashingMachine\Commands\RunCommand->getMeasuresFromCommit() at /root/washingmachine/src/Commands/RunCommand.php:195
TheCodingMachine\WashingMachine\Commands\RunCommand->execute() at /root/washingmachine/vendor/symfony/console/Command/Command.php:252
Symfony\Component\Console\Command\Command->run() at /root/washingmachine/vendor/symfony/console/Application.php:918
Symfony\Component\Console\Application->doRunCommand() at /root/washingmachine/vendor/symfony/console/Application.php:222
Symfony\Component\Console\Application->doRun() at /root/washingmachine/vendor/symfony/console/Application.php:129
Symfony\Component\Console\Application->run() at /root/washingmachine/washingmachine:31

run [-c|--clover CLOVER] [-j|--crap4j CRAP4J] [-u|--gitlab-url GITLAB-URL] [-t|--gitlab-api-token GITLAB-API-TOKEN] [-p|--gitlab-project-name GITLAB-PROJECT-NAME] [-r|--commit-sha COMMIT-SHA] [-b|--gitlab-job-id GITLAB-JOB-ID] [-a|--gitlab-build-name GITLAB-BUILD-NAME] [-e|--gitlab-pipeline-id GITLAB-PIPELINE-ID] [-s|--job-stage JOB-STAGE] [-f|--file FILE] [-i|--open-issue] [--add-comments-in-commits]

[lowtower]

I have forked washingmachine from github to gitlab. Maybe, I have setup something essentially wrong. Have a look at the following job: https://gitlab.com/lowtower/washingmachine/-/jobs/64352444

[moufmouf]

Ok, thanks for the logs!

It seems there is an issue with unzipping the artifact. Looking at the code of the washingmachine, I realize the displayed error message might not be correct. I just made some changes on the washingmachine to have a better error message (https://github.com/thecodingmachine/washingmachine/pull/24/files). Could you simply run again the build and send me the logs? The new logs might be more explicit (I hope!)

[lowtower]

Hello David, thank for looking at it! How can I create a project from a specific commit? I tried the following, without success

composer create-project thecodingmachine/washingmachine washingmachine 2.0#e3cafe14ea02a81c91893c9274d49c25e3674a4d

[lowtower]

Hello David, found out that I have to specify the packagist-name and not the branch name, then it works:

- composer create-project --stability dev thecodingmachine/washingmachine washingmachine 2.0.x-dev#2e3df6e3d17628092a0a665db4f25f9410d08125

This is the output (PR, JOB)

$ /root/washingmachine/washingmachine run -vvv --add-comments-in-commits --open-issue
[debug] Called API, got 2 merge requests
Pipeline current commit: 7a69011af78ef887c80074c3f06f09aec9e4deac
Target branch: test_ci
Target commit: b1cb0c0d7f286792fd70565c9412256941904679
Last common commit: b1cb0c0d7f286792fd70565c9412256941904679
[debug] Looking for latest pipeline for commit b1cb0c0d7f286792fd70565c9412256941904679
[debug] Analysing 29 pipelines to find pipeline for commit b1cb0c0d7f286792fd70565c9412256941904679
[debug] Found pipeline 20924422 for commit b1cb0c0d7f286792fd70565c9412256941904679

Warning: ZipArchive::getStatusString(): Invalid or uninitialized Zip object in /root/washingmachine/src/Commands/RunCommand.php on line 323

In RunCommand.php line 323:
                                                                               
  [RuntimeException]                                                           
  An error occurred while unzipping artifact from commit b1cb0c0d7f286792fd70  
  565c9412256941904679. Error 19:                                              
                                                                               

Exception trace:
 TheCodingMachine\WashingMachine\Commands\RunCommand->getMeasuresFromCommit() at /root/washingmachine/src/Commands/RunCommand.php:195
 TheCodingMachine\WashingMachine\Commands\RunCommand->execute() at /root/washingmachine/vendor/symfony/console/Command/Command.php:252
 Symfony\Component\Console\Command\Command->run() at /root/washingmachine/vendor/symfony/console/Application.php:918
 Symfony\Component\Console\Application->doRunCommand() at /root/washingmachine/vendor/symfony/console/Application.php:222
 Symfony\Component\Console\Application->doRun() at /root/washingmachine/vendor/symfony/console/Application.php:129
 Symfony\Component\Console\Application->run() at /root/washingmachine/washingmachine:31

run [-c|--clover CLOVER] [-j|--crap4j CRAP4J] [-u|--gitlab-url GITLAB-URL] [-t|--gitlab-api-token GITLAB-API-TOKEN] [-p|--gitlab-project-name GITLAB-PROJECT-NAME] [-r|--commit-sha COMMIT-SHA] [-b|--gitlab-job-id GITLAB-JOB-ID] [-a|--gitlab-build-name GITLAB-BUILD-NAME] [-e|--gitlab-pipeline-id GITLAB-PIPELINE-ID] [-s|--job-stage JOB-STAGE] [-f|--file FILE] [-i|--open-issue] [--add-comments-in-commits]

[lowtower]

Hello David,

is there anything else I can do to track down the problem? I have already upgraded php-gitlab-api to version 9.9 with no effect.

Obviously, the zip-File is not downloaded correctly. In the RunCommand line 319 the artifact is dumped from $buildService, which is an instance of TheCodingMachine\WashingMachine\Gitlab\BuildService. There is a remark "// Call seems broken". I have checked the calls to $this->client->jobs->artifactsByRefName() and $this->client->jobs->artifacts() in Gitlab/Api/Jobs and both seem to be correct calls to the Gitlab api.

Gitlab/Api/Jobs.php:

    /**
     * @param int|string $project_id
     * @param int $job_id
     * @return StreamInterface
     */
    public function artifacts($project_id, $job_id)
    {
        return $this->getAsResponse("projects/".$this->encodePath($project_id)."/jobs/".$this->encodePath($job_id)."/artifacts")->getBody();
    }
    /**
     * @param int|string $project_id
     * @param string $ref_name
     * @param string $job_name
     * @return StreamInterface
     */
    public function artifactsByRefName($project_id, $ref_name, $job_name)
    {
        return $this->getAsResponse("projects/".$this->encodePath($project_id)."/jobs/artifacts/".$this->encodePath($ref_name)."/download", array(
            'job' => $job_name
        ))->getBody();
    }

Gitlab api:

GET /projects/:id/jobs/:job_id/artifacts
GET /projects/:id/jobs/artifacts/:ref_name/download?job=name

Cheers, Lowtower.

[moufmouf]

Hey @lowtower ,

Sorry for the delay.

You are right, the ZIP file is not downloaded correctly. I've looked into the PHP-gitlab-api code and error handling seems correct there...

Looking it the error message you get, I see:

[RuntimeException]                                                           
  An error occurred while unzipping artifact from commit b1cb0c0d7f286792fd70  
  565c9412256941904679. Error 19:          

Error 19 is "ZIPARCHIVE::ER_NOZIP" which means the archive you get is not a valid ZIP file....

I'll try to update the issue this week, maybe release a new version to check that there is actually something in the ZIP file (it might be empty...)

[lowtower]

Hello @moufmouf, could that be a configuration issue? I don't keep the artifacts for ever, but at least two weeks or so. Mmh, the same happens with "fresh" PR's.

[lowtower]

Any news on this?

[lowtower]

Hello @moufmouf,

I would really like to help, but I have no idea how. It would be nice, if You could guide me to debug the issue.

Thanks in advance, Lowtower

[moufmouf]

Hey @lowtower

Sorry for the lack of answer...

In your logs, you should see something like:

[RuntimeException]
An error occurred while unzipping artifact from commit b1cb0c0d7f286792fd7565c9412256941904679. Error 19:

In Gitlab UI, could you:

• check the Gitlab CI Job for commit "b1cb0c0d7f286792fd7565c9412256941904679" (or whatever number you get)
• look if there are any artifact
• if there is no artifact, please tell me (the issue might be that the artifact has expired)
• try to download the artifact and check that this is a valid Zip file

[lowtower]

Hello David,

i created some new Merge requests to be sure to have the artifacts. When merging into an arbitrary branch it doesn't work, but when merging into master it works fine (now?). I have no restriction on branch master in my .gitlab-ci.yml. Is that a restriction on gitlab.com or in washingmachine?

Thanks for Your help, Cheers, Lowtower.

[lowtower]

• check the Gitlab CI Job for commit "b1cb0c0d7f286792fd7565c9412256941904679" (or whatever number you get)
• look if there are any artifact

I have tried several things on gitlab.com and cannot confirm, that it works when pulling against a Master-branch!

Artifacts are created for jobs, not commits nor pipelines, right?

I see, that the job ids per pipeline are retrieved in Gitlab\BuildService.php. Then all jobs are checked against $job['name'], $job['stage'] and $job['status'], which should be fine. Another check could be added to either $job['coverage'] != null and/or isset($job['artifacts_file']). Also dumping the job id into the logs would be helpful to see if the correct job was found. Created a PR25 for it. Also, the Error message "An error occurred while unzipping artifact from commit ..." is misleading as it is an artifact from a job instead!

With these additions, maybe then, I can dig a bit deeper!

I checked existence of artifacts.zip file!

Cheers, LowTower.

[lowtower]

Hello David,

another try to track down everything from the beginning:

• The results from the merge job
• project-ID: 5891986
• pipeline-ID: 27148758
• job-ID: 86731755
• I checked the list of jobs with the api
• I checked the availability of the artifacts file with the api and it is a valid zip file.
• all seems well, but still I get the error message:

PHP Warning:  ZipArchive::getStatusString(): Invalid or uninitialized Zip object in /root/washingmachine/src/Commands/RunCommand.php on [line 323](https://github.com/thecodingmachine/washingmachine/blob/e1b89c12d72a75ebb38cc2dde9100734d3db1d80/src/Commands/RunCommand.php#L319)`.

In the BuildService the artifacts are retrieved from PHP-Gilab-API\Jobs In PHP-Gilab-API\Jobs the call to the api seems to be correct. Maybe the problem lies in the StreamWrapper.

Don't know, how to check that! Any suggestions?

[lowtower]

Hello David,

I tried a lot of debugging using $this->logger->debug() and dumping all sorts of objects and variables, etc. washingmachine sometimes seem to work and then it doesn't. I cannot find a reason.

What I find strange is the fact that in BuildService.php Line 197 the artifacts are retrieved and when I debug $artifactContent, I get something like

[debug] This resource has been moved temporarily to https://gitlab-artifacts.s3.amazonaws.com/1e/77/1e778c68cf11b7cdf100d86b3fbeb68c0f21d80cc6f6c4901a8604024f482fe5/2018_08_08/87813326/74923464/artifacts.zip?X-Amz-Expires=600&X-Amz-Date=20180808T165106Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJTOFWQ3GL4O3Q3FA/20180808/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=71127017da8e6a8cdf12eccabeaab0596cee42e7cc84f7a604b464d30cff5e0c.

When I debug a var_dump, I get something like

class GuzzleHttp\Psr7\Stream#219 (7) {
  private $stream =>
  resource(212) of type (stream)
  private $size =>
  NULL
  private $seekable =>
  bool(true)
  private $readable =>
  bool(true)
  private $writable =>
  bool(true)
  private $uri =>
  string(10) "php://temp"
  private $customMetadata =>
  array(0) {
  }
}

What I stumble upon is the wording temporarily

Nevertheless, the problem is that sometimes the $result in RunCommand.php is true and sometimes not (instead an integer - error code from \ZipArchive->open()). In my case $result equals 19, what means

#define ZIP_ER_NOZIP 19 /* N Not a zip archive */

In that case, the method getStatusString() called in the error message doesn't make sense.

Currently, it seems to work properly! :confused:

[moufmouf]

Oooooooooooh! Got it!

You are using Gitlab.com and I'm using a self-hosted Gitlab. That's why I never encountered the problem.

On gitlab.com, it seems the artifacts are sometimes hosted on Amazon S3 (and sometimes not... :cry: )

I googled "This resource has been moved temporarily to" and here is what I found:

https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5168

So it seems that when downloading the artifact, we get a HTTP 302 response (302 = moved temporarily) :smile:

So what we need to do know is to submit a PR to m4tthumphrey/php-gitlab-api since this lib is the one responsible for fetching the artifacts!

[moufmouf]

Just opened the PR: https://github.com/m4tthumphrey/php-gitlab-api/pull/345

[lowtower]

Great,

thanks a lot :smiley: