gitbucket icon indicating copy to clipboard operation
gitbucket copied to clipboard
verified publisher icon gitbucket

API request to `contents/` is slow

Open ziggystar opened this issue 5 years ago • 6 comments

Before submitting an issue to GitBucket I have first:

Issue

Impacted version: 4.35.3

Deployment mode: Tomcat

Problem description:

  • accessing the content API call /api/v3/repos/<group>/<repo>/contents with or without ?ref is slow for a large repository

I have a large repository (17k commits, 700MB on disk). The API call to contents/ to the root, which contains 49 objects (dirs and files) takes 10 seconds.

  • a call to a directory with 104 entries takes 32s
  • 30 entries takes 5s
  • 10 entries 2.5s

This also causes a problem with Jenkins, since within the log I see all those calls running into a time out.

I think this is a performance problem in gitbucket, because to me it appears that this call should perform much faster.

ziggystar avatar Feb 11 '21 09:02 ziggystar

I've found the MaxFiles and lowering it (to 10) solves the problem. Apparently the problem seems to be caused by the used git library, which appears to be rather slow.

As I understand, to compute the result for the contents API call, it is not necessary to use the complete information FileInfo provides. e.g. the last commit and the corresponding message are not needed.

In the resulting API response, only the file SHAs are missing, and these are not computed by the (presumably slow) call to JGitUtil.getFileList: List[JGitUtil.FileInfo].

ziggystar avatar Feb 11 '21 17:02 ziggystar

Thanks for investigation. Sounds like we can optimize it. I think we can deal with it in 4.36.0.

takezoe avatar Feb 14 '21 06:02 takezoe

@ziggystar I created https://github.com/gitbucket/gitbucket/pull/2686 to optimize the performance of Contents API, but I'm not sure how much effective it is. Could you try that branch if possible to compare the performance?

takezoe avatar Feb 21 '21 23:02 takezoe

@takezoe I can clone and compile the project. How can I run it?

I tried sbt run and sbt package for creating the war. But the JettyLauncher classes are not bundled with the project. I could not find any instructions about this. Best would be to run it from IDE, but I could not figure out how to do this.

ziggystar avatar Feb 23 '21 13:02 ziggystar

You can create an executable war file by sbt executable: https://github.com/gitbucket/gitbucket/blob/master/doc/build.md

takezoe avatar Feb 23 '21 16:02 takezoe

Same problem.

shinsenter avatar Aug 26 '21 06:08 shinsenter