Limit number of issues / PR retrieved on magit-status

[humitos]

Hi! This is not a bug or something that is not working properly, but a kind of question/feature request.

Is is possible to limit the number of issues / PR retrieved when magit-status is loaded? For example, with the RTD project (https://github.com/rtfd/readthedocs.org) it's very slow to open the magit-status buffer and it's very annoying.

• Recent commits (48) is 48 lines.
• Pull Request (26) is 274 lines.
• Issues (491) is 6557 lines.

I'm not sure, but I suppose that rendering all of those issues/pr takes a lot of time (not sure how to measure this properly, anyway)

I've being using this in other small projects and I never had this issue. Thanks!

[vermiculus]

It would definitely take a lot of time, and it's something I've been wanting to dig into as well. I'd like to do something similar to magit's commit log -- load more sections with a keybinding or perhaps even a section-button (like issue labels) -- but it hasn't been working out.

I've got plans to create a 'dashboard view' of a single repository (similar in structure to the current dashboard); would a link to this dashboard from the status buffer be a good stand-in for 'load the rest of the issues'?

[humitos]

something similar to magit's commit log -- load more sections with a keybinding or perhaps even a section-button (like issue labels) -- but it hasn't been working out.

This would be the best for me :) , but I have no idea what were your problems or if it's complicated or no. I'm not an elisp dev :(

I'm not following with regarding the dashboard view. Can you explain this a little more?

I'm thinking in something very simple to begin with and fix the issue: have a variable telling magithub how many PR and Issues it should show in the magit-status buffer.

Also, I don't understand why the latest issue retrieved by magithub is the 3233 when the lastone at the moment is 3277 (https://github.com/rtfd/readthedocs.org/issues).

[vermiculus]

Re your last question, have you refreshed GitHub data with C-u g?

[humitos]

Ups, sorry. With that hotkey I get all the new issues (I took like 2 minutes :P )

It seems that the cache was playing with me.

[vermiculus]

About a 'dashboard view', I'm thinking of something with a similar interface to the current dashboard, but dedicated to the GitHub-y parts of the project. Things like CI status might very well belong in the magit-status buffer, but listing issues and pull requests seems like too much info -- especially for larger projects.

This view still wouldn't have incremental loading of issues, but it would allow the magit-status sections to load only up to a certain number of issues (like you suggest) without losing the ability to see all the issues easily.

---

If we want the ability to 'load more issues', we could actually simplify it to a keybinding (+ looks like the current binding in magit-log-mode) and increase our variable's value in the current buffer before we refresh. It looks like this is how the commit-log works:

+ runs the command magit-log-double-commit-limit (found in
magit-log-mode-map), which is an interactive Lisp closure in
`~/dotfiles/.emacs.d/elpa/magit-20171116.427/magit-log.el'.

It is bound to +.

(magit-log-double-commit-limit)

Double the number of commits the current log buffer is limited to.

[humitos]

This view still wouldn't have incremental loading of issues, but it would allow the magit-status sections to load only up to a certain number of issues (like you suggest) without losing the ability to see all the issues easily.

😍, this would be great for me.

I'm sorry that I can't help you to code this, but I'm open to help you on testing or something else if I my knowledge allows me :)

On the other hand, if you can point me where I can change the limit of issues/pr as a hack for the current code I would really appreaciate because I didn't find it and I had to disable magithub. I was also thinking on creating a filter under ...

... after two hours of learning...

I found a solution that helps me to keep magithub enabled in RTD repository: filter the issues with a number bigger than 3000

(defun issue-number-greater-than (issue)
  (> (alist-get 'number issue) 3000))

(setq magithub-issue-issue-filter-functions '(issue-number-greater-than))

So, this will be enough for now to me :)

[vermiculus]

Heyyy, that works! I knew generalizing it would come in handy. You could also do a function to only have issues updated in the last two weeks or so. Here's a function definition that's coming soon in a push to develop:

(defun magithub--parse-time-string (iso8601)
  (require 'parse-time)
  (parse-iso8601-time-string (concat iso8601 "+00:00")))

You can define a filter function like this:

(defun issue-filter-to-days (days)
  (lambda (issue)
    (< (/ (time-to-seconds
           (time-since
            (magithub--parse-time-string
             (alist-get 'created_at issue))))
          86400)
       days)))

(add-to-list 'magithub-issue-issue-filter-functions
             (issue-filter-to-days 14))

That will filter display to only those issues updated in the last two weeks. Ideally, we'd only request issues updated in the last two weeks. That would be some work on the cache to remember this kind of information (i.e., when each cached form was updated.

[humitos]

That will filter display to only those issues updated in the last two weeks.

You are my hero! Want to be my friend? 😝 I'll definetly use something like that.

Ideally, we'd only request issues updated in the last two weeks

Yeah, I can image.

Thanks for the tip.

[PierreTechoueyres]

In case someone is interested here is some little improvements around the ideas of filters to mitigate the too long delay when there is too many PR or issues:

(with-eval-after-load 'magithub ;; magithub limit filters
  (require 'parse-time)

  (defmacro magithub--time-number-of-days-since-string (iso8601)
    `(time-to-number-of-days
      (time-since
       (parse-iso8601-time-string
	(concat ,iso8601 "+00:00")))))

  (defun issue-filter-to-days (days type)
    `(lambda (issue)
       (let ((created_at (magithub--time-number-of-days-since-string
			  (alist-get 'created_at issue)))
	     (updated_at (magithub--time-number-of-days-since-string
			  (alist-get 'updated_at issue))))
	 (or (< created_at ,days) (< updated_at ,days)))))
  
  (defun magithub-filter-maybe (&optional limit)
    "Add filters to magithub only if number of issues is greter than LIMIT."
    (let ((max-issues (length (magithub-issues)))
	  (max-pull-requests (length (magithub-pull-requests)))
	  (limit (or limit 15)))
      (when (> max-issues limit)
	(add-to-list (make-local-variable 'magithub-issue-issue-filter-functions)
		     (issue-filter-to-days limit "issues")))
      (when (> max-pull-requests limit)
	(add-to-list (make-local-variable 'magithub-issue-pull-request-filter-functions)
		     (issue-filter-to-days limit "pull-requests")))))

  (add-to-list 'magit-status-mode-hook #'magithub-filter-maybe))

This checks if there is more than 15 PR or issues. If so it adds filter to one or another to keep those who have been update or created in the past 15 days.

Maybe you could add an entry on the FAQ ?

[vermiculus]

@PierreTechoueyres It would at the very least be a good wiki page entry 😄

[PierreTechoueyres]

Hi @vermiculus, Of course without bugs this would be better ...

(with-eval-after-load 'magithub ;; magithub limit filters
  (require 'parse-time)

  (defmacro magithub--time-number-of-days-since-string (iso8601)
    `(time-to-number-of-days
      (time-since
       (parse-iso8601-time-string
	(concat ,iso8601 "+00:00")))))

  (defun issue-filter-to-days (days type)
    `(lambda (issue)
       (let ((created_at (magithub--time-number-of-days-since-string
			  (alist-get 'created_at issue)))
	     (updated_at (magithub--time-number-of-days-since-string
			  (alist-get 'updated_at issue))))
	 (or (< created_at ,days) (< updated_at ,days)))))

  (defun magithub-filter-maybe (&optional limit)
    "Add filters to magithub only if number of issues is greter than LIMIT."
    (let ((max-issues (length (ignore-errors (magithub-issues))))
	  (max-pull-requests (length (ignore-errors (magithub-pull-requests))))
	  (limit (or limit 15)))
      (when (> max-issues limit)
	(add-to-list (make-local-variable 'magithub-issue-issue-filter-functions)
		     (issue-filter-to-days limit "issues")))
      (when (> max-pull-requests limit)
	(add-to-list (make-local-variable 'magithub-issue-pull-request-filter-functions)
		     (issue-filter-to-days limit "pull-requests")))))

  (add-to-list 'magit-status-mode-hook #'magithub-filter-maybe))

Now, this doesn't throw errors when the repository isn't linked with an github one.

[humitos]

This works like a champ! Thanks @PierreTechoueyres!

One thing that I found is that pressing g over a PR or issue it causes

apply: Wrong number of arguments: (0 . 0), 1

and the *magit* buffer gets blank. Could this be related to this filter? Although, by removing this code I got the same problem.

[PierreTechoueyres]

Hello @humitos, I've provided an quick an dirty fix in in PR #186. To @vermiculus: I'm not really certain this is the right fix. I'm guessing it's almost the contrary : I'm just hidding an symptom.

[vermiculus]

This should be fixed now by 3725281fcd67fb5ff5d1a5b6a82767b136036fdd.

[PierreTechoueyres]

Hello @vermiculus, I think you've closed this issue by error. The initial problem is about filtering which I think isn't solved. Am I wrong ?

[vermiculus]

Oh bullocks, you're right. I was preoccupied with the error.

[iamriel]

@PierreTechoueyres Thank you very much! Works like a charm even in spacemacs :)