`cf security-groups` is extremely slow on large systems

[bkrannich]

Issue

Running a cf security-groups on a large Cloud Foundry system is extremely slow. For 13.500 security groups, the runtime is > 3 hours. I would hope that this can be way quicker given that cloud controller already has a security_group table containing the entries in question.

Context

see above

Steps to Reproduce

Run cf security-groups in a large CF system.

Expected result

Runtime in the minutes range at best.

Current result

Runtime > 3 hours.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/165179334

The labels on this github issue will be updated when the story is started.

[gberche-orange]

@bkrannich I also reported this same problem in the related V3 API proposal for future V3 performance enhancement. Might be a good place to look at currently proposed V3 endpoint and provide feedback

[weymanf]

It appears that CAPI returns rather quickly when calling the cf security-groups. The real slowdown is when cf cli acts as an admin user. It would use the default per-page result which is 50, and the cf security-groups would try to walk through 201 pages to find 10000 security-groups.

@bkrannich Can you create this issue in https://github.com/cloudfoundry/cli?

[Gerg]

I'll transfer this issue to the CLI repo.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/176041626

The labels on this github issue will be updated when the story is started.

[weymanf]

I have tested running queries in our mysql database in bosh-lite. It returns in seconds with all 13000 fake records.

I have also tested cf curl /v3/security_groups but it looks like we limit the max page results.

I've ran cf security-groups and it's definitely slow ~45 seconds. It is slow at 10000+ security groups because we use the default per page and walk through 201 pages with 50 results each.

Looks like we can't safely bump up the walk limit per page number because users can configure the max limit https://github.com/cloudfoundry/capi-release/blob/6387524550d336afd40de43152a7b1abbfc25921/jobs/cloud_controller_ng/spec#L898