BITMAKER-1932: API control max number of simultaneous jobs

[mgonnav]

Description

Control from the API how many jobs are running simultaneously and scale this number according to the current scale of our job manager engine (for Kubernetes, the number of available nodes).

Issue

• https://tasks.bitmaker.dev/issues/1932

Checklist before requesting a review

• [X] I have performed a self-review of my code.
• [X] My code follows the style guidelines of this project.
• [X] I have made corresponding changes to the documentation.
• [X] New and existing tests pass locally with my changes.
• [X] If this change is a core feature, I have added thorough tests.
• [X] If this change affects or depends on the behavior of other estela repositories, I have created pull requests with the relevant changes in the affected repositories. Please, refer to our official documentation.
• [X] I understand that my pull request may be closed if it becomes obvious or I did not perform all of the steps above.

[mgonnav]

@joaquingx This is an extra feature to control the number of jobs to launch in Kubernetes based on the current cluster size. The idea is that if we can keep the number of jobs close to the limit, it will help Kubernetes autoscale better when it detects this situation. This PR needs further testing since the method get_scale_size() may not work on all platforms.