cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard
verified publisher icon apache

CPU to Memory weight based algorithm to order cluster

Open sudo87 opened this issue 7 months ago • 22 comments

Description

This PR introduces new value: "COMBINED" for config: "host.capacityType.to.order.clusters", which will be used to order cluster, host and pods based on CPU and Memory both.

COMBINED will work with "host.capacityType.to.order.clusters.cputomemoryweight" and overall capacity for cluster/pod/host will be computed based on CPU and memory using weight factor.

The allocator will need to first calculate the combined allocation/usage metric (as follows), before sorting the clusters/pods/hosts to return a ordered list of hosts by this metric, for example: For each host, define metric as:

Metric = CPU * weight + Memory * (1-weight)

Doc PR: https://github.com/apache/cloudstack-documentation/pull/524

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI
  • [ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • [ ] Major
  • [x] Minor

Bug Severity

  • [ ] BLOCKER
  • [ ] Critical
  • [ ] Major
  • [ ] Minor
  • [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

sudo87 avatar Jun 10 '25 09:06 sudo87

Codecov Report

:x: Patch coverage is 76.10063% with 38 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 16.58%. Comparing base (fb6adac) to head (b040691). :warning: Report is 177 commits behind head on main.

Files with missing lines Patch % Lines
...rc/main/java/com/cloud/deploy/FirstFitPlanner.java 71.08% 22 Missing and 2 partials :warning:
...gent/manager/allocator/impl/FirstFitAllocator.java 48.14% 14 Missing :warning:
Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #10997      +/-   ##
============================================
+ Coverage     16.56%   16.58%   +0.01%     
- Complexity    14010    14036      +26     
============================================
  Files          5758     5758              
  Lines        511578   511717     +139     
  Branches      62192    62216      +24     
============================================
+ Hits          84756    84870     +114     
- Misses       417350   417374      +24     
- Partials       9472     9473       +1
Flag Coverage Δ
uitests 3.91% <ø> (ø)
unittests 17.48% <76.10%> (+0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Jun 10 '25 09:06 codecov[bot]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Jun 11 '25 12:06 github-actions[bot]

@blueorangutan package

sudo87 avatar Jun 12 '25 08:06 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 12 '25 09:06 blueorangutan

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13745

blueorangutan avatar Jun 12 '25 10:06 blueorangutan

@blueorangutan package

sudo87 avatar Jun 12 '25 11:06 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 12 '25 12:06 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13751

blueorangutan avatar Jun 12 '25 13:06 blueorangutan

@sudo87 will you create a doc PR for this as well?

DaanHoogland avatar Jun 13 '25 14:06 DaanHoogland

@sudo87 will you create a doc PR for this as well?

Yes @DaanHoogland, doc pr will be needed for this change.

sudo87 avatar Jun 13 '25 15:06 sudo87

@blueorangutan package

sudo87 avatar Jun 17 '25 16:06 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 17 '25 17:06 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13821

blueorangutan avatar Jun 17 '25 18:06 blueorangutan

@blueorangutan test

sudo87 avatar Jun 18 '25 03:06 sudo87

@sudo87 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan avatar Jun 18 '25 03:06 blueorangutan

[SF] Trillian test result (tid-13554) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 2456 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10997-t13554-kvm-ol8.zip Smoke tests completed. 8 look OK, 133 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
runTest Error 0.00 test_2fa.py
runTest Error 0.00 test_account_access.py
runTest Error 0.00 test_accounts.py
runTest Error 0.00 test_affinity_groups_projects.py
runTest Error 0.00 test_affinity_groups.py
runTest Error 0.00 test_annotations.py
runTest Error 0.00 test_async_job.py
runTest Error 0.00 test_attach_multiple_volumes.py
runTest Error 0.00 test_backup_recovery_dummy.py
runTest Error 0.00 test_backup_recovery_veeam.py
runTest Error 0.00 test_direct_download.py
runTest Error 0.00 test_certauthority_root.py
runTest Error 0.00 test_cluster_drs.py
runTest Error 0.00 test_console_endpoint.py
runTest Error 0.00 test_create_list_domain_account_project.py
runTest Error 0.00 test_create_network.py
runTest Error 0.00 test_deploy_vgpu_enabled_vm.py
runTest Error 0.00 test_deploy_virtio_scsi_vm.py
runTest Error 0.00 test_deploy_vm_extra_config_data.py
runTest Error 0.00 test_deploy_vm_iso.py
runTest Error 0.00 test_deploy_vm_iso_uefi.py
runTest Error 0.00 test_deploy_vm_root_resize.py
runTest Error 0.00 test_deploy_vms_in_parallel.py
runTest Error 0.00 test_deploy_vms_with_varied_deploymentplanners.py
runTest Error 0.00 test_deploy_vm_with_userdata.py
runTest Error 0.00 test_diagnostics.py
runTest Error 0.00 test_disk_offerings.py
runTest Error 0.00 test_disk_provisioning_types.py
runTest Error 0.00 test_domain_disk_offerings.py
runTest Error 0.00 test_domain_network_offerings.py
runTest Error 0.00 test_domain_service_offerings.py
runTest Error 0.00 test_guest_os.py
runTest Error 0.00 test_domain_vpc_offerings.py
runTest Error 0.00 test_enable_account_settings_for_domain.py
runTest Error 0.00 test_metrics_api.py
runTest Error 0.00 test_events_resource.py
runTest Error 0.00 test_gateway_on_shared_networks.py
runTest Error 0.00 test_global_acls.py
runTest Error 0.00 test_global_settings.py
runTest Error 0.00 test_guest_vlan_range.py
runTest Error 0.00 test_host_control_state.py
runTest Error 0.00 test_hostha_simulator.py
runTest Error 0.00 test_host_ping.py
runTest Error 0.00 test_image_store_object_migration.py
runTest Error 0.00 test_import_unmanage_volumes.py
runTest Error 0.00 test_internal_lb.py
runTest Error 0.00 test_ipv4_routing.py
runTest Error 0.00 test_ipv6_infra.py
runTest Error 0.00 test_iso.py
runTest Error 0.00 test_kubernetes_clusters.py
runTest Error 0.00 test_kubernetes_supported_versions.py
runTest Error 0.00 test_list_accounts.py
runTest Error 0.00 test_list_disk_offerings.py
runTest Error 0.00 test_list_domains.py
runTest Error 0.00 test_list_hosts.py
runTest Error 0.00 test_list_ids_parameter.py
runTest Error 0.00 test_list_service_offerings.py
runTest Error 0.00 test_list_storage_pools.py
runTest Error 0.00 test_list_volumes.py
runTest Error 0.00 test_loadbalance.py
runTest Error 0.00 test_login.py
runTest Error 0.00 test_migration.py
runTest Error 0.00 test_ms_maintenance_and_safe_shutdown.py
runTest Error 0.00 test_multipleips_per_nic.py
runTest Error 0.00 test_nested_virtualization.py
runTest Error 0.00 test_network_acl.py
runTest Error 0.00 test_network_ipv6.py
runTest Error 0.00 test_network_permissions.py
runTest Error 0.00 test_network.py
runTest Error 0.00 test_nic_adapter_type.py
runTest Error 0.00 test_nic.py
runTest Error 0.00 test_non_contigiousvlan.py
runTest Error 0.00 test_nonstrict_affinity_group.py
runTest Error 0.00 test_outofbandmanagement_nestedplugin.py
runTest Error 0.00 test_outofbandmanagement.py
runTest Error 0.00 test_over_provisioning.py
runTest Error 0.00 test_password_server.py
runTest Error 0.00 test_persistent_network.py
runTest Error 0.00 test_portable_publicip.py
runTest Error 0.00 test_portforwardingrules.py
runTest Error 0.00 test_primary_storage.py
runTest Error 0.00 test_primary_storage_scope.py
runTest Error 0.00 test_privategw_acl_ovs_gre.py
runTest Error 0.00 test_privategw_acl.py
runTest Error 0.00 test_projects.py
runTest Error 0.00 test_public_ip_range.py
runTest Error 0.00 test_purge_expunged_vms.py
runTest Error 0.00 test_pvlan.py
runTest Error 0.00 test_quarantined_ips.py
runTest Error 0.00 test_regions.py
runTest Error 0.00 test_register_userdata.py
runTest Error 0.00 test_reset_configuration_settings.py
runTest Error 0.00 test_reset_vm_on_reboot.py
runTest Error 0.00 test_resource_accounting.py
runTest Error 0.00 test_resource_detail.py
runTest Error 0.00 test_resource_names.py
runTest Error 0.00 test_restore_vm.py
runTest Error 0.00 test_router_dhcphosts.py
runTest Error 0.00 test_router_dns.py
runTest Error 0.00 test_router_dnsservice.py
runTest Error 0.00 test_routers_iptables_default_policy.py
runTest Error 0.00 test_routers_network_ops.py
runTest Error 0.00 test_routers.py
runTest Error 0.00 test_scale_vm.py
runTest Error 0.00 test_secondary_storage.py
runTest Error 0.00 test_service_offerings.py
runTest Error 0.00 test_set_sourcenat.py
runTest Error 0.00 test_sharedfs_lifecycle.py
runTest Error 0.00 test_snapshots.py
runTest Error 0.00 test_ssvm.py
runTest Error 0.00 test_storage_policy.py
runTest Error 0.00 test_templates.py
runTest Error 0.00 test_update_security_group.py
runTest Error 0.00 test_usage_events.py
runTest Error 0.00 test_usage.py
runTest Error 0.00 test_vm_autoscaling.py
runTest Error 0.00 test_vm_deployment_planner.py
runTest Error 0.00 test_vm_life_cycle.py
runTest Error 0.00 test_vm_lifecycle_unmanage_import.py
runTest Error 0.00 test_vm_schedule.py
runTest Error 0.00 test_vm_snapshot_kvm.py
runTest Error 0.00 test_vm_snapshots.py
runTest Error 0.00 test_vm_strict_host_tags.py
runTest Error 0.00 test_vnf_templates.py
runTest Error 0.00 test_volumes.py
runTest Error 0.00 test_vpc_ipv6.py
runTest Error 0.00 test_vpc_redundant.py
runTest Error 0.00 test_vpc_router_nics.py
runTest Error 0.00 test_vpc_vpn.py
runTest Error 0.00 test_webhook_delivery.py
runTest Error 0.00 test_webhook_lifecycle.py
runTest Error 0.00 test_host_maintenance.py
runTest Error 0.00 test_hostha_kvm.py

blueorangutan avatar Jun 18 '25 05:06 blueorangutan

@blueorangutan package

sudo87 avatar Jun 19 '25 05:06 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 19 '25 05:06 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13835

blueorangutan avatar Jun 19 '25 06:06 blueorangutan

@blueorangutan package

sudo87 avatar Jun 19 '25 09:06 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 19 '25 09:06 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13843

blueorangutan avatar Jun 19 '25 11:06 blueorangutan

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Jul 03 '25 07:07 github-actions[bot]

@blueorangutan package

sudo87 avatar Jul 04 '25 06:07 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jul 04 '25 06:07 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14022

blueorangutan avatar Jul 04 '25 08:07 blueorangutan

@blueorangutan package

sudo87 avatar Jul 09 '25 10:07 sudo87

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jul 09 '25 10:07 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14116

blueorangutan avatar Jul 09 '25 11:07 blueorangutan

@blueorangutan test

RosiKyu avatar Jul 11 '25 05:07 RosiKyu