apache
Improvements to quota tariffs APIs and UI
Description
This PR proposes some improvements to quota tariffs APIs and UI. Two parameters have been added to the quotaTariffList API: id and listonlyremoved. The first one refers to the tariff's ID, and the other to a flag that indicates whether only the removed tariffs will be listed. The keyword API parameter has also been changed in order to filter tariffs that contain a given keyword in their names.
Additionally, a validation has been added to check if the calling user has access to the tariff's activation rules. They will have access if they have permission to create or update tariffs.
The quota tariff's UI has also been refactored. Now, more fields of the quotaTariffList response are displayed, and the operator can easily create, edit and filter tariffs.
To filter tariffs by usage type, the listUsageType API response has been refactored, adding the name of the usage type in uppercase letters, separated by underscores, such as ALLOCATED_VM.
Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] Enhancement (improves an existing feature and functionality)
- [X] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [X] Major
- [ ] Minor
Screenshots:
Tariffs details
Tariffs list view
Create form
Update form
How Has This Been Tested?
listUsageTypes API
I logged in CloudMonkey with a root admin user and I executed the listUsageTypes command, receiving the following response.
listUsageTypes
> listUsageTypes
{
"count": 25,
"usagetype": [
{
"description": "Running Vm Usage",
"id": 1,
"name": "RUNNING_VM"
},
{
"description": "Allocated Vm Usage",
"id": 2,
"name": "ALLOCATED_VM"
},
{
"description": "IP Address Usage",
"id": 3,
"name": "IP_ADDRESS"
},
{
"description": "Network Usage (Bytes Sent)",
"id": 4,
"name": "NETWORK_BYTES_SENT"
},
{
"description": "Network Usage (Bytes Received)",
"id": 5,
"name": "NETWORK_BYTES_RECEIVED"
},
{
"description": "Volume Usage",
"id": 6,
"name": "VOLUME"
},
{
"description": "Template Usage",
"id": 7,
"name": "TEMPLATE"
},
{
"description": "ISO Usage",
"id": 8,
"name": "ISO"
},
{
"description": "Snapshot Usage",
"id": 9,
"name": "SNAPSHOT"
},
{
"description": "Security Group Usage",
"id": 10,
"name": "SECURITY_GROUP"
},
{
"description": "Load Balancer Usage",
"id": 11,
"name": "LOAD_BALANCER_POLICY"
},
{
"description": "Port Forwarding Usage",
"id": 12,
"name": "PORT_FORWARDING_RULE"
},
{
"description": "Network Offering Usage",
"id": 13,
"name": "NETWORK_OFFERING"
},
{
"description": "VPN users usage",
"id": 14,
"name": "VPN_USERS"
},
{
"description": "VM Disk usage(I/O Read)",
"id": 21,
"name": "VM_DISK_IO_READ"
},
{
"description": "VM Disk usage(I/O Write)",
"id": 22,
"name": "VM_DISK_IO_WRITE"
},
{
"description": "VM Disk usage(Bytes Read)",
"id": 23,
"name": "VM_DISK_BYTES_READ"
},
{
"description": "VM Disk usage(Bytes Write)",
"id": 24,
"name": "VM_DISK_BYTES_WRITE"
},
{
"description": "VM Snapshot storage usage",
"id": 25,
"name": "VM_SNAPSHOT"
},
{
"description": "Volume on secondary storage usage",
"id": 26,
"name": "VOLUME_SECONDARY"
},
{
"description": "VM Snapshot on primary storage usage",
"id": 27,
"name": "VM_SNAPSHOT_ON_PRIMARY"
},
{
"description": "Backup storage usage",
"id": 28,
"name": "BACKUP"
},
{
"description": "Bucket storage usage",
"id": 29,
"name": "BUCKET"
},
{
"description": "Network usage",
"id": 30,
"name": "NETWORK"
},
{
"description": "VPC usage",
"id": 31,
"name": "VPC"
}
]
}
As noticed, the name field has been added and is being populated as expected. The id and description fields have not undergone any changes.
Quota tariffs
Tests setup
Before testing, I deleted all default tariffs that existed in the database, and I verified that the quotaTariffList parameters were being identified correctly.
help quotaTariffList
> help quotaTariffList
quotaTariffList: Lists all quota tariff plans
API Params Type Description
========== ==== ===========
enddate date The end date of the quota tariff. The rec
ommended format is "yyyy-MM-dd'T'HH:mm:s
sZ" (e.g.: "2023-01-01T12:00:00+0100");
however, the following formats are also
accepted: "yyyy-MM-dd HH:mm:ss" (e.g.: "
2023-01-01 12:00:00") and "yyyy-MM-dd" (
e.g.: "2023-01-01" - if the time is not
added, it will be interpreted as "23:59:
59"). If the recommended format is not u
sed, the date will be considered in the
server timezone.
id string The quota tariff's id.
keyword string List by keyword
listall boolean False will list only not removed quota ta
riffs. If set to true, we will list all,
including the removed ones. The default
is false.
listonlyremoved boolean If set to true, we will list only the rem
oved tariffs. The default is false.
name string The name of the quota tariff.
page integer
pagesize integer
startdate date The start date of the quota tariff. The r
ecommended format is "yyyy-MM-dd'T'HH:mm
:ssZ" (e.g.: "2023-01-01T12:00:00+0100")
; however, the following formats are als
o accepted: "yyyy-MM-dd HH:mm:ss" (e.g.:
"2023-01-01 12:00:00") and "yyyy-MM-dd"
(e.g.: "2023-01-01" - if the time is no
t added, it will be interpreted as "00:0
0:00"). If the recommended format is not
used, the date will be considered in th
e server timezone.
usagetype integer Usage type of the resource
quotaTariffList
> quotaTariffList
# no output
Quota tariffs view
Create quota tariff through UI
Once the Create Quota Tariff button is clicked, a form is rendered on the screen, with all the available API fields, except for activationrule.
Create Quota Tariff form
Creation of a tariff with only the required values
The required fields are name, usage type and value. When creating a tariff with only them, the start date is automatically set to now, and the other fields are populated correctly.
Tariffs list view
Tariff details
Creation of a tariff filling all fields
Creation form
Tariffs list view
Tariffs details
Update quota tariff through UI
When selecting the option to edit a tariff, its description, value and end date are populated correctly.
Update form
Update all fields
If at least one field is changed, then the API request to quotaTariffUpdate is performed, passing in only the changed values.
Update form
HTTP request payload
Update no fields
If there are no changed fields, the API request to quotaTariffUpdate is not performed.
Remove quota tariff through UI
When the delete button is clicked, a modal is rendered asking for confirmation. Once it is confirmed, the quota tariff is removed, and its removed field is set to the time that the operation was performed.
Filter tariffs
The tariffs can be filtered by name, usage type and whether they have been removed.
Filtering by usage type
Filtering by name
Filtering the removed ones
Codecov Report
Attention: Patch coverage is 49.41176% with 43 lines in your changes missing coverage. Please review.
Project coverage is 15.55%. Comparing base (
cc1dcf5) to head (b87b0d6). Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #9225 +/- ##
==========================================
Coverage 15.54% 15.55%
- Complexity 11997 12010 +13
==========================================
Files 5496 5499 +3
Lines 481522 481853 +331
Branches 62238 62125 -113
==========================================
+ Hits 74857 74930 +73
- Misses 398378 398637 +259
+ Partials 8287 8286 -1
| Flag | Coverage Δ | |
|---|---|---|
| uitests | 4.17% <ø> (-0.01%) |
:arrow_down: |
| unittests | 16.32% <49.41%> (+<0.01%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Description
This PR proposes some improvements to quota tariffs APIs and UI. Two parameters have been added to the
quotaTariffListAPI:idandlistonlyremoved. The first one refers to the tariff's ID, and the other to a flag that indicates whether only the removed tariffs will be listed. ThekeywordAPI parameter has also been changed in order to filter tariffs that contain a given keyword in their names.Additionally, a validation has been added to check if the calling user has access to the tariff's activation rules. They will have access if they have permission to create or update tariffs.
The quota tariff's UI has also been refactored. Now, more fields of the
quotaTariffListresponse are displayed, and the operator can easily create, edit and filter tariffs.To filter tariffs by usage type, the
listUsageTypeAPI response has been refactored, adding the name of the usage type in uppercase letters, separated by underscores, such asALLOCATED_VM.Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [x] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [x] Major
- [ ] Minor
Screenshots:
Tariffs details Tariffs list view Create form Update form
How Has This Been Tested?
listUsageTypesAPII logged in CloudMonkey with a root admin user and I executed the
listUsageTypescommand, receiving the following response.
listUsageTypesAs noticed, thenamefield has been added and is being populated as expected. Theidanddescriptionfields have not undergone any changes.Quota tariffs
Tests setup
Before testing, I deleted all default tariffs that existed in the database, and I verified that the
quotaTariffListparameters were being identified correctly.
help quotaTariffListquotaTariffListQuota tariffs viewCreate quota tariff through UI
Once the
Create Quota Tariffbutton is clicked, a form is rendered on the screen, with all the available API fields, except foractivationrule.Create Quota Tariff form
Creation of a tariff with only the required values
The required fields are name, usage type and value. When creating a tariff with only them, the start date is automatically set to now, and the other fields are populated correctly.
Tariffs list view Tariff details
Creation of a tariff filling all fields
Creation form Tariffs list view Tariffs details
Update quota tariff through UI
When selecting the option to edit a tariff, its description, value and end date are populated correctly.
Update form
Update all fields
If at least one field is changed, then the API request to
quotaTariffUpdateis performed, passing in only the changed values.Update form HTTP request payload
Update no fields
If there are no changed fields, the API request to
quotaTariffUpdateis not performed.Remove quota tariff through UI
When the delete button is clicked, a modal is rendered asking for confirmation. Once it is confirmed, the quota tariff is removed, and its
removedfield is set to the time that the operation was performed.Filter tariffs
The tariffs can be filtered by name, usage type and whether they have been removed.
Filtering by usage type Filtering by name Filtering the removed ones
It would be ideal if we could have the possibility of assigning different rates for some domains, as, in our day-to-day business with the B2B public, it is common to negotiate different rates depending on the level of allocation and reservation that the client requests.
It would be ideal if we could have the possibility of assigning different rates for some domains, as, in our day-to-day business with the B2B public, it is common to negotiate different rates depending on the level of allocation and reservation that the client requests.
@bigsys-it, yes, for sure.
With the Quota Plugin, this can be achieved through activation rules. These rules consist of logical expressions, written in JavaScript, which should evaluate to either a boolean or a numeric value. If the expression evaluates to a boolean, the tariff's value will be applied based on the result. If it evaluates to a numeric value, that value will directly become the tariff's value.
For instance, if there's a need to assign a specific value for a given domain, the following activation rule could be applied:
if (domain.id === 'ba820fc7-12ea-11ef-9500-d283eea8b15e') {
return 150;
} else {
return 175;
}
Currently, applying a rule to a tariff can only be done through CloudMonkey and its UI support is beyond the scope of this PR. I strongly advise reviewing the PR that introduced this feature (#5909) along with its specifications (#5891). They provide more detailed examples and technical insights that could be extremely helpful.
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
@JoaoJandre, I've added UI support for the position parameter. Could you take a look at this one, please?
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10418
LGTM, did some testing in the UI: created tariffs, updated them, deleted them, opened their details, all looked good.
@JoaoJandre, thanks for testing!
@DaanHoogland a [SL] Trillian-Jenkins test job (alma9 mgmt + kvm-alma9) has been kicked to run smoke tests
[SF] Trillian test result (tid-10931) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 52924 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9225-t10931-kvm-alma9.zip Smoke tests completed. 135 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_03_ping_in_ssvm_success | Failure |
15.51 | test_diagnostics.py |
| test_08_arping_in_ssvm | Failure |
5.24 | test_diagnostics.py |
| test_06_purge_expunged_vm_background_task | Failure |
340.45 | test_purge_expunged_vms.py |
@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10464
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10623
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
[SF] Trillian test result (tid-11067) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 52755 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9225-t11067-kvm-ol8.zip Smoke tests completed. 138 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_04_nonsecured_to_secured_vm_migration | Error |
359.39 | test_vm_life_cycle.py |
@DaanHoogland the error does not look related, should we rerun?
@JoaoJandre , I also saw a GHA failures so I rekicked those.
and this looks like a typical "the lab is too busy" error. It will probably fail again in another location. We can try later tominght...
Exception: Job failed: {accountid : '2b219d3e-5944-11ef-9017-1e00a30001a9', account : 'admin', domainid : 'f47b9e58-5943-11ef-9017-1e00a30001a9', domainpath : 'ROOT', userid : '2b222f99-5944-11ef-9017-1e00a30001a9', cmd : 'org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin', jobstatus : 2, jobprocstatus : 0, jobresultcode : 530, jobresulttype : 'object', jobresult : {errorcode : 530, errortext : 'Unable to start a VM [b8743ae9-b733-44fa-97da-254286f0fec4] due to [Unable to create a deployment for VM instance {"id":390,"instanceName":"i-258-390-VM","type":"User","uuid":"b8743ae9-b733-44fa-97da-254286f0fec4"}].'}, jobinstancetype : 'VirtualMachine', jobinstanceid : 'b8743ae9-b733-44fa-97da-254286f0fec4', created : '2024-08-13T18:09:54+0000', completed : '2024-08-13T18:10:08+0000', jobid : 'd0601697-b66c-446b-b6ec-5a7254463869'}
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
[SF] Trillian test result (tid-11076) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 58871 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9225-t11076-kvm-ol8.zip Smoke tests completed. 139 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|