[WIP] Dynamic and Static Routing

[weizhouapache]

Description

This PR contains 3 features

• IPv4 Static Routing (Routed mode) #9346 Design document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=306153967

• AS Numbers Management #9410 Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/BGP+AS+Numbers+Management

• Dynamic routing Design Document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=315492858

• Document: https://github.com/apache/cloudstack-documentation/pull/419

• TODO: unit tests

Please note, this requires a new systemvm template which has frr installed.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
• [ ] build/CI
• [ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [ ] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 31.97888% with 2963 lines in your changes missing coverage. Please review.

Project coverage is 15.76%. Comparing base (85765c3) to head (a47678d). Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ache/cloudstack/network/RoutedIpv4ManagerImpl.java	0.00%	995 Missing :warning:
...er/src/main/java/com/cloud/bgp/BGPServiceImpl.java	3.20%	241 Missing and 1 partial :warning:
...ain/java/com/cloud/network/vpc/VpcManagerImpl.java	14.61%	93 Missing and 18 partials :warning:
...src/main/java/com/cloud/api/ApiResponseHelper.java	0.00%	103 Missing :warning:
.../apache/cloudstack/network/dao/BgpPeerDaoImpl.java	0.00%	99 Missing :warning:
...oudstack/network/dao/BgpPeerNetworkMapDaoImpl.java	0.00%	99 Missing :warning:
.../network/dao/Ipv4GuestSubnetNetworkMapDaoImpl.java	0.00%	93 Missing :warning:
...rc/main/java/com/cloud/dc/dao/ASNumberDaoImpl.java	0.00%	80 Missing :warning:
.../java/org/apache/cloudstack/network/BgpPeerVO.java	0.00%	68 Missing :warning:
.../schema/src/main/java/com/cloud/dc/ASNumberVO.java	0.00%	67 Missing :warning:
... and 91 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #9470      +/-   ##
============================================
+ Coverage     15.61%   15.76%   +0.14%     
- Complexity    12119    12507     +388     
============================================
  Files          5555     5619      +64     
  Lines        486283   491158    +4875     
  Branches      62524    59754    -2770     
============================================
+ Hits          75944    77426    +1482     
- Misses       401966   405277    +3311     
- Partials       8373     8455      +82     

Flag	Coverage Δ
uitests	4.05% <ø> (-0.08%)	:arrow_down:
unittests	16.58% <31.97%> (+0.17%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10516

[blueorangutan]

[SF] Trillian test result (tid-10998) Environment: kvm-rocky8 (x3), Advanced Networking with Mgmt server r8 Total time taken: 55178 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t10998-kvm-rocky8.zip Smoke tests completed. 136 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_10_bgp_peers	Failure	0.29	test_ipv4_routing.py
test_04_nonsecured_to_secured_vm_migration	Error	428.61	test_vm_life_cycle.py

[blueorangutan]

[SF] Trillian test result (tid-10999) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 54460 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t10999-kvm-alma9.zip Smoke tests completed. 137 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_10_bgp_peers	Failure	0.30	test_ipv4_routing.py

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10524

[blueorangutan]

[SF] Trillian test result (tid-11006) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 61660 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11006-kvm-alma9.zip Smoke tests completed. 136 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_05_ping_in_cpvm_success	Failure	15.55	test_diagnostics.py
test_01_vpc_site2site_vpn	Failure	343.70	test_vpc_vpn.py

[blueorangutan]

[SF] Trillian test result (tid-11007) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 54818 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11007-kvm-rocky8.zip Smoke tests completed. 137 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_secure_vm_migration	Error	134.39	test_vm_life_cycle.py
test_01_secure_vm_migration	Error	134.40	test_vm_life_cycle.py

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10553

[blueorangutan]

[SF] Trillian test result (tid-11021) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 52414 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11021-kvm-alma9.zip Smoke tests completed. 138 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[blueorangutan]

[SF] Trillian test result (tid-11020) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 52988 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11020-kvm-rocky8.zip Smoke tests completed. 138 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 10569

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10570

[blueorangutan]

[SF] Trillian test result (tid-11029) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 53153 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11029-kvm-rocky8.zip Smoke tests completed. 137 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestClusterDRS>:setup	Error	0.00	test_cluster_drs.py

[blueorangutan]

[SF] Trillian test result (tid-11028) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 58027 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9470-t11028-kvm-alma9.zip Smoke tests completed. 138 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[weizhouapache]

Great (and mammoth) work @weizhouapache! I did try to review changes but it is very high-level and may need more eyes. Definitely will need a lot of testing. Great to see many integration tests (we may add some for asnnumber/range) and I hope we will add unit tests at some point as discussed. I've added some comments but may add some more next week. Largely the changes look good. Only some minor concerns:

• Many new APIs with very similar sounding names. It may get confusing. Maybe if it can be reviewed and consolidated. If not would need a good documentation
• A lot of file changes are due to new arguments in createNetwork, createNetworkOffering, etc methods which could have been possible avoided using overloaded methods.
• While creating API responses we are querying DB multiple times for a single entity. It maybe worth looking if that can be improved either with views or using some details parameter in the calling APIs

thanks @shwstppr !

I have added the APIs to doc PR: https://github.com/apache/cloudstack-documentation/pull/419 hope users get better understanding

I have addressed most of your comments. Regarding the db queries, considering the tables (for ipv4 subnets, bgp peers, as numbers) are comparably very small (with vm_instances, networks or volumes table), we could improve in next stage if users have issues.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10691

[weizhouapache]

@blueorangutan test