openwhisk icon indicating copy to clipboard operation
openwhisk copied to clipboard

Prepare to integrate new scheduler into apache/openwhisk-deploy-kube

Open hunhoffe opened this issue 3 years ago • 6 comments

Description

I am working on integrating the new scheduler into openwhisk-deploy-kube (see this draft PR here). While most of the changes that need to be made are within the apache/openwhisk-deploy-kube repo, some changes need to be made here for full support.

Namely:

  • Scheduler images need to be build and pushed to dockerhub
  • The scheduler and controller both need to option to use the akka cluster discovery in the case that they are replicated; the existing mechanism of using seedNodes is not a great fit for Kubernetes deployments

I tested locally (mostly using Kubernetes deployments) but I'm not sure the best way to test the tooling and changes to travis/CI.

Note that while I did, in general, try to integrate the scheduler into the tooling, I did not attempt to add it to dev/src/main/groovy/intellijRunConfig.groovy, as I don't have experience in this area.

My changes affect the following components

  • [ ] API
  • [x] Controller
  • [ ] Message Bus (e.g., Kafka)
  • [ ] Loadbalancer
  • [x] Scheduler
  • [ ] Invoker
  • [ ] Intrinsic actions (e.g., sequences, conductors)
  • [ ] Data stores (e.g., CouchDB)
  • [ ] Tests
  • [x] Deployment
  • [ ] CLI
  • [x] General tooling
  • [ ] Documentation

Types of changes

  • [ ] Bug fix (generally a non-breaking change which closes an issue).
  • [x] Enhancement or new feature (adds new functionality).
  • [ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

  • [x] I signed an Apache CLA.
  • [x] I reviewed the style guides and followed the recommendations (Travis CI will check :).
  • [ ] I added tests to cover my changes.
  • [ ] My changes require further changes to the documentation.
  • [ ] I updated the documentation where necessary.

hunhoffe avatar Jul 12 '22 03:07 hunhoffe

Codecov Report

Merging #5278 (806cbe7) into master (138f3d9) will decrease coverage by 4.60%. The diff coverage is 40.00%.

@@            Coverage Diff             @@
##           master    #5278      +/-   ##
==========================================
- Coverage   81.03%   76.42%   -4.61%     
==========================================
  Files         239      239              
  Lines       14245    14249       +4     
  Branches      594      602       +8     
==========================================
- Hits        11543    10890     -653     
- Misses       2702     3359     +657     
Impacted Files Coverage Δ
...rg/apache/openwhisk/core/scheduler/Scheduler.scala 10.75% <25.00%> (+0.36%) :arrow_up:
...e/loadBalancer/ShardingContainerPoolBalancer.scala 89.26% <100.00%> (ø)
...core/database/cosmosdb/RxObservableImplicits.scala 0.00% <0.00%> (-100.00%) :arrow_down:
...ore/database/cosmosdb/cache/CacheInvalidator.scala 0.00% <0.00%> (-100.00%) :arrow_down:
...e/database/cosmosdb/cache/ChangeFeedConsumer.scala 0.00% <0.00%> (-100.00%) :arrow_down:
...core/database/cosmosdb/CosmosDBArtifactStore.scala 0.00% <0.00%> (-95.85%) :arrow_down:
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala 0.00% <0.00%> (-93.90%) :arrow_down:
...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala 0.00% <0.00%> (-92.31%) :arrow_down:
...enwhisk/connector/kafka/KamonMetricsReporter.scala 0.00% <0.00%> (-83.34%) :arrow_down:
...e/database/cosmosdb/cache/KafkaEventProducer.scala 0.00% <0.00%> (-78.58%) :arrow_down:
... and 13 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Jul 13 '22 05:07 codecov-commenter

Apologies for making the PR live - then draft - then live again, but I believe it is in a good state now and ready for review!

hunhoffe avatar Jul 13 '22 13:07 hunhoffe

Hi Erika, have you filed an Individual Contributors License Agreement with Apache (https://www.apache.org/licenses/contributor-agreements.html#clas)? I did a quick search and I don't think I found one for you, but the tooling isn't always perfect.

If you haven't done so already please go ahead and do that.

dgrove-oss avatar Jul 14 '22 18:07 dgrove-oss

@dgrove-oss I did submit an ICLA in the past, so I should be good to go. To be on the safe side and check for clerical errors, I have sent an email asking for confirmation that my form is on file.

hunhoffe avatar Jul 15 '22 02:07 hunhoffe

ok, cool. That's fine then. The tool for looking up ICLAs changed recently. Now you have to search by email address instead of by name and I haven't figured out how to do fuzzy searches yet.

dgrove-oss avatar Jul 15 '22 13:07 dgrove-oss

Rebased to master so I can make sure new changes integrate well into openwhisk-deploy-kube, also - I did confirm my ICLA is good.

hunhoffe avatar Jul 21 '22 15:07 hunhoffe

Rebased to master, finished addressing comments about changes to controller (by removing changes to the controller)

hunhoffe avatar Sep 01 '22 23:09 hunhoffe

Thank you, Erika, for your effort. 👍

style95 avatar Sep 05 '22 01:09 style95