fix: downgrade to spark3.1
Related Issues/PRs
#xxx
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?
- [ ] I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.
Does this PR change any dependencies?
- [ ] No. You can skip this section.
- [ ] Yes. Make sure the dependencies are resolved correctly, and list changes here.
Does this PR add a new feature? If so, have you added samples on website?
- [ ] No. You can skip this section.
- [ ] Yes. Make sure you have added samples following below steps.
- Find the corresponding markdown file for your new feature in
website/docs/documentationfolder. Make sure you choose the correct classestimators/transformersand namespace. - Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
- Make sure the
DocTablepoints to correct API link. - Navigate to website folder, and run
yarn run startto make sure the website renders correctly. - Don't forget to add
<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples. - Make sure the
WebsiteSamplesTestsjob pass in the pipeline.
Hey @serena-ruan :wave:! Thank you so much for contributing to our repository :raised_hands:. Someone from SynapseML Team will be reviewing this pull request soon. We appreciate your patience and contributions :100:!
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
Codecov Report
Merging #1646 (073f03b) into spark3.1 (7abf839) will decrease coverage by
16.22%. The diff coverage isn/a.
@@ Coverage Diff @@
## spark3.1 #1646 +/- ##
=============================================
- Coverage 85.83% 69.61% -16.23%
=============================================
Files 272 272
Lines 14230 14230
Branches 767 767
=============================================
- Hits 12215 9906 -2309
- Misses 2015 4324 +2309
| Impacted Files | Coverage Δ | |
|---|---|---|
| .../execution/streaming/continuous/HTTPSourceV2.scala | 0.00% <0.00%> (-92.09%) |
:arrow_down: |
| ...che/spark/sql/execution/streaming/HTTPSource.scala | 0.00% <0.00%> (-90.00%) |
:arrow_down: |
| ...ql/execution/streaming/continuous/HTTPSinkV2.scala | 0.00% <0.00%> (-89.75%) |
:arrow_down: |
| ...soft/azure/synapse/ml/cognitive/AudioStreams.scala | 0.00% <0.00%> (-87.88%) |
:arrow_down: |
| ...ft/azure/synapse/ml/cognitive/AzureSearchAPI.scala | 2.59% <0.00%> (-87.02%) |
:arrow_down: |
| ...osoft/azure/synapse/ml/cognitive/AzureSearch.scala | 7.19% <0.00%> (-80.58%) |
:arrow_down: |
| ...rosoft/azure/synapse/ml/geospatial/Geocoders.scala | 15.21% <0.00%> (-80.44%) |
:arrow_down: |
| ...re/synapse/ml/geospatial/CheckPointInPolygon.scala | 15.00% <0.00%> (-80.00%) |
:arrow_down: |
| ...ql/execution/streaming/DistributedHTTPSource.scala | 0.00% <0.00%> (-78.34%) |
:arrow_down: |
| ...se/ml/cognitive/MultivariateAnomalyDetection.scala | 11.29% <0.00%> (-77.42%) |
:arrow_down: |
| ... and 45 more |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
/azp run
Azure Pipelines successfully started running 1 pipeline(s).
/azp run
Azure Pipelines successfully started running 1 pipeline(s).