SynapseML icon indicating copy to clipboard operation
SynapseML copied to clipboard
verified publisher icon microsoft

[QUESTION] How to get the latest Spark 3.1 version?

Open Ishitori opened this issue 3 years ago • 5 comments

Hi team,

I see that the Spark3.1 branch is updated regularly, and I wonder what is the latest Spark 3.1 compatible version which I could use? Is there is a regularly updated list of available downloads?

Thank you.

AB#1972681

Ishitori avatar Sep 09 '22 16:09 Ishitori

Hey @Ishitori :wave:! Thank you so much for reporting the issue/feature request :rotating_light:. Someone from SynapseML Team will be looking to triage this issue soon. We appreciate your patience.

github-actions[bot] avatar Sep 09 '22 16:09 github-actions[bot]

Hi @Ishitori, could you try com.microsoft.azure:synapseml_2.12:0.9.5-35-e962330b-SNAPSHOT? Thx

serena-ruan avatar Sep 14 '22 01:09 serena-ruan

Yes, working! Thanks for sharing. Is there a way to follow the updates for Spark 3.1?

Ishitori avatar Sep 14 '22 21:09 Ishitori

@Ishitori We don't regularly update Spark3.1 for now, as we've upgraded the main branch to Spark3.2. If you'd like to migrate to Spark3.2, then you could find latest versions in our regular releases. We might also update Spark3.1 versions in our installation guidance at the bottom of the website.

serena-ruan avatar Sep 19 '22 07:09 serena-ruan

Hi @Ishitori, is there a version of synapseml 0.9.5 that supports Spark 3.0.1 (in particular, with working saving of trained LIghtGBM)? Or has its support been completely dropped already?

fonhorst avatar Sep 21 '22 11:09 fonhorst