BigData-Notes
BigData-Notes copied to clipboard
OS-EDU
大数据组件学习记录~~
Bumps [mysql:mysql-connector-java](https://github.com/mysql/mysql-connector-j) from 5.1.44 to 8.0.28. Changelog Sourced from mysql:mysql-connector-java's changelog. Changelog https://dev.mysql.com/doc/relnotes/connector-j/en/ Version 8.3.0 Fix for Bug#107107 (Bug#34101635), Redundant "Reset stmt" when setting useServerPrepStmts&cachePrepStmts to true. Thanks to Marcos...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps org.apache.spark:spark-core_2.12 from 3.0.0 to 3.3.3. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
Bumps [mysql:mysql-connector-java](https://github.com/mysql/mysql-connector-j) from 5.1.27 to 8.0.28. Changelog Sourced from mysql:mysql-connector-java's changelog. Changelog https://dev.mysql.com/doc/relnotes/connector-j/en/ Version 8.3.0 Fix for Bug#107107 (Bug#34101635), Redundant "Reset stmt" when setting useServerPrepStmts&cachePrepStmts to true. Thanks to Marcos...
当前项目数据源较多，目前各个指标均存放在 MySQL 中，后续可能会同步到 Hive、Hudi 以及 ClickHouse 等数据库中。关于 MySQL 数据全量同步到 Hive 使用的是 DataX，但由于其支持的数据源较少，因此需要调研新的数据集成与同步框架。 在初步对比 flink_cdc 和 seatunnel 之后，考虑使用门槛，先调研 flink_cdc～～
**FINISH:** - [Spark-Core](https://github.com/QuakeWang/BigData-Notes/tree/main/code/SparkTutorial/spark-core)：编写一篇使用 RDD 计算热门商品的博客 - [SparkSQL](https://github.com/QuakeWang/BigData-Notes/tree/main/code/SparkTutorial/spark-sql)：结合《Spark 权威指南》完善对于 DataSet 的使用 - [SparkStreaming](https://github.com/QuakeWang/BigData-Notes/tree/main/code/SparkTutorial/spark-streaming)