data-microservices
data-microservices copied to clipboard
My data microservices with Spring talk
- Start rabbitmq-server
- Start redis-server
- Delete contents of ~/outputs
Time | Log
cd /Users/mminella/Documents/IntelliJWorkspace/time/apps/time-source-rabbit/target/java -jar time-source-rabbit-2.0.0.BUILD-SNAPSHOT.jar --spring.cloud.stream.bindings.output.destination=ticktockcd /Users/mminella/Documents/IntelliJWorkspace/log/apps/log-sink-rabbit/target/java -jar log-sink-rabbit-2.0.0.BUILD-SNAPSHOT.jar --server.port=9191 --spring.cloud.stream.bindings.input.destination=ticktock
Scale source
$ cd /Users/mminella/Documents/IntelliJWorkspace/spring-cloud-stream-app-starters/apps$ java -jar time-source-rabbit/target/time-source-rabbit-1.0.3.BUILD-SNAPSHOT.jar --spring.cloud.stream.bindings.output.destination=ticktock --server.port=9191
5:00 Somewhere
cd /Users/mminella/Documents/IntelliJWorkspace/time/apps/time-source-rabbit/target/java -jar time-source-rabbit-2.0.0.BUILD-SNAPSHOT.jar --spring.cloud.stream.bindings.output.destination=fiveoclockinputcd /Users/mminella/Documents/IntelliJWorkspace/data-microservices/five-oclock-transformerjava -jar target/five-oclock-transformer-0.0.1-SNAPSHOT.jar --spring.cloud.stream.bindings.input.destination=fiveoclockinput --spring.cloud.stream.bindings.output.destination=fiveoclockoutput --server.port=8585cd /Users/mminella/Documents/IntelliJWorkspace/log/apps/log-sink-rabbit/target/java -jar log-sink-rabbit-2.0.0.BUILD-SNAPSHOT.jar --server.port=9191 --spring.cloud.stream.bindings.input.destination=fiveoclockoutput
Task demo
- Create database:
mysql:> create database scdf; $ cd /Users/mminella/Documents/IntelliJWorkspace/data-microservices/task-demo$ java -jar target/task-demo-0.0.1-SNAPSHOT.jar
Twitter Streaming Demo
cd /Users/mminella/Documents/IntelliJWorkspace/data-microservices/bin./dataflow.shcd /Users/mminella/Documents/IntelliJWorkspace/data-microservices/binjava -jar spring-cloud-dataflow-shell-1.4.0.RELEASE.jardataflow:> app listdataflow:> app import --uri http://bit.ly/Celsius-SR1-stream-applications-rabbit-mavendataflow:> app listdataflow:> stream create --definition "twitterstream --twitter.credentials.consumerKey=<CONSUMER_KEY> --twitter.credentials.consumerSecret=<CONSUMER_SECRET> --twitter.credentials.accessToken=<ACCESS_TOKEN> --twitter.credentials.accessTokenSecret=ACCESS_TOKEN_SECRET> | file --directory='/Users/mminella/output'" --name ingest --deploydataflow:> stream create --name tweetsCount --definition ":ingest.twitterstream > field-value-counter --field-name=entities.hashtags.text --name=tweetsCountTags" --deploy
Debugging Twitter
dataflow:> app register --type sink --name log --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.0.3.BUILD-SNAPSHOTdataflow:> stream create --definition "twitter | log" --name logTwitter --deploy
Task Demo on SCDF
app register --type task --name boring_task --uri maven://io.spring.cloud.task:task-demo:0.0.1-SNAPSHOTtask create --name myBatchJob --definition 'boring_task'