spark2.0-examples icon indicating copy to clipboard operation
spark2.0-examples copied to clipboard

Question in the blog of structured streaming part 11

Open bithw1 opened this issue 7 years ago • 0 comments

Hi @phatak-dev First, thank you very much for the great effort to write the structured streaming series. I got a question: in http://blog.madhukaraphatak.com/introduction-to-spark-structured-streaming-part-11/

Running the Example Enter the below records in socket console. These are records for AAPL with time stamps.

First Event The first records is for time Wed, 27 Apr 2016 11:34:22 GMT.

1461756862000,"aapl",500.0 Spark outputs below results which indicates start of window


Batch: 0

+---------------------------------------------+----------+ |window |sum(value)| +---------------------------------------------+----------+ |[2016-04-27 17:04:20.0,2016-04-27 17:04:30.0]|500.0 | +---------------------------------------------+----------+

The first record is 2016-04-27 11:34:22, why in batch 0 the window interval is 2016-04-27 17:04:20.0,2016-04-27 17:04:30.0, I thought it should be around 2016-04-27 11:34:22,

bithw1 avatar Jul 23 '18 23:07 bithw1