amazon-kinesis-client-python icon indicating copy to clipboard operation
amazon-kinesis-client-python copied to clipboard
verified publisher icon awslabs

properties: initialPositionInStream = LATEST not working

Open bachzz opened this issue 5 years ago • 2 comments

my KCL workers are getting records via TRIM_HORIZON even when I've set initialPositionInStream to LATEST. Anyone experiencing the same issue? solution? Thanks.

bachzz avatar Feb 13 '20 03:02 bachzz

Did you set 'shard iterator type' when you get shard iterator? I think you have to set ShardIteratorType like below. get_shard_iterator(StreamName=my_stream_name,ShardId=my_shard_id,ShardIteratorType='LATEST')

or

String getIterator(String shardId, LATEST, String sequenceNumber)

Gyuul avatar Feb 13 '20 05:02 Gyuul

I think those functions belong to boto3 kcl. I'm using RecordProcessor in sample_kclpy_app.py https://github.com/awslabs/amazon-kinesis-client-python/blob/master/samples/sample_kclpy_app.py and I'm not sure how to set ShardIteratorType='LATEST' in code. I can only do that in properties file though

bachzz avatar Feb 13 '20 07:02 bachzz