AWS re:Invent 2020: Top 5 best practices for data streaming with Amazon Kinesis

Latency and throughput are key considerations in stream processing workloads. In this session, you learn tips for optimizing throughput and latency in stream processing architectures. You also learn about scenarios when it’s necessary to find trade-offs between latency and throughput and about scalability. Services discussed in this session include Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, and Amazon Kinesis Data Analytics for Apache Flink.
Learn more about re:Invent 2020 at bit.ly/3c4NSdY
Subscribe:
More AWS videos bit.ly/2O3zS75
More AWS events videos bit.ly/316g9t4
#AWS #AWSEvents

[@SergiiStarodubtsev]

14:00 maybe to explain right away why in single shard we have 2 partition keys, A and B. While one would think that one shard may have msgs with single partition key in it. Otherwise why not use 2 shards. To mention that's one of configurations, depending on the data spread strategy.

[@SergiiStarodubtsev]

I think what could be cleared out is how come we got 1 mb per second on input and 2 mb per second on output, where that +1 mb of data comes from? Like I put 1 apple 🍎 in 1 second and got 2 🍎 🍏 in another second as result. That wasn't clear in the beginning, and the question was poped out immediately.

[@ziadrida3119]

Lambda has 1ms or better latency?! 16:38

[@SergiiStarodubtsev]

09:30 Ecs,also supports many languages, Java python and other languages. To the point to comparing to Lambda by that characteristic . Along with the kinesis library, ksl, that takes cares shard consumption to the Ecs tasks. 16:05 is where is chosen to explain it. Lambda lose to many parameters if follow the matrix. To KSL, for example.

[@awssupport]

Hey there! 👋 While we're unable to address these questions directly, we understand where you're coming from. I've sent off your concerns to our service team for review. 🔍 ^KS

[@SergiiStarodubtsev]

@@awssupport that's is not a concern, that's friendly feedback )

[@subhamoyguha7861]

I like to use Amazon Data Kinesis to get the access log of Amazon Api Gateway. How to do it ?

[@awssupport]

Great question! To obtain access logs for API Gateway, take a look at the doc here: go.aws/3P8ZMGM. If you need additional assistance, these options may be helpful: go.aws/get-help. ⬅️ ^LG

[@JanekBogucki]

kzbin.info/www/bejne/i3aWZXaNds2HadE Should say "Time taken with 10 EFO" not 20.