Wednesday, October 16, 2019

AWS Kinesis vs. Apache Kafka

Kinesis vs. Kafka

Kinesis works with streaming data.
Kafka works with streaming data too.

Kinesis Streams is like Kafka Core.
Kinesis Analytics is like Kafka Streams.
A Kinesis Shard is like Kafka Partition.
They are similar and get used in similar use cases.

Data is stored in Kinesis for default 24 hours, and you can increase that up to 7 days.
Kafka records are by default stored for 7 days and you can increase that until you run out of disk space. In fact, you can decide by the size of the data or by date. You can even use compaction with Kafka so it only stores the latest timestamp per key per record in the log.