Saturday, June 13, 2015

Big Data event processing: Spark, Storm, Twitter Heron (!)

Infographic: The Four V's of Big Data | The Big Data Hub
The Four V's of Big Data

Choose your real-time weapon: Storm or Spark? @ InfoWorld
this is for Velocity: steaming data

bigdata - Apache Spark vs. Apache Storm - Stack Overflow

Spark, Storm and Real Time Analytics @ InfoQ

Twitter Has Replaced Storm with Heron @ InfoQ

"Twitter has replaced Storm with Heron which provides up to 14 times more throughput and up to 10 times less latency on a word count topology, and helped them reduce the needed hardware to a third.

Twitter used Storm to analyze large amounts of data in real time for years, and open sourced it back in 2011. The project was later incubated at Apache, becoming a top level project last fall. Having a quarterly release cycle, Storm has reached version 0.9.5 and is approaching the stable and desired version 1.0. But all this time, Twitter has been working on a replacement called Heron because Storm is no longer up to the task for their real-time processing needs.

Twitter’s new real-time requirements are: “billions of events per minute; have sub-second latency and predictable behavior at scale;"

"Compatibility with Storm: Heron provides full backward compatibility with Storm,"

"overall 3x reduction in hardware"

Removing the 140 character limit from Direct Messages - Announcements - Twitter Developers

History of Apache Storm and lessons learned - thoughts from the red planet - thoughts from the red planet (by Nathan Marz, creator of Storm)

Twitter Heron: Stream Processing at Scale | the morning paper

No comments: