Sunday, March 04, 2012

Microsoft SQL Server StreamInsight

Microsoft StreamInsight, part of the recent SQL Server 2008 R2 release,
is a new platform for building rich applications
that can process high volumes of event stream data with near-zero latency


Databases are expanding way beyond traditional relational and transnational processing.
Microsoft SQL Server is in each version getting more capability,
that could sometimes be confusing. In fact, it is relatively easy:
SQL Server is a trusted name in IT, so anything data-related
Microsoft bundles with SQL server... Instead of NoSQL they call it "Big Data".

One of such "new features" is processing of "stream data".
For example, monitoring continuously changing stock prices for trading,
or monitoring status of large number of device, such as power meter readers, etc.
In such cases there is a HUGE amount of data, and key is to detect events in them.

A new "keyword" is "Complex Event Processing", (CEP)
part of broader "Business Intelligence" (BI) tools.

StreamInsight is a .NET tool, programmed from VisualStudio.
It is part of SQL Server since version 2008 R2 (except Express edition).

Here a brief video introduction: Microsoft Showcase: Stream Insight

Here are slides/presentation @ SlideShare

An article @ MSDN Magazine

Official Microsoft introduction: Microsoft StreamInsight

Effectively analyze large amounts of event data streaming in from multiple sources. Derive insights from critical information in near real time by using Microsoft StreamInsight. Monitor, analyze, and act on data in motion and make informed decisions almost instantaneously.




An article in March 2012 MSDN Magazine (not on web yet),
appropriately written by Microsoft Research people,
mentions that is expected 50 Billion (!) network connected devices by 2020.
That is 7 networked devices for every human on the planet, all sharing data!
That is a lot of data! StreamInsight is a tool that could help.
They also mentioned using using Netduino hardware platform
and .NET micro edition for programming such "sensor" devices...

Related story... Trading on Stock Exchange (New York) is moving HUGE amounts of money
and it is highly time-sensitive. Being able to buy or sell a few milliseconds faster
makes millions of dollars difference. So, to get "closer to action" companies are installing
private optical cables for example between Boston and New York...

Microsoft StreamInsight is a tool for applications like that :)

No comments: