Tuesday, December 30, 2014

Netflix, Rx: millions of messages per second

Mantis: Netflix's Event Stream Processing System @ InfoQ
"The authors discuss Netflix's new stream processing system that supports a reactive programming model, allows auto scaling, and is capable of processing millions of messages per second."



Programming languages: salaries and demand

Programming languages - salaries and demand (October 2014) | Gooroo:

Overall salary vs demand

USA salary vs demand

Monday, December 29, 2014

lists: Essential Computer Programming Books

Six Essential Language Agnostic Programming Books - Scott Hanselman

Essential Language Agnostic Computer Programming Books @ Amazon by Hanselman

Amazon.com: Building A Solid Programming Core

OmniSharp - Cross platform .NET development

OmniSharp - Cross platform .NET development by OmniSharp
"You can use your editor PLUS get to use great features like Intellisense (not just Autocomplete), Add Reference, Format Document, and lots more. Develop anywhere, deploy anywhere (and to Azure!)"

"We now have plugins for these editors:
  • Sublime
  • Brackets from Adobe
  • Atom from GitHub
  • Emacs
  • Vim"
It is time to make programming languages, and documents, extensible and "semantic"...
Something like Lisp, with "more normal" syntax.

IoT: free Windows is like DOS

Microsoft’s IoT Developer Program Puts Windows on Little Boards | MAKE

"Since the Intel Galileo doesn’t support a display and has only 256 MB of RAM, it’s probably not surprising that you’re working with a stripped-down version of Windows on it as well. It’s clear that it’s an embedded version of Windows, but this one, like the version of Windows for small tablets, is free (as in beer).
So what does Windows look like without a user interface? It looks a lot like DOS, a throwback to the early days of PCs (though it’s not DOS; instead of COMMAND.COM, your command prompt is CMD.EXE).
...Microsoft has ported the Arduino/Wiring libraries to their Windows for the Internet of Things, so you’ll be using Visual C++ to write code against the Arduino API. It looks a lot like Arduino source..."
CMD.EXE

It would be much more effective to put node.js as a default tool with packages for integration with Windows and Azure... 

Skype (voice) Translator: speech to text, translation, text to speech

Skype Translator – How it Works – - Skype Blogs
"...first phase of the Skype Translator preview program which will kick-off with English and Spanish as the first two languages...

Skype Translator is the result of decades of research in speech recognition, automated translation, and general machine learning technologies, combined with an intense focus on the user experience. This next phase in the Skype Translator journey is an exciting milestone and we’re looking forward to sharing it with those who use Windows 8.1 and have signed-up for the Spanish language via the Skype Translator sign-up page."

translatorinfographic

Windows installer based on C#

Using C# and Wix# to Build Windows Installer Packages
"Wix# (WixSharp) is a deployment authoring framework targeting Windows Installer (MSI). Wix# allows building complete MSI setups from a deployment specification expressed with C# syntax. The typical Wix# source file for building MSI contains plain-vanilla C# code, which uses a C# class structure to define (mimic) WiX entities."

(Convenient, but possibly inefficient...)

Sunday, December 28, 2014

DB vs. Semantic Web: Closed vs Open World Assumption

A random find about important issue of modeling of data by using relational model and "semantic web". Relation model is mathematically clean, rigid and predictable, while alternative is fuzzy...
Trouble is, real world data are fuzzy. A complex theory behind, in part based on technological limitations of the time... For semantic data to be usable, some constraints are needed.


The Closed World Assumption: O'Reilly - Safari Books Online (video, subscription required)
By: C.J. Date
"The Closed World Assumption (CWA) is an extremely important concept in the database world, despite the fact that it isn’t usually spelled out explicitly. Basically what it says is this: 
Everything stated by the database, either explicitly or implicitly, is true; everything else is false. 
This presentation explains the CWA in detail and shows why it’s preferred over its rival,The Open World Assumption (OWA). In particular, it examines the claims, sometimes heard, to the effect that the database community operates under the CWA while the semantic web community operates under the OWA. It also explains how “missing information” can be handled without any need for nulls or three-valued logic."

from the presentation, about relational model, based on Ted Codd
"TYPES are sets of things we can talk about; 
RELATIONS are (true) statements about those things!"
  1. Types and relations are both necessary 
  2. Thy are not the same thing
  3. They are sufficient (as well as necessary)
So database is a "named container for relational variables", a "logical system", 
with axioms (tupples), rules of inference and theorems (queries)".

"Integrity is most important feature of databases"



The Chris Date Seminar: The Closed World of Databases meets the Open World of the Semantic Web: Event Material

The Third Manifesto
"by Hugh Darwen and C.J. Date, our proposed foundation for future database systems"
(and language "D")

The Closed World of Databases Meets the Open World of the Semantic Web - ESIWiki
"...The important issue of missing information is something that can be interpreted differently between the CW and OW therefore how to deal with it and in fact the question of what is missing information will be explored..."
Open-world assumption - Wikipedia, the free encyclopedia
"In a formal system of logic used for knowledge representation, the open-world assumption is the assumption that the truth value of a statement may be true irrespective of whether or not it is known to be true. It is the opposite of the closed-world assumption, which holds that any statement that is true is also known to be true.

The open-world assumption (OWA) codifies the informal notion that in general no single agent or observer has complete knowledge, and therefore cannot make the closed-world assumption. The OWA limits the kinds of inference and deductions an agent can make to those that follow from statements that are known to the agent to be true. In contrast, the closed world assumption allows an agent to infer, from its lack of knowledge of a statement being true, anything that follows from that statement being false."

The Open World Assumption: Elephant in the Room | AI3:::Adaptive Information
'...open world assumption with facile summaries such as it means that a lack of information does not imply the missing information to be false..."

d3.js data visualization: sleep in USA

As expected, personal (fitness) big data are being visualized for new insights... Using d3.js.
It just may be that data are not typical, since those who wear Jawbone may not be typical.

In addition to d3.js, the page is using JavaScript library Highcharts - Highmaps JavaScript maps
that is "free for non-commercial use" with Creative Commons — Attribution-NonCommercial 3.0 Unported — CC BY-NC 3.0 license.

Which Cities Get the Most Sleep? - The Jawbone Blog
UP24 Navy Blue

This Map Claims to Show When People Go to Sleep - ABC News
PHOTO: Average bedtime by U.S. county, according to Jawbone UP users.
LOOK: What Time Does America Go To Bed? @ huffingtonpost

Saturday, December 27, 2014

IoT: programming language?

With focus on efficient sensors for IoT, C++ may become a language of choice for small "things".
Apparently both Intel and Microsoft think so.
In particular with new C++11 and C++14 features, the language may be usable (enough).

Getting Started for C/C++ (Eclipse) - Galileo & Edison @Intel

NuGet Gallery | Microsoft IoT C++ SDK 1.0.0
NuGet Gallery | Galileo C++ SDK 1.0.6

Another portable alternative,  likely to be more popular is node.js (or its new "fork" io.js).
Javascript and the Internet of Things- Postscapes
Building an IoT Project with Intel Galileo and Node.js | Dr Dobb's
Intel(R) XDK IoT Edition node.js Templates | Intel® Developer Zone

Apple Swift could be very interesting, since it is both efficient and high-level modern language,
is supported by Apple, with smart people and virtually unlimited resources...
When Apple starts making IoT gadgets, that may be a good choice.
HomeKit - Apple Developer
HealthKit - Apple Developer

Google's Go is less efficient for small devices, and is optimized for parallel programming.

Finally, obvious candidates are Java and C#,
since they could reasonably run on many platforms,
but GC (memory management) makes them less usable for "real time"...

Great diversity...






VS2015: Bower, Grunt

Manage Client-Side Web Development in Visual Studio 2015, Using Grunt and Bower | The ASP.NET Site
With Visual Studio 2015, you can:
  • Easily manage client-side packages like jQuery, Bootstrap, or Angular.
  • Automatically run build tasks such as LESS compilation, JavaScript minification, JSLint, or JavaScript unit tests.
  • Take advantage of a rich ecosystem of tools created by the web development community.
To enable these scenarios, Visual Studio 2015 Preview has built-in support for several popular third-party tools:
  • Bower. Described as a “package manager for the web,” Bower lets you install and restore client-side packages, include JavaScript and CSS libraries. For server-side libraries like the MVC 6 framework, you will still use NuGet Package Manager.
  • Grunt and Gulp. Grunt and Gulp are JavaScript-based task runners. If you aren’t familiar with these, a task runner is an app that automates routine development tasks. Right now, the ASP.NET 5.0 project template uses Grunt.
  • npm (Node Package Manager). npm is a package manager that was originally created for Node.js. Bower, Grunt, and Gulp all use npm.


ASP.NET 5: MVC 6 Web API, Entity Framework 7

Getting started with ASP.NET 5 MVC 6 Web API & Entity Framework 7 - Bit of Technology
"One of the main new features of ASP.NET 5 is unifying the programming model and combining MVC, Web API, and Web Pages in single framework called MVC 6. 

In previous versions of ASP.NET (MVC 4, and MVC 5) there were overlapping in the features between MVC and Web API frameworks, but the concrete implementation for both frameworks was totally different, with ASP.NET 5 the merging between those different frameworks will make it easier to develop modern web applications/HTTP services and increase code reusability."
(and limited backward compatibility)

Startup

ScottGu's Blog - Introducing ASP.NET 5
Monday, February 23, 2015




IoT + Azure + Mobile demo, Hanselman @ Connect()

Cloud development with Azure and Visual Studio | Connect(); Microsoft Visual Studio vNext & Azure | Channel 9
Scott Hanselman, November 12, 2014


MyShuttle.biz demo apps from Connect(); Visual Studio and Azure event - Cesar de la Torre [Microsoft] - BLOG - Site Home - MSDN Blogs

image

IoT, W3C: "Web of Devices"

Web of Devices - W3C

Thursday, December 25, 2014

Dashboarding: Data Analysis

Dashboarding: The Developers’ Role in Data Analysis @ InfoQ
by Seth Juarez on Dec 06, 2014

Visual display 
of
the most important information needed to achieve
one or more objectives
which
fits entirely on a single computer screenso it can be
monitored at a glance


Categorizing dashboards:
  • strategic – long term decisions
  • analytical – intermediate decisions, interactive, explorative
  • operational – Immediate decisions
12 mistakes:
  • too big
  • missing context
  • excessive detail
  • deficient measure
  • inappropriate visualization
  • meaningless variety
  • poorly designed visualization
  • large lie factor
  • poor arrangement
  • bad highlighting
  • chartjunk
  • unattractive display
Simplicity works best
  • reduce non-data pixels (data-ink ratio)
  • eliminate all unnecessary non-data pixels
  • de-emphasize and regularize the non-data pixels that remain
highlight appropriately
  • pre-attentive attributes of visual p
  • – color – hue / intensity (HSL)
  • – position – x, y location
  • – form – orientation, line length, line
  • shape, added marks, enclosure
  • – motion – Me! Me!




Business Dashboard Fundamentals – Pluralsight Training

AngularJS + SVG = Radian, (without D3.js)

Radian - Home
"Radian is an open source JavaScript library that makes it easy to embed plots in HTML documents. It is being developed as part of OpenBrain’s BayesHive project."


<svg ng-attr-height="{{graph.height}}" ng-attr-width="{{graph.width}}"> <circle ng-repeat="circle in circles" ng-attr-cx="{{circle.x}}" ng-attr-cy="{{circle.y}}" ng-attr-r= "{{circle.r}}"> </circle> </svg>

$scope.circles = [ {'x': 15, 'y': 20, 'r':30}, {'x': 35, 'y': 60, 'r':20}, {'x': 55, 'y': 10, 'r':40}, ]



Web Components, Polymer

WebComponents.org
a place to discuss and evolve web component best-practices
Welcome - Polymer
Web Components usher in a new era of web development based on encapsulated and interoperable custom elements that extend HTML itself. Built atop these new standards, Polymer makes it easier and faster to create anything from a button to a complete application across desktop, mobile, and beyond.






basic-calendar-month – Component Kitchen
<basic-calendar-month></basic-calendar-month>

Small form factor for IoT?

Major computing platforms:

  • PC: Intel
  • Mobile: ARM
  • IoT: ?
Smaller does not always mean slower or more affordable... but often it is. 
Expected scale of IoT could justify another platform shift, in HW and SW...


Small form factor - Wikipedia, the free encyclopedia


Next Unit of Computing - Wikipedia, the free encyclopedia
Amazon.com: intel nuc: Electronics

OptiPlex 9020 Micro PC Desktop | Dell

Edison Getting Started Guide - Windows | Intel Communities

D3.js API reference

D3.js is a powerful library, not a framework, so it requires careful using of available APIs.
In fact it also requires solid knowledge of SVG and CSS, in addition to JavaScript.

API Reference · mbostock/d3 Wiki

Time Formatting · mbostock/d3 Wiki

It would be useful to have a standardized way for higher-level reusable modules,
to avoid re-inventing standard visual elements, something like c3.js but general-purpose...
Maybe with web components?

Data Visualization Libraries Based on D3.JS - Mike McDearmon

Dasku
dc.js graph

DataMaps


D3 Tutorial Table of Contents | DashingD3js.com


Wednesday, December 24, 2014

reveal.js for presentations (+ d3.js)



reveal.js - The HTML Presentation Framework

hakimel/reveal.js @ GitHib

Reveal.js tutorial | Reveal.js Step by Step for Beginners

Reveal.js – A Framework For Amazing HTML Presentation







Online IDEs

13 Cloud IDEs For Web Developers

Edit, Compile and Execute Programs Online| Online IDE @ Coding Ground
Code in almost all popular languages using Coding Ground.
Edit, compile, execute and share your projects, 100% cloud.


a really large number of programming languages!

What is the best online python IDE? - Quora

Ideone.com - Online Compiler and IDE >> C/C++, Java, PHP, Python, Perl and 40+ other compilers and interpreters

repl.it
Run, Save, and Share your code online, no installation needed
Python JavaScript Ruby

Try Python!

Python Cloud IDE | Python Fiddle

Visual Studio Online "Monaco" | Channel 9
"A series of short videos designed to help you get started with development in the Visual Studio Online "Monaco""

IoT Programming

Programming the Internet of Things with Android @ Linda.com (video course)
"The "Internet of Things" is a catchy way of describing the variety of devices connected through the Internet. This includes webcams, wearable tech like the Pebble and Android Wear watches, car sensors, appliances, and even rudimentary robots. With the Internet of Things (IoT), you can manipulate them from the web. In this course, Michael Lehman shows how to create "things" and build companion apps to monitor and record their activities from Android devices."

IoT is a favorite marketing term now, to sell everything related to machine communication...

Android (and iOS) apps may be a good way to control some devices,
but "real" IoT should be more open by using simpler, web based tools (and apps)


d3.js tax data visualization

What Do You Work For? | DataVizChallenge.org
"What if you paid your federal taxes with your entire income starting January first?
Where would your money go?"

.net: SecureString

SecureString Class (System.Security) @ MSDN
"Represents text that should be kept confidential. The text is encrypted for privacy when being used, and deleted from computer memory when no longer needed. This class cannot be inherited.

An instance of the System.String class is both immutable and, when no longer needed, cannot be programmatically scheduled for garbage collection; that is, the instance is read-only after it is created and it is not possible to predict when the instance will be deleted from computer memory. Consequently, if a String object contains sensitive information such as a password, credit card number, or personal data, there is a risk the information could be revealed after it is used because your application cannot delete the data from computer memory."

Writing Compilers with Phil Trelford @ .NET Rocks!

gadget, idea: wireless button "Flic"

Flic: The Wireless Smart Button | Indiegogo
$19/button, raised $450 K

gadget, idea: Microsoft Universal Mobile Keyboard

Amazon.com: Microsoft Universal Mobile Keyboard for iPad, iPhone, Android devices, and Windows tablets: Computers & Accessories $79
"Operating system switch allows you to easily switch between devices"

a clever way of holding a tablet in inclined position

Microsoft Universal Mobile Keyboard hands-on: Solid, but not quite universal | PCWorld
microsoft universal windows mobile keyboard ridges

Tuesday, December 23, 2014

IoT: .NET Gadgeteer (netduino)

.NET Gadgeteer - Microsoft Research
".NET Gadgeteer was created by researchers at Microsoft as an internal prototyping tool, but because of external interest, particualrly from educators and hobbyists, we turned it into open source software which now has a vibrant hardware ecosystem from multiple manufacturers.

The platform is built on the .NET Micro Framework, which allows small devices to be programmed in the C# language and make use of Visual Studio’s programming and debugging tools."










AngularJS 1.4

AngularJS 1.4 Announced, More to Come in 2015 @ InfoQ

"Work on AngularJS 1.4 has begun. After announcing their plans for version 2.0, the team has decided to advance version 1.x with new features and continue active development on it throughout 2015.
...


the team decided to limit the scope of version 1.4 in order to release it by ng-conf, the AngularJS focused conference on March 5, 2015. "I'd rather have a smaller release and get it done by ng-conf
...
1.4 will have some breaking changes for some developers."

So 1.3, 1.4, 2.0, all have breaking changes. With JavaScript it is all constant change :)

Sunday, December 21, 2014

JavaScript: Console API Reference

Console API Reference - Google Chrome

Example of console.assert()




Intel IoT Developer Kit

IoT - Just add Inspiration | Intel® Developer Zone:
"...is a complete hardware & software solution to help you explore the IoT space and create innovative projects. Here is everything you need to get going; a development board and starter kit including Yocto Linux system, Eclipse and XDK IDE, IoT Cloud Analytics, a set of libraries, and more (see below for details). Order either the Intel® Galileo or Intel® Edison board and the Grove Starter Kit Plus - Intel® IoT Edition."
Seeed Studio Bazaar, Boost ideas, Extend the Reach
$79.00

Saturday, December 20, 2014

mapsmarker, piwik - Web Analytics Reports

DataMaps.eu - Web Analytics Reports:

"Dashboard" with widgets, drag-drop maps, zoom-in/out







Piwik

Free Web Analytics Software

Open Analytics Platform
Piwik is an open source project (GPL)


d3.js: Angular Datamaps

dmachat/angular-datamaps · GitHub:

Provides an Angular directive to wrap https://github.com/markmarkoh/datamaps 
and easily build data maps in your Angular application.
Datamap example

Azure Security with Mark Russinovich

Play by Play: Azure Security with Mark Russinovich – Pluralsight Training:

"There are many security questions to answer when considering migrating from an on-premise infrastructure to the cloud. Is my data secure from bad guys? Is it safe from the cloud provider? Are they mining customer data? Once we move virtual machines into the cloud, is there anything we can do as administrators, or that the cloud provider can do to make our systems secure? In this Play by Play, we tackle these questions and more with Mark Russinovich, CTO of Microsoft Azure."



Mark Russinovich - Wikipedia, the free encyclopedia

Microsoft Bing 2015 Predictions: IoT, Cybersecurity, Machine Learning

Microsoft Business Predictions for 2015: IoT matures, cybersecurity bulks up, and machine learning gives businesses a big advantage - The Official Microsoft Blog:

"IoT continues to mature across a range of industry sectors, and companies are focused on the actionable potential IoT holds for businesses today. People are eager to understand and implement IoT in their businesses, a trend that’s reinforced by a 430 percent increase in searches for IoT on Bing in the last 12 months."


IoT: “Internet of Moving Things”

“Internet of Moving Things” Startup Eyes U.S. Cities – Next City:

"Many transit riders are familiar with onboard WiFi, but as of September, in Porto, Portugal, buses, taxis and even garbage trucks actually act as moving hot spots, supplying Internet access to around 60,000 people each month. 



This system, which creator Veniam calls an “Internet of moving things” is a kind of mesh network. Using an onboard device called a NetRider, vehicles communicate with both fixed infrastructure and other vehicles. The startup’s website depicts the widely distributed signals as a vast, city-wide spiderweb, with anyone in range able to log on."


Drone Videos

The 17 best drone videos that gave a whole new perspective to 2014 - Quartz





Friday, December 19, 2014

Interview with Jeff Bezos

When I Asked Jeff Bezos The Tough Questions -- No Profits, The Book Controversies, The Phone Flop -- He Proved Why He's A Genius CEO | Business Insider:

"Experiments are, by their very nature, prone to failure. A few big successes compensate for dozens and dozens of things that didn’t work.
...
What really matters is, companies that don’t continue to experiment, companies that don’t embrace failure, they eventually get in a desperate position where the only thing they can do is a Hail Mary bet at the very end of their corporate existence. Whereas companies that are making bets all along, even big bets, but not bet-the-company bets, prevail. I don’t believe in bet-the-company bets. That’s when you’re desperate. That’s the last thing you can do."

Atlas: Netflix's Primary Telemetry Platform

The Netflix Tech Blog: Introducing Atlas: Netflix's Primary Telemetry Platform:

...platform and ecosystem... for time-series telemetry and its capabilities
and announce the open-sourcing of its underlying foundation

  • In-Memory Storage
  • Engineering for Performance
  • Engineering for Resilience
  • Engineering for Cost

IoT: sensors, networks, distributed computing

Cheap sensors, fast networks, and distributed computing - O'Reilly Radar (free report)

"The trifecta of cheap sensors, fast networks, and distributing computing are changing how we work with data. But making sense of all that data takes help, which is arriving in the form of machine learning. Here’s one view of how that might play out."

Data_Emerging_Trends_Tech_COMP_freedownloadbanner

ideas: replacing rearview mirror with camera (Acura)

Cadillac replacing rearview mirror with camera:
“In addition to the increased field of view, the technology eliminates any rear seat, rear pillar or passenger obstructions, allowing the driver an unimpeded view of the lanes behind and traditional blind-spots
...
Thanks to a high dynamic range, the camera’s video feed reduces glare and allows a crisper image in low-light situations, versus a traditional glass electrochromatic, or auto-dimming, rearview mirror,” Cadillac noted in a release. “The in-mirror display is an industry-leading 1280 by 240-pixel TFT-LCD display with 171 pixels per inch, combined with a HD camera designed specifically to enhance rear view lane width and maximize low-light situations.”


.NET: Speech Recognition and Synthesys

Voice Recognition - Speech Recognition with .NET Desktop Applications:
"... how to get started with speech recognition and speech synthesis in Windows console applications, Windows Forms applications, and Windows Presentation Foundation (WPF) applications.

...you can also add speech capabilities to Windows Phone apps, ASP.NET Web apps, Windows Store apps, Windows RT apps and Xbox Kinect, but the techniques to do so are different from those presented in this article."

Microsoft Speech Platform (downloads)


A nice article with step-by-step guide for making Windows voice apps.
The setup is a non-trivial.
It is interesting to compare this with cloud-based (Google Now) or mobile (Siri, Cortana) agents,
that just show in web browser or in a smartphone app, and "just work" (or entertain :)


Thursday, December 18, 2014

Google in the news

Google Is Going Through A Rough Transition -- And There's Some Pessimism Inside The Company | Business Insider:
"People are searching for products on Amazon, rather than using Google. The only reason search makes money for Google is that people use it to search for products they would like to buy on the internet, and Google shows ads for those products. Increasingly, however, people are going straight to Amazon to search for products. Desktop search queries on Amazon increased 47% between September 2013 and September 2014, according to ComScore."

(note: Amazon CEO is investor in Business Insider)

Google Is Now Worth More Than the Entire Russian Stock Market | TIME

Google to Pull Engineering Operations in Russia

The rise and rise of Sundar Pichai in Google (and Larry Page is loving it) : BIZ TECH : Tech Times

Here's Everything You Need To Know About Sundar Pichai, The Second Most Important Person At Google | Business Insider India

The Google Way of Attacking Problems - HBR

Google's 2014 Holiday Gift: An LG Watch - Business Insider
lg g watch