Big Data in Memory

Storing data in custom binary format can significantly reduce storage and increase performance.

This is in particular visible with managed VM languages, like Java (48 Bytes for string "ABC" !!!)

With availability of large-memory computers, big data could fit in memory of a single machine...

Java for Low Latency - You’ve Got to Be Joking!
by John Davies

Big Data in Memory @ InfoQ
by John Davies

Big Data in Memory @ SlidesShare





Gartner's 19 In-memory Databases for Big Data Analytics

VisualStudio Code

Visual Studio Code - Code Editing. Redefined.

video course online: Visual Studio Code – Pluralsight Training
by John Papa
sample code: johnpapa/pluralsight-vscode-samples

presentation: Visual Studio Code: A Deep Dive on the Redefined Code Editor for OS X, Linux and Windows | Build 2015 | Channel 9

by Chris Dias, Erich Gamma, John Papa

interview: Meet the Visual Studio Code team | Build 2015 | Channel 9

by Chris Dias, Erich Gamma, Seth Juarez

Erich Gamma - Wikipedia, the free encyclopedia

"Erich Gamma is a Swiss computer scientist and co-author of the influential software engineering textbook, Design Patterns: Elements of Reusable Object-Oriented Software. He co-wrote the JUnit software testing framework with Kent Beck and led the design of the Eclipse platform's Java Development Tools He also worked on the IBM Rational Jazz project.

He joined the Microsoft Visual Studio team in 2011 and leads a development lab in Zürich, Switzerland that has developed the "Monaco" suite of components for browser-based development, found in products such as Visual Studio Online, Visual Studio Code, Azure Mobile Services, Azure Web Sites, and the Office 365 Development tools."

In-memory Databases, OrigoDB

podcast: Herding Code 209: Robert Friberg on In-memory Databases and OrigoDB

Robert Friberg (https://about.me/robertfriberg, @RobertFriberg)

OrigoDB

Video: IMDB showdown – comparing OrigoDB, Redis and SQL Server Hekaton

Slides: IMDB Showdown - OrigoDB, Redis and Hekaton

Devrex Labs @ GitHub

Protocol Buffers

SQL Server In-Memory OLTB (Hekaton)

Redis

VoltDB

Web Sites Performance Links

podcast: Mobile Web Sites with Chris Love @ .NET Rocks!

WebPageTest 

Steve Souders

PerfBudget on Grunt kraken.io 

 
HTTP Archive Trends 

What Does my Site Cost

Github's Top Coding Languages

Github's Top Coding Languages Show Open Source Has Won | WIRED

data optimization: Traveling 48 States

one year family trip in RV, while working remotely:

48StatesProject.com MAP

"...travel through all of the lower 48 states in one year's time"

by Jonathan “J.” Tower

podcast interview: Yet Another Podcast – Show 140: J. Tower | Jesse Liberty

AngularJS | Enterprise Developers Guild

The Definitive Road Trip? It's Data-Driven : NPR
"Randy Olson, a Ph.D. candidate at Michigan State University and a self-proclaimed "data tinkerer," believes he's devised a route that could allow a family to hit a landmark in each of the Lower 48 states...

Travelling salesman problem - Wikipedia, the free encyclopedia
"The travelling salesman problem (TSP) asks the following question: Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city? It is an NP-hard problem in combinatorial optimization, important in operations research and theoretical computer science."

visualization: The Age of Homes

Following is an interesting visualization map, but it could be misleading
because of using random color to represent ranges of values...
Big data needs smart visualization techniques... but what is a good way?

The Age of Homes in All 50 States | Zillow Blog
"...single-family houses built from 1900 to 2014 to see which decades are most represented by the current housing stock..."


I.e., what is a good way to color-code ranges of values on a chart or a map?

Diverging Color Maps for Scientific Visualization

... mapping scalar values to colors... allows us to view scalar fields by coloring surfaces and volumes. ... rainbow color map... which naively sweeps through the most saturated colors, is well known for its ability to obscure data, introduce artifacts, and confuse users.

Vis Lies 2013 – The Blog of Dr. Moron

 bad vs. good  

Based on this observation, next visual map is still not optimal,
while better than just using random colors as on the first map. 

Climate Map | Meandering Through Mathematics

elections are a typical example of using color-maps. 

In that case two main colors are easy choice, 

but different population density creates another challenge: 

colored surface not proportional to number of values. 

Geographical hotspot analysis for election data | The SAS Training Post

A better way is to utilize "bubble map"

Here is one example from Microsoft "Power Map" reporting tool. 

Better, but still not perfect. 

Microsoft Power Map | Reporting content from SQL Server Pro

In all, color map visualization challenge appears to be not solved still. 

If there is a good solution, it is hiding well from Google :)

IoT Windows 10 AllJoyn with Visual Studio 2015

Jumpstart your Win10 IoT development with the AllJoyn Studio for Visual Studio 2015 | Coding4Fun Blog | Channel 9


of all house devices, why is a toaster so important?

Markdown with Visual Studio Code

Markdown editing with Visual Studio Code



Alternative: https://atom.io/ editor

markdown-preview

node.js resources

nodejs.org

Nodeschool.io

Build and run anode.js app on Azure

Node.js Tools for Visual Studio

socket.io Chat Application

links from: Intro to Node.js - MS Dev Show Podcast

IoT security: Hacking Amazon's WiFi Button

Amazon's has created a very clever and useful Dash Button gadget for easy re-ordering.
The communication protocol is based on WiFi, and somebody already figured out
how this could be intercepted to reuse the button for other purposes,
by just configuring router differently, and then respond to event locally.

This may be an opportunity for Amazon (or somebody else?)
to sell similar buttons with SDK as a general purpose IoT tool.
There will be many places where this could be very useful.

How I Hacked Amazon’s $5 WiFi Button to track Baby Data — Medium



mentioned on podcast: Intro to Node.js - MS Dev Show Podcast

desktop web app: GitHub Desktop

Another hybrid desktop app using embedded web browser for GUI

GitHub Desktop



podcast: GitHub Desktop with Amy Palamountain @ .NET Rocks!

tool, replacement for "ClickOnce" on Windows:

Squirrel
Server-driven updates for native (desktop) apps

Cloud-Native Application Architectures

Coding in a cloud-based enterprise - O'Reilly Radar
Mapping the future of development by designing for distributed architectures.

Migrating to Cloud-Native Application Architectures

The cloud-native future - O'Reilly Radar
"Moving beyond ad-hoc automation to take advantage of patterns that deliver predictable capabilities."

Windows 10 VS 2015 Project Templates

Visual Studio 2015 comes only with "blank" Windows 10 app template....
No time, or to get more variety of app designs?
Here is one solution, from "community":

Go beyond "Blank App" with these Windows 10 VS 2015 Project Templates | Coding4Fun Blog | Channel 9

"UWP: new VS2015 project templates

• Blank App, a basic XAML project with a single XAML page. Similar to the standard Blank App template, included in the Windows 10 tools, but without the Application Insights telemetry stuff.
• SplitView App, a multi-page app project with adaptive split view layout
• Composition App, a framework-less project for creating apps using the visual layer and Windows.UI.Composition APIs
• Composition XAML App, a project for both XAML and Windows.UI.Composition APIs

The UWP Templates extension supports Visual Studio 2015 Community Edition and up. The extension is available in the Visual Studio Gallery. The full source code of the extension is available at GitHub."

JsonML

JsonML - Wikipedia, the free encyclopedia

"JsonML, the JSON Markup Language is a lightweight markup language used to map between XML(Extensible Markup Language) and JSON (JavaScript Object Notation). It converts an XML document or fragment into a JSON data structure for ease of use within JavaScript environments such as a web browser, allowing manipulation of XML data without the overhead of an XML parser."

JSON:

{"person": {
    "address": {
        "city": "Anytown",
        "postalCode": "98765-4321",
        "state": "CA",
        "street": "12345 Sixth Ave",
        "type": "home"
    },
    "created": "2006-11-11T19:23",
    "firstName": "Robert",
    "lastName": "Smith",
    "modified": "2006-12-31T23:59"
}}

JsonML (Array JSON), to preserve XML's elements order:

["person",
  {"created":"2006-11-11T19:23",
   "modified":"2006-12-31T23:59"},
  ["firstName", "Robert"],
  ["lastName", "Smith"],
  ["address", {"type":"home"},
    ["street", "12345 Sixth Ave"],
    ["city", "Anytown"],
    ["state", "CA"],
    ["postalCode", "98765-4321"]
  ]
]

<!-- XML representation of a person record -->

<person created="2006-11-11T19:23" modified="2006-12-31T23:59">
<firstName>Robert</firstName>
<lastName>Smith</lastName>
<address type="home">
<street>12345 Sixth Ave</street>
<city>Anytown</city>
<state>CA</state>
<postalCode>98765-4321</postalCode>
</address>
</person>

---

Guide to Blockchain Technology

6-part tutorial about Blockchain (Bitcoin) technology

A Complete Beginners Guide to Blockchain Technology - Blockstrap Blockstrap

IoT: node.js with Microsoft Edge Chakra engine

CodeChat 035 - Node Chakra | CodeChat | Channel 9

"...a version of Node.js that works with Chakra - the JavaScript engine that underlies Microsoft Edge.
Why would you do this? Well, because ... Windows 10 is not exactly discriminant about which devices it supports... To really work everywhere there are times when the V8 engine isn't going to work.

They're doing it right too. They're not creating a proprietary branch - a road you can't come back from. The newfound versatility they're putting into Node.js is on its way to being contributed right to the main project. That's why this Node.js thing is so awesome - because such a rich community is contributing to it.

• Node+Chakra Blog - http://aka.ms/node-chakra-annoucement-blog
• Installer for Node.js Tools for Windows IoT - https://github.com/ms-iot/ntvsiot/releases/tag/v1.1
• Node+Chakra on Github - https://github.com/Microsoft/node
• IoT Getting Started - http://aka.ms/iot-getstarted
• IoT Samples - http://aka.ms/iotsamples "

Node.js and io.js are settling their differences, merging back together
Node.js and io.js will officially merge codebases under a new entity: the ‘Node Foundation.’
A number of partners have come onboard to sponsor the new organization and help it stay independent, including Famous, IBM, Intel, Joyent, Microsoft and PayPal.

Node.js and io.js Merge Under the Node Foundation @ InfoQ

"The io.js TC has voted to join the upcoming Node Foundation. The project will be named Node.js but it will be based on io.js’ repository."

Essential Steps: Long Term Support for Node.js | NodeSourcee

Windows 10 Dev Guide @ Channel 9

Developer’s Guide to Windows 10 | Channel 9

"...a two day whirl-wind tour of the Windows 10 developer platform

... will give you detail on everything you need to know in order to build compelling application experiences on Windows 10. You'll leave fully prepared to build an outstanding app on Windows 10"

web dev tutorials: Azure DocumentDB, Aurelia+node.js

tutorial: Build a web application with ASP.NET MVC using DocumentDB | Microsoft Azure



DocumentDB - NoSQL data management | Microsoft Azure

podcast interview: Diving into Aurelia with Julie Lerman @ .NET Rocks!
using Entity Framework (EF6/EF7) to connect to NoSQL DocumentDB database also.

ASP.NET 5 @ GitHub

Alternative: using Aurelia for web UI with data binding to Web API

tutorial: zewa666/aurelia-node  @ GitHub
"This is a NodeJS Express App bundled with the Skeleton App of the Aurelia platform. It sets up a standard navigation-style app using gulp to build your ES6 code with the babel compiler. Karma/Jasmine testing is also configured.
For more info please visit the official site: http://www.aurelia.io/"

tools: Automating VM Deployment

podcast interview: Automating Deployment with Matt Wrock @ .NET Rocks!

"... a completely automated process to build new VMs with services, tools and your software all pre-configured and ready to go...."

• BoxStarter 
• Chocolatey 
• Microsoft Deployment Toolkit 
• Vagrant 
• Atlas 
• Packer 
• Boxcutter
• NuGet

web marketing: Business vs Technical

7 TIPS FOR WRITING WEB COPY FOR THE BUSINESS VS THE TECHNICAL DECISION MAKER

• "Don’t confuse offerings and audience:
  business decision maker (BDM) (save money, or be proud of choices)
  vs technical decision maker (TDM) (solve challenges, interesting technology)
• Use consistent terminology:
  use the same word to describe them in other sections
• Industry jargon confuses the BDM. 
• Use original pictures not technical renderings. 
• Clean up old content
• White space is a good thing
• Web site maps are a specific section of the web site."

podcast interview: Talking to Decision Makers with Eileen Fisher @ .NET Rocks!

"how do you get to "yes" with business decision makers?"

Windows Server 2016 Containers Preview (Docker)

ScottGu's Blog - Announcing Windows Server 2016 Containers Preview
"...release the first preview of Windows Server Containers as part of our Windows Server 2016 Technical Preview 3 release. We’re also announcing great updates from our close collaboration with Docker, including enabling support for the Windows platform in the Docker Engine and a preview of the Docker Engine for Windows

...

Directly from the Azure Marketplace, users can now deploy a Windows Server 2016 virtual machine pre-configured with the container feature enabled and Docker Engine installed
... quick start guide has all of the details including screen shots and a walkthrough video so take a look here https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/azure_setup.

Containers: Docker, Windows and Trends | Microsoft Azure Blog
by MARK RUSSINOVICHCTO, Microsoft Azure

Containers 101 with Microsoft and Docker | The Containers Channel | Channel 9

tool: RestSharp

RestSharp - Simple REST and HTTP Client for .NET

Simple REST and HTTP API Client for .NET

GitHubrestsharp/RestSharp @ GitHub

Twitter as Semantic Web platfrom

An interesting analysis of usage of Twitter @ and # identifiers to extract meaning from messages.

Semantics of Social Media | Kurt Cagle | LinkedIn

Lack of #DataScientists poses @SKA_telescope problem:http://www.itweb.co.za/mobilesite/index.php?option=com_content&view=article&id=145278 … #BigData #Astronomy#DataScience (see also @LSST)

This entry (chosen pretty much at random) has the following characteristics:
@user-references (@SKA_telescope)
#topical-references (#BigData)
shortened links (http://goo.gl/YrvpMd)
Account originator (@KirkDBorne)
Message identifier (not shown, but every twitter message has one, typically a GUID: f9732f07-ab02-46d4-b869-215e36576dc5, which I'll shorten to f9732f07)
A time stamp (not shown)
Retweet-reference (not shown, but typically an internal link to a previous tweet).
Additional human text message.

---

After many years it is clear that good ideas of Semantic Web could not be utilized without platform.
There is no much value for standard without tools that support it, and both standards and tools
need to evolve quickly, as "classic web" did.

Semantic Web related technology is used used by Google and others, but data on the web are still very unstructured and difficult to process. It may be time for "data smart-er web..."

Windows 10 Virtualized Security: "Isolated User Mode"

Finally, visualization technology is being used to isolate individual processes...

Isolated User Mode in Windows 10 with Dave Probert | Seth Juarez | Channel 9
"... interesting security issue ... how do we protect ourselves (process) from kernel exploits? The solution he lays out (and actually forms part of Windows 10) is genius and lays the foundation for some truly interesting innovations down the road..."

User mode and kernel mode (Windows Drivers) @ MSDN

Desktop Web Apps with Chrome

Starting Google Chrome in application mode - Super User

from the command line:

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --app=http://google.com

How to Create Application Shortcuts in Google Chrome for Mac


What Are Chrome Apps? - Google Chrome

• "Chrome Apps can integrate seamlessly into the desktop and look more like desktop applications than traditional web apps.
• Chrome Apps for Desktop have no omnibox (address bar) and tab strip like normal browser-based apps, because like native desktop apps, they don’t live in a browser.
• The Chrome App Launcher makes it easy for users to find and start your Chrome App.
• OS X users can use the integrated spotlight search to find and start your app.
• Chrome Apps for Desktop can access the host computer’s file system and make use of hardware features (like USB, Bluetooth and attached human interface devices)."

How to Turn Web Apps Into First-Class Desktop Citizens

videos: Clean Coders by Bob Martin

Clean Coders by Bob Martin

"Training Videos.With Personality.For Software Professionals."

Live from That Conference - MS Dev Show Podcast

52 Weeks of Xamarin by Jesse Liberty

52 Weeks of Xamarin – Week 1 | Jesse Liberty

"This launches 52 Weeks of Xamarin, an opinionated series in which (Jesse) will post (at least) one instructional blog post on Xamarin a week. In this series I’ll be talking about Xamarin.Forms as well as “Xamarin-native” iOS and Android."

Xamarin.Forms and Data – Pluralsight Training by Jesse Liberty

Jesse Liberty | LinkedIn

(Isomorphic) Universal JavaScript = react.js + node.js @ Netflix.com

Isomorphic JavaScript - The future of web app development

"Isomorphic JavaScript apps are JavaScript applications that can run both client-side and server-side. The backend and frontend share the same code."



Netflix.com Adopts Universal JavaScript, Drops Java From Rendering Pipeline
"The team behind Netflix.com has switched from a Java-based HTML renderer to a JavaScript one that they can run on both the server and the client.
.

To decrease the time users spent waiting, Netflix has removed the Java server-based renderer and reduced the amount of HTML sent down the wire. In a post describing the change, Kristofer Baxter, said that by not rendering pieces of the app that the user will never see (among other changes), they've been able to speed up the time til interaction by 70%.
...
"Universal JavaScript", Netflix architected their new UI renderer so that they can run it in the browser or in a node.js server.
...
While Netflix has chosen React.js to provide them with this ability, it isn't the only option.
..."

The Netflix Tech Blog: Making Netflix.com Faster

Universal JavaScript — Medium

Go Lang: Gopher Academy @ YouTube

Gopher Academy - YouTube


Building Resilient Services with Go – Tech Talk - Fog Creek Blog
SSH reverse proxy written in Go


Go compiler re-write in Go Lang

IoT: Agriculture 3.0

Farming Goes High-Tech | On Point with Tom Ashbrook


The Internet of Things and the Future of Farming - The New York Times
"...Venture capitalists seem to share some of Mr. Donny’s optimism. In the first half of this year, venture investment in so-called agtech start-ups reached $2.06 billion in 228 deals..."

Feeding 9 Billion | National Geographic
Nice web presentation...

Home - OnFarm
software tool for monitoring and automation of agriculture

ReactiveX: Reactive programming links

Many links about Reactive Extension / Programming on many platfroms:

ReactiveX - Links to More Information

ReactiveX

"An API for asynchronous programming with observable streams"

Introduction to Rx : a free, on-line book by Lee Campbell

Reactive Android @ InfoQ

Reactive programming - Wikipedia, the free encyclopedia

Windows 10: change lock & login screen background

How to Customize the Lock Screen on Windows 8 or 10



How to Change the Login Screen Background on Windows 10

Change the Windows 10 log-in screen background to a solid color - CNET

regedit.exe =>
HKEY_LOCAL_MACHINE > Software > Policies > Microsoft > Windows > System

New > DWORD (32-bit) Value  "DisableLogonBackgroundImage" = 1

to change the color of this new log-in background, just go to

Settings > Personalization > Colors and pick a new "accent color".

How to Change or Disable Login Screen Background Image in Windows 10? - AskVG

Docker for Windows

Get Started with Docker (for Windows)

Install Docker for Windows
"Your machine must be running Windows 7.1, 8/8.1 or newer to run Docker Toolbox. Windows 10 is not supported (yet)"
...
Because Docker relies on Linux-specific features, you can’t run Docker natively in Windows. Instead, you must install the Docker Toolbox application. The application installs a VirtualBox Virtual Machine (VM), Docker itself, and the Docker Toolbox management tool. These three things allow you to run Docker on Windows.

Docker Client for Windows is Now Available | Microsoft Azure Blog

IoT: Google + Carnegie-Mellon + ...

A Look Inside Google and Carnegie-Mellon's IoT Campus | Fast Company | Business + Innovation
"For the "Internet of Things" to thrive, all it needs is for all devices to get along—which is currently wishful thinking. Last week, however, Google announced a partnership with Carnegie-Mellon University, which is leading a collaboration of faculty from several other academic institutions on a project to jump-start the Internet of Things revolution. Their plan: Build a universal platform that lets any device talk to any other device. And fittingly, that master-key solution will be open source.

...

The joint project between CMU, Cornell, Stanford, Illinois at Urbana-Champaign, and Google wants to wipe away the private-industry middlemen that keep sensors in separate sandboxes by creating a new, open platform: GIoTTO."

CMU Leads Google Expedition To Create Technology for “Internet of Things”

“We funded the Open Web of Things expedition to encourage universities to explore various aspects of system design that could help enable the Internet of Things,” said Maggie Johnson, director of university relations for Google. “From the many excellent proposals received, we’ve chosen Carnegie Mellon to lead because of their vision for a living laboratory, validating system design through daily use. Cornell, Illinois and Stanford were selected to join based on their unique approaches for tackling critical challenges related to privacy & security, systems & protocols and HCI."

""Snap2It" lets users link to a printer or projector simply by taking a smartphone photo of it."

CMU turning campus, Pittsburgh into Internet of Things testbed - Pittsburgh Business Times

"...To create its living lab, CMU will saturate its campus with sensors and infrastructure, recruit student and other campus members to create and use new IoT apps, and eventually expand efforts to the wider Pittsburgh community..."

IoT: Singapore 2.0: "Smart Nation" bet

Singapore 2.0: Lee Seeks Smart City Revamp as Old Model Ebbs - Bloomberg Business
"For 50 years, Singapore has punched above its weight class, thanks to a run of political stability, long-term planning, transparency and openness to investment. Along the way, the tiny Southeast Asian nation turned from a center of colonial administration and trading into a major container port, an oil-refining hub, an electronics manufacturer, and a banking center.
...
Singapore’s traditional pillars of growth are cracking. Electronics exports have repeatedly declined in recent years, labor productivity fell for a fourth quarter in the first three months of the year and the economy contracted
...
administration committed S$16.1 billion (US $11.5) to research and development from 2011 to 2015, a 20 percent increase over the previous period

...

A glimpse of that vision can be seen in Jurong Lake District, a 360-hectare development area in the island’s west where the government has installed more than 1,000 sensors to monitor and control everything from vehicles to trash cans.

Microservices vs. Code

Microservices: Software that Fits in Your Head @ Info

"Dan North describes a model for thinking about the age of code and argues for replaceability as a first class concern. He also discovers that by optimizing for both replace-ability and consistency one can end up with something that looks a lot like microservices."slides: Microservices: software that fits in your head // Speaker Deck

"What is the goal of software development?

To sustainability minimize lead time to (positive) business impact.

The goal is not to produce software!

...

Code is not the asset…

- writing code costs 

- waiting for code costs 

- changing code costs 

- understanding code costs

Code is the cost!

...

Microservices can be a 

Replaceable Component 

Architecture

- if you choose to optimize for 

replace-ability and consistency 

- smaller is not necessarily better 

- more replaceable is better

text of slides: Microservices: software that fits in your head - SSSSLIDE

book: Software, Faster by Dan North [Leanpub PDF/iPad/Kindle]

Microservices @ MartinFowler


book: Building Microservices - O'Reilly Media