Thursday, June 01, 2017

AI: How Deep Neural Networks Learn, Visually

Neural Networks can have many layers, and every layer builds on results of previous layer(s).
So every layer has a purpose for processing data, but usually this is a "blackbox".
Recent scientific results reveal some inner workings of the layers (see link to PDF below)

The beauty of this layered approach is that by "re-training" ("fine tuning") of the model, one or a few top layers, already trained model can be reused for processing many related tasks, reducing training time significantly. This may enable distributed learning, since remote and mobile devices could have enough processing power to "learn" on top base models created on large cloud based neural networks and shared in compact form.

Metaphorically, like text-based books are methods of distributing human knowledge, there could be "AI Books" to distributed Neural Network models for AI tools and devices. A brave new world!

How convolutional neural networks see the world
"Keras is a Deep Learning library for Python, that is simple, modular, and extensible."

"VGG16 (also called OxfordNet) is a convolutional neural network architecture named after the Visual Geometry Group from Oxford, who developed it. It was used to win the ILSVR (ImageNet) competition in 2014."

Visualizing and Understanding Convolutional Networks  : PDF
"Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we address both issues."

ImageNet .org
"ImageNet is an image database organized according to the WordNet hierarchy (currently only the nouns), in which each node of the hierarchy is depicted by hundreds and thousands of images.

"Feature Learning Escapades

Stanford University CS231n: Convolutional Neural Networks for Visual Recognition

"Convolutional Neural Networks", mentioned even in Apple WWDC keynote: )
(as a feature of Mac OS)

Deep Visualization Toolbox - YouTube

Nested Virtualization: VMs inside VMs

Nested Virtualization | Microsoft Docs
"Nested virtualization is a feature that allows you to run Hyper-V inside of a Hyper-V virtual machine. In other words, with nested virtualization, a Hyper-V host itself can be virtualized. Some use cases for nested virtualization would be to run a Hyper-V Container in a virtualized container host, set-up a Hyper-V lab in a virtualized environment, or to test multi-machine scenarios without the need for individual hardware. Prerequisites:
  • A Hyper-V host running Windows Server 2016 or Windows 10 Anniversary Update.
  • A Hyper-V VM running Windows Server 2016 or Windows 10 Anniversary Update.
  • A Hyper-V VM with configuration version 8.0 or greater.
  • An Intel processor with VT-x and EPT technology."
Can you run a virtual machine inside another virtual machine? Would this make you safe from viruses that can migrate outside virtual machines? - Quora
"With VMware vSphere, yes, you can and it works very well - no speed penalty. VMware calls it Nested VMs."

"Nested virtualization is the ability to run a hypervisor inside of a virtual machine. The hypervisor that runs on the real hardware is called a level 0 or L0; the hypervisor that runs as a guest on L0 is called level 1 or L1; a guest that runs on the L1 hypervisor is called a level 2 or L2."

"EC2 uses Xen virtualization."

WebAssembly vs asm.js; Code Cartoons

Inside WebAssembly with Mozilla Fellow David Bryant on the Hanselminutes Technology Podcast: Fresh Air for Developers

Code Cartoons by Lin Clark

A cartoon intro to WebAssembly ★ Mozilla Hacks – the Web developer blog
A graph showing another performance spike in 2017 with a question mark next to it

Why WebAssembly is Faster Than asm.js ★ Mozilla Hacks – the Web developer blog

WebAssembly .org

"The wasm stack machine is designed to be encoded in a size- and load-time-efficient binary format. WebAssembly aims to execute at native speed by taking advantage of common hardware capabilities available on a wide range of platforms.

WebAssembly describes a memory-safe, sandboxed execution environment that may even be implemented inside existing JavaScript virtual machines"

WebAssembly | MDN

asm.js. org

From ASM.JS to WebAssembly – Brendan Eich

WebAssembly | Luke Wagner's Blog

WebAssembly logo voting · Issue #980 · WebAssembly/design · GitHub


WebKit Now Has Full Support for WebAssembly
"Apple Safari has full support for WebAssembly including preparation for future integration with ECMAScript Modules and threads. Apple has added full support for WebAssembly to WebKit, enhancing the Safari browser to be able to run code written in other languages. WebKit supports WebAssembly on x86, x64 and ARM 64 platforms."