Thursday, December 13, 2018

serveless: AWS Lambda Layers & Custom Runtimes

the future of cloud development is "serverless", AWS made it clear on re:Invent 2018

New for AWS Lambda – Use Any Programming Language and Share Common Components | AWS News Blog

AWS Lambda Layers - AWS Lambda

"You can configure your Lambda function to pull in additional code and content in the form of layers. A layer is a ZIP archive that contains libraries, a custom runtime, or other dependencies. With layers, you can use libraries in your function without needing to include them in your deployment package."






Wednesday, December 12, 2018

open source: AWS Firecracker: microVMs, containers for serverless

Firecracker Logo TitleFirecracker – Lightweight Virtualization for Serverless Computing | AWS News Blog

firecracker-microvm/firecracker: Secure and fast microVMs for serverless computing @ GitHub

"Firecracker is an open source virtualization technology that is purpose-built for creating and managing secure, multi-tenant container and function-based services that provide serverless operational models. Firecracker runs workloads in lightweight virtual machines, called microVMs, which combine the security and isolation properties provided by hardware virtualization technology with the speed and flexibility of containers.

The main component of Firecracker is a virtual machine monitor (VMM) that uses the Linux Kernel Virtual Machine (KVM) to create and run microVMs."

Monday, December 10, 2018

SQL Server / Azure SQL Database: Graph DB Architecture


SQL Server (starting with 2017)

SQL Graph Architecture | Microsoft Docs
Users can create one graph per database. A graph is a collection of node and edge tables. Node or edge tables can be created under any schema in the database, but they all belong to one logical graph

sql-graph-architecture

Amazon: SQL => NoSQL, Oracle => Aurora, DynamoDB


AWS re:Invent 2018 - Keynote with Werner Vogels - YouTube

Amazon DynamoDB - Migrations

AWS Database Migration Service and Amazon DynamoDB: What You Need to Know | AWS Database Blog



Using an Amazon DynamoDB Database as a Target for AWS Database Migration Service - AWS Database Migration Service

Amazon Aurora FAQs – Amazon Web Services (AWS)


"Amazon Aurora MySQL delivers up to five times the performance of MySQL without requiring any changes to most MySQL applications; similarly, Amazon Aurora PostgreSQL delivers up to three times the performance of PostgreSQL."

Sunday, December 09, 2018

k8s in-security

​Kubernetes' first major security hole discovered | ZDNet

"Kubernetes has become the most popular cloud container orchestration system by far, so it was only a matter of time until its first major security hole was discovered. And the bug, CVE-2018-1002105, aka the Kubernetes privilege escalation flaw,"

Kubernetes: The smart person's guide

data: Rockset: Serverless Schemaless SQL


"SQL analytics for Data Lakes"

Rockset
Build Apps Without Pipelines
Serverless search and analytics for data-driven apps


true


rockset/rocksdb-cloud: A library that provides an embeddable, persistent key-value store for fast storage optimized for AWS @ GitHub
RocksDB-Cloud is a C++ library that brings the power of RocksDB to AWS, Google Cloud and Microsoft Azure. It leverages the power of RocksDB to provide fast key-value access to data stored in Flash and RAM systems.


RocksDB-Cloud is API compatible and data format compatible with RocksDB.
RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)



Saturday, December 08, 2018

Omi vs. React, Angular, Vue

Tencent/omi: Next generation web framework in 4kb JavaScript (Web Components + JSX + Proxy + Store + Path Updating), Easy to MVVM.

omi
(China's) Tencent vs. Facebook, Google...


link from  .NET Rocks! vNext

books: .NET Core & ASP.NET Core in Action

GitHub repo is now a normal part of tech books.
I expect Jupyther notebooks will be too, in "a few" years...

book coverManning | ASP.NET Core in Action

andrewlock/asp-dot-net-core-in-action: Source code examples for ASP.NET Core in Action


Manning | .NET Core in Action

dmetzgar/dotnetcoreinaction: Companion code from .NET Core in Action by Manning Publishing



Hanselminutes Technology Podcast - Fresh Air and Fresh Perspectives for Developers - ASP.NET Core in Action with Andrew Lock


Project Jupyter | JupyterHub

Samsung QLC SSD: lower price, lower write speed

Here is how "clouds" can offer various storage price/performance...

Samsung 860 QVO 1TB and 4TB SATA SSD Review - Samsung Goes QLC | PC Perspective

"While QLC writes are far slower than what we are used to seeing from a modern SSD, SLC caching is the answer to bridging that performance gap."

IoT: Raspberry Pi 3 A+: $25

New product: Raspberry Pi 3 Model A+ on sale now at $25 - Raspberry Pi

"you can now get the 1.4GHz clock speed, 5GHz wireless networking and improved thermals of Raspberry Pi 3B+ in a smaller form factor, and at the smaller price of $25. Meet the Raspberry Pi 3 Model A+."

with Raspberry Pi 3 Model B+ shipping in volume, we’re able to fill that gap by releasing Raspberry Pi 3 Model A+. Raspberry Pi 3 Model A+ incorporates most of the neat enhancements we made to its big brother, and features:

  • A 1.4GHz 64-bit quad-core ARM Cortex-A53 CPU
  • 512MB LPDDR2 SDRAM  (vs. 1024 of 3 B+)
  • Dual-band 802.11ac wireless LAN and Bluetooth 4.2/BLE
  • Improved USB mass-storage booting
  • Improved thermal management"

Azure: Cloud Native Application Bundle (CNAB)

Introducing CNAB: a cloud-agnostic format for packaging and running distributed applications

CNAB: Cloud Native Application Bundles


​CNAB: Docker and Microsoft's Cloud Native Application Bundle | ZDNet

Brendan Burns, Microsoft software engineer and Kubernetes co-founder tweeted: "Imagine installing a complete distributed application from a USB stick." Ultimately, that's exactly what CNAB will mean to you.


AWS re:invent 2018 summary


Keeping Up With AWS re:Invent 2018 – marknca – Medium


The Cloudcast - Weekly Cloud Computing Podcast - All of the 2018 AWS reInvent Announcements



Infrastructure


AWS US-East GovCloud - https://aws.amazon.com/blogs/aws/aws-govcloud-us-east-now-open/


Route 53 for Hybrid Clouds - https://aws.amazon.com/blogs/aws/new-amazon-route-53-resolver-for-hybrid-clouds/


AWS Global Accelerator - https://aws.amazon.com/blogs/aws/new-aws-global-accelerator-for-availability-and-performance/


AWS Transit Gateway (hub & spoke VPC routing) - https://aws.amazon.com/blogs/aws/new-use-an-aws-transit-gateway-to-simplify-your-network-architecture/




Compute


EC2 Predictive Scaling (powered by AI) - https://aws.amazon.com/blogs/aws/new-predictive-scaling-for-ec2-powered-by-machine-learning/


AWS A1 Compute (powered by ARM) - https://aws.amazon.com/blogs/aws/new-ec2-instances-a1-powered-by-arm-based-aws-graviton-processors/


AWS C5n Compute (100Gb Networking) - https://aws.amazon.com/blogs/aws/new-c5n-instances-with-100-gbps-networking/


Hibernate EC2 Instances - https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-instances/


New Snowball Edge (more Compute & GPU) - https://aws.amazon.com/blogs/aws/coming-soon-snowball-edge-with-more-compute-power-and-a-gpu/


AWS Firecracker (Open Source, Lightweight Virtualization) - https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/


AWS Outposts (Managed Private Cloud) https://aws.amazon.com/blogs/enterprise-strategy/aws-outposts-a-new-dimension-to-aws-cloud/




Storage


AWS DataSync - Fast, Automated Data Transfer - https://aws.amazon.com/blogs/aws/new-aws-datasync-automated-and-accelerated-data-transfer/


AWS Managed SFTP - https://aws.amazon.com/blogs/aws/new-aws-transfer-for-sftp-fully-managed-sftp-service-for-amazon-s3/


S3 Cost Optimization via Automated Tiering - https://aws.amazon.com/blogs/aws/new-automatic-cost-optimization-for-amazon-s3-via-intelligent-tiering/





File Systems


Amazon FXs for Windows File Server - https://aws.amazon.com/blogs/aws/new-amazon-fsx-for-windows-file-server-fast-fully-managed-and-secure/


Amazon FXs for Lustre - https://aws.amazon.com/blogs/aws/new-amazon-fsx-for-lustre/




Databases


Amazon DynamoDB Transactions - https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/


Amazon DynamoDB On-Demand - https://aws.amazon.com/blogs/aws/amazon-dynamodb-on-demand-no-capacity-planning-and-pay-per-request-pricing/


Amazon Forecast - Time Series Forecasting - https://aws.amazon.com/blogs/aws/amazon-forecast-time-series-forecasting-made-easy/


Kinesis Data Analytics for Java - https://aws.amazon.com/blogs/aws/new-amazon-kinesis-data-analytics-for-java/


Account Management


AWS Resource Access Manager (cross-account sharing) - https://aws.amazon.com/blogs/aws/new-aws-resource-access-manager-cross-account-resource-sharing/


Private Marketplace on AWS Marketplace - https://aws.amazon.com/blogs/aws/new-aws-marketplace-makes-it-easier-to-govern-software-procurement-with-private-marketplace/


AWS CloudWatch Logs Insight - https://aws.amazon.com/blogs/aws/new-amazon-cloudwatch-logs-insights-fast-interactive-log-analytics/


AWS Cloud Maps - custom maps of applications - https://aws.amazon.com/blogs/aws/aws-cloud-map-easily-create-and-maintain-custom-maps-of-your-applications/


AWS Well-Architected Tool (review against best-practices) - https://aws.amazon.com/blogs/aws/new-aws-well-architected-tool-review-workloads-against-best-practices/


AWS License Manager - https://aws.amazon.com/blogs/aws/new-aws-license-manager-manage-software-licenses-and-enforce-licensing-rules/




Machine Learning


Machine Learning Training [Free] - https://aws.amazon.com/training/learning-paths/machine-learning/


Amazon Elastic Inference - GPU Powered Deep Learning Inference Acclerator - https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/


Amazon Sagemaker - Ground Truth - https://aws.amazon.com/blogs/aws/amazon-sagemaker-ground-truth-build-highly-accurate-datasets-and-reduce-labeling-costs-by-up-to-70/


Amazon Machine Learning Algorihms and Models in Marketplace - https://aws.amazon.com/blogs/aws/new-machine-learning-algorithms-and-model-packages-now-available-in-aws-marketplace/


Amazon SageMaker RL (Reinforced Learning) - https://aws.amazon.com/blogs/aws/amazon-sagemaker-rl-managed-reinforcement-learning-with-amazon-sagemaker/


AWS DeepRacer - Use Sagemaker RL - https://aws.amazon.com/blogs/aws/aws-deepracer-go-hands-on-with-reinforcement-learning-at-reinvent/


Amazon Sagemaker “Neo” - Run Once, Run Anywhere - https://aws.amazon.com/blogs/aws/amazon-sagemaker-neo-train-your-machine-learning-models-once-run-them-anywhere/




Serverless


AWS Amplify Console - continuous deployment and hosting service for modern web applications with serverless backends - https://aws.amazon.com/about-aws/whats-new/2018/11/announcing-aws-amplify-console


New AWS Services in Step Functions - https://aws.amazon.com/blogs/aws/new-compute-database-messaging-analytics-and-machine-learning-integration-for-aws-step-functions/


AWS Toolkits for popular IDEs (Pycharm, IntelliJ, VisualStudioCode) - https://aws.amazon.com/blogs/aws/new-aws-toolkits-for-pycharm-intellij-preview-and-visual-studio-code-preview/


New AWS Services in Step Functions - https://aws.amazon.com/blogs/aws/new-compute-database-messaging-analytics-and-machine-learning-integration-for-aws-step-functions/


New Lambda Programming Languages (and BYO), and Share Lambda components - https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/




New Amazon Businesses


AWS Ground Station (managed satellite ground stations) - https://aws.amazon.com/blogs/aws/aws-ground-station-ingest-and-process-data-from-orbiting-satellites/


AWS Elemental MediaConnect (Ingest & Distribution of Video) - https://aws.amazon.com/blogs/aws/new-aws-elemental-mediaconnect-for-ingestion-and-distribution-of-video-in-the-cloud/


Amazon Comprehend Medical (NLP for Healthcare) - https://aws.amazon.com/blogs/aws/amazon-comprehend-medical-natural-language-processing-for-healthcare-customers/


AWS DeepRacer - Use Sagemaker RL - https://aws.amazon.com/blogs/aws/aws-deepracer-go-hands-on-with-reinforcement-learning-at-reinvent/


AWS RoboMaker - Develop, Test, Deploy, Manage Robotics Apps - https://aws.amazon.com/blogs/aws/aws-robomaker-develop-test-deploy-and-manage-intelligent-robotics-apps/


Amazon Personalize - Recommendation Engine for Everyone - https://aws.amazon.com/blogs/aws/amazon-personalize-real-time-personalization-and-recommendation-for-everyone/

Friday, December 07, 2018

Flutter 1.0

Cross-Platform Mobile Toolkit Flutter Now Stable

"At its Flutter Live conference in London, Google launched the first stable release of Flutter, a Dart-based mobile development platform aiming to make it possible to write apps that share a single code base across iOS and Android with a native UI. Additionally, Google has announced an experimental web-based implementation of Flutter’s runtime which aims to bring Flutter apps to standard web browsers."

Wednesday, December 05, 2018

Open Source: WPF, WinForms, WinUI


Microsoft Open Sources WPF, WinForms, and WinUI @ InfoQ

"at Microsoft Connect 2018, the company announced the released of the first preview of .NET Core 3.0. Contained within that is a bigger announcement: the company is releasing WPF, Windows Forms (WinForms), and WinUI as open source projects on GitHub. They have all been released under the MIT license, giving developers wide latitude to use them in their own applications."


GitHub: WPF, Windows Forms, and WinUI. (Backward-compatible versions of Windows UI features including UWP XAML controls, and Fluent styles and materials.)

Open Source versions are based on .NET Core, not on original Windows implementations.
.NET Core is portable on many platforms and high-performance, so this is an interesting option for portable UI.

Microsoft Connect(); 2018 | Channel 9

Tuesday, December 04, 2018

Microsoft + Chromium?

"Fake News?" Would make sense, though...

Report: Microsoft is scrapping Edge, switching to just another Chrome clone | Ars Technica

Microsoft said to be working on stripped-down 'Windows Lite' OS to compete with Chrome OS - TechSpot

Microsoft is building its own Chrome browser to replace Edge - The Verge

Microsoft is ditching Edge for a Chrome-based browser - 9to5Google

Microsoft reportedly rebuilding its Edge browser on Google Chrome foundation - CNET

Microsoft is building a Chromium-powered web browser that will replace Edge on Windows 10 | Windows Central


Azure ML

Azure Machine Learning Service | Microsoft Azure

Azure Machine Learning Services Now Available @ InfoQ


Microsoft Enhances Azure For Running Container, IoT And Machine Learning Workloads @ Forbes

"After going back and forth with the ML PaaS strategy, Microsoft finally got it right. Firstly, the service is based on the popular Jupyter Notebooks for developing ML models in Python. Microsoft built native APIs for other Azure services such as storage and compute that data scientists can mix and match with familiar modules like Numpy and Pandas. The service can be used to test the model locally before running the training job on expensive but powerful GPU clusters. Developers and data scientists can use ML framework of their choice. The service has excellent support for Sckit-learn, TensorFlow, CNTK, PyTorch among others."

Get Started with Azure Machine Learning | Microsoft Connect(); 2018 | Channel 9


Sunday, December 02, 2018

GitHub: top programming languages

The State of the Octoverse: top programming languages of 2018 | The GitHub Blog

Top programming languages by contributor
Fastest growing languages by contributor



gRPC & Protocol Buffers: High Performance APIs

protocolbuffers/protobuf: Protocol Buffers - Google's data interchange format @ GitHub

Building High Performance APIs In Go Using gRPC And Protocol Buffers

"Protocol Buffers, also referred as protobuf, is Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. Protocol Buffers are smaller, faster, and simpler that provides high performance than other standards such as XML and JSON. 


By using protocol buffers, you can define your structured data, then you generate source code for your choice of programming language using the protocol buffer compiler named protoc, to write and read your structured data using it. The current version of protocol buffers is proto3. The proto3 version currently supports generated code in variety of languages including C++, Go, Java, Python, Ruby, and C#
.

---

gRPC is a high performance, open-source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems. The gRPC framework is developed and open-sourced by Google. Google has been using a lot of the underlying technologies and concepts in gRPC for a long time for their many products including several of Google’s cloud products.

gRPC follows HTTP semantics over HTTP/2. It allows you to build services with both synchronous and asynchronous communication model. It supports traditional Request/Response model and bidirectional streams. Its capability for building full-duplex streaming lets you use it for advanced scenarios where both client and server applications can send stream of data asynchronously.
Mobile

By default, gRPC uses Protocol Buffers as the Interface Definition Language (IDL) and as its underlying message interchange format"
grpc / grpc.io

grpc / gRPC Motivation and Design Principles.



10x Not 10% Product management

10x Not 10% Product management by orders of magnitude by Ken Norton at Mind the Product 2015 - YouTube

great visual: film camera market => digital camera market => smartphone market



10x Not 10% - Product management by orders of magnitude - Ken Norton

Film camera sales, 1977-2007
Ken Norton - Product Manager and Google Ventures Partner

Ken Norton - Ken Norton

great interview podcast!
GV’s Ken Norton | Internet History Podcast



Saturday, December 01, 2018

Jenkins X on Kubernetes

Getting started with Jenkins X | Opensource.com

"Jenkins X is an open source system that offers software developers continuous integration, automated testing, and continuous delivery, known as CI/CD, in Kubernetes. Jenkins X-managed projects get a complete CI/CD process with a Jenkins pipeline that builds and packages project code for deployment to Kubernetes and access to pipelines for promoting projects to staging and production environments."
Developer commits changes, project deployed to staging

jx create spring - YouTube

hybrid cloud: AWS Outposts

AWS Outposts Overview page

"AWS Outposts bring native AWS services, infrastructure, and operating models to virtually any data center, co-location space, or on-premises facility. You can use the same APIs, the same tools, the same hardware, and the same functionality across on-premises and the cloud to deliver a truly consistent hybrid experience. Outposts can be used to support workloads that need to remain on-premises due to low latency or local data processing needs.

AWS Outposts come in two variants: 1) VMware Cloud on AWS Outposts allows you to use the same VMware control plane and APIs you use to run your infrastructure, 2) AWS native variant of AWS Outposts allows you to use the same exact APIs and control plane you use to run in the AWS cloud, but on-premises."

product-page-diagram_frontier_how-it-works_Final

AI cloud service: Amazon Forecast

Time Series Forecasting | Machine Learning | Amazon Forecast

"Accurate time-series forecasting service, based on the same technology used at Amazon.com
No machine learning experience required"

Seer_HowitWorks_Final

AI cloud service: Amazon Personalize

Real-time personalization and recommendation | Amazon Personalize | AWS

"Amazon Personalize is a machine learning service that makes it easy for developers to create individualized recommendations for customers using their applications."
product-page-diagram_amazon_personalize_how-it-works

AI service: Amazon Textract (OCR++)

Amazon Textract | Extract Text & Data | AWS

"Amazon Textract is a service that automatically extracts text and data from scanned documents. Amazon Textract goes beyond simple optical character recognition (OCR) to also identify the contents of fields in forms and information stored in tables."

Optical Character Recognition (OCR)

AI/ML/cloud: Amazon Elastic Inference + AWS Inferentia

"AI co-processor"

Amazon Elastic Inference - Amazon Web Services

"Amazon Elastic Inference allows you to attach low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost of running deep learning inference by up to 75%. Amazon Elastic Inference supports TensorFlow, Apache MXNet, and ONNX models, with more frameworks coming soon."

Amazon Elastic Inference supports TensorFlow and Apache MXNet models, with additional frameworks coming soon.

tensorflow_logo_200px

AL/ML: AWS DeepRacer

AWS DeepRacer - the fastest way to get rolling with machine learning

"AWS DeepRacer is the fastest way to get rolling with machine learning, literally. Get hands-on with a fully autonomous 1/18th scale race car driven by reinforcement learning, 3D racing simulator, and global racing league."

CAR: 18th scale 4WD with monster truck chassis
CPU: Intel Atom™ Processor
MEMORY: 4GB RAM
STORAGE: 32GB (expandable)
WI-FI: 802.11ac
CAMERA: 4 MP camera with MJPEG
SOFTWARE: Ubuntu OS 16.04.3 LTS, Intel® OpenVINO™ toolkit, ROS Kinetic


Amazon FSx: cloud flexible filesystems


New – Amazon FSx for Windows File Server – Fast, Fully Managed, and Secure | AWS News Blog

New – Amazon FSx for Lustre | AWS News Blog

Amazon FSx for Windows File Server | Cloud File Storage | Amazon Web Services

FSx_Windows_File_Server_How-it-Works

Amazon FSx for Lustre | File Storage | AWS

AWS Re:Invent: New Machine Learning, Data, Infrastructure Services - The New Stack

AWS re:Invent 2018 Keynote - Andy Jassy - YouTube

Amazon Managed Blockchain

Amazon Managed Blockchain

"Amazon Managed Blockchain is a fully managed service that makes it easy to create and manage scalable blockchain networks using the popular open source frameworks Hyperledger Fabric and Ethereum*."
Product-Page-Diagram_AWS-Taiga_Final

AWS re:Invent 2018 Keynote - Andy Jassy - YouTube

Amazon Quantum Ledger Database (QLDB)

Amazon QLDB

"Fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log. Owned by a central trusted authority."

Product-Page-Diagram_AWS-Quantum(2)

announced at:

Amazon Timestream: cloud time series database for IoT

Amazon Timestream

"Amazon Timestream is a fast, scalable, fully managed time series database service for IoT and operational applications that makes it easy to store and analyze trillions of events per day at 1/10th the cost of relational databases. Driven by the rise of IoT devices, IT systems, and smart industrial machines, time-series data — data that measures how things change over time — is one of the fastest growing data types. Time-series data has specific characteristics such as typically arriving in time order form, data is append-only, and queries are always over a time interval."

Timestream - How it works

announced ad