Thursday, January 30, 2020

Python Apps Packaging


https://talkpython.fm/episodes/show/245/python-packaging-landscape-in-2020.

Official Python Docker image: hub.docker.com
Docker: docker.com
Vagrant: vagrantup.com
PyInstaller: pyinstaller.org
Briefcase: beeware.org
Pex: github.com
Shiv: github.com
pipx: pypi.org/project/pipx
PyOxidixer: gregoryszorc.com
Nuitka: nuitka.net
Cython: cython.org
Flatpak: flatpak.org
Snapcraft: snapcraft.io

+
Numba: A High Performance Python Compiler (to LLVM)

Pyston compiler cranks up Python thanks to LLVM | InfoWorld

UPS += electric_vans * 10000 + self_driving

UPS orders 10,000 electric delivery vans from startup Arrival

"UPS announced today that they are investing in UK-based EV startup Arrival and ordering 10,000 electric delivery vans from them in order to electrify their fleet."

UPS is buying thousands of electric vans and teaming up with Waymo to accelerate the future of delivery



"... UPS will use some of (Alphabet/Google subsidiary) Waymo’s self-driving Chrysler Pacifica minivans to shuttle packages between some of its stores in the Metro Phoenix area and its hub in Tempe, Arizona. The minivans won’t be fully driverless; Waymo says it will keep trained safety drivers in the front seat to monitor operations. Despite the limited nature of the pilot, both Waymo and UPS say a “long-term plan” between the companies remains possible."

Hyundai and Kia put over $110M into UK electric delivery vehicles startup Arrival

"Last year Hyundai announced a $35 billion commitment to develop self-driving technology and electric vehicles. As part of that, it wants to release 23 types of electric vehicles by 2025.
Last week, it unveiled a flying taxi concept with Uber at the CES tech conference in Las Vegas."


UPS teams up with Waymo to test self-driving delivery vans

"Waymo, the self-driving arm of Google's (GOOG) parent company Alphabet, will use its autonomous Chrysler Pacifica minivans to bring packages from UPS stores in the Phoenix area to a UPS sorting facility in Tempe, Arizona"
Waymo will transport UPS packages from stores in Phoenix to a facility in Tempe.

Monday, January 27, 2020

AWS SAM: Serverless Application Model

AWS Serverless Application Model - Amazon Web Services

The AWS Serverless Application Model (SAM) is an open-source framework for building serverless applications. It provides shorthand syntax to express functions, APIs, databases, and event source mappings. With just a few lines per resource, you can define the application you want and model it using YAML. During deployment, SAM transforms and expands the SAM syntax into AWS CloudFormation syntax, enabling you to build serverless applications faster.

To get started with building SAM-based applications, use the AWS SAM CLI. SAM CLI provides a Lambda-like execution environment that lets you locally build, test, and debug applications defined by SAM templates. You can also use the SAM CLI to deploy your applications to AWS.


AWS Serverless Application Model (SAM) Command Line Interface – Build, Test, and Debug Serverless Apps Locally (blog post)

Sunday, January 26, 2020

AWS Certifications

AWS Certification - Validate AWS Cloud Skills - Get AWS Certified




Which AWS certification is right for me? @ A Cloud Guru



How I cleared All 11 AWS Certifications, in a good long time
list of some useful resources

dev font: JetBrains Mono

Font is a 'big deal' for developers that spend significant time in front of screen and code.
The new font from JetBrains is optimized for code.
Free and open source

www.infoq.com/news/2020/01/jetbrains-mono

JetBrains Mono. A typeface for developers_

"Consider this in contrast to some other fonts. Consolas, for example, has slightly wider letters. However, they are still rather small, which forces you to increase the size by one point to make the font more readable. As a result, lines of code tend to run longer than expected.
JetBrains Mono’s standard-width letters help keep lines to the expected length."


jetbrains.com/lp/mono/#how-to-install

boingboing.net/2020/01/16/jetbrains-mono-is-a-free-open.html

Saturday, January 25, 2020

Cruise self-driving

GM’s Cruise to unveil first car built to operate with no driver

"Cruise has raised $7.25 billion from investors including Honda Motor Co., Japan’s SoftBank Vision Fund and T. Rowe Price Associates Inc. Those investments have given Cruise a value of $19 billion, GM has said."


Deno (.js) in Rust

dotJS 2019 - Bert Belder - Deno @YouTube



//github.com/denoland/deno

//deno.land



What is Rust and why is it so popular? - Stack Overflow Blog

"Rust has been Stack Overflow’s most loved language for four years in a row, indicating that many of those who have had the opportunity to use Rust have fallen in love with it. However, the roughly 97% of survey respondents who haven’t used Rust may wonder, “What’s the deal with Rust?”

Material Design Web UI Frameworks

how reliable are lists like this?

20 Best Material Design Web UI Frameworks for Websites & Applications - Super Dev Resources
material design web ui frameworks

Why is Material-UI for React so popular download?
@material-ui/core - npm
430K downloads/week

vs. #1
mdbootstrap - npm
5K /week

vs. #2
https://www.npmjs.com/package/material-kit
0.06 /week (!)




energy efficiency: Manitoba Hydro Building

wikipedia.org/wiki/Manitoba_Hydro_Place

Manitoba Hydro Place received LEED Platinum certification in May 2012, making it the most energy efficient office tower in North America

The building's bioclimatic, energy-efficient design features a 377 ft (115 m) tall solar chimney, a geo-thermal HVAC system using 280 five-inch tubes bored 380 feet into an underground aquifer,[10] 100% fresh air (24 hours a day, year round, regardless of outside temperature)[8] and a one-meter-wide double exterior wall with computer-controlled motorized vents that adjust the building's exterior skin throughout the day and evening. Together, the various elements of the design enable a 70% energy savings over a typical large office tower.

"one of "the most energy-efficient office towers in the world"




Friday, January 24, 2020

AWS CLI alternative awless.io, in GoLang

awless.io, a mighty CLI for AWS - Henri - Medium
//github.com/wallix/awless in GoLang
terminal icon

AWS CLI is written in Python,  and is also open source
aws/aws-cli: Universal Command Line Interface for Amazon Web Services @GitHub
//github.com/aws/aws-cli

The aws-cli package includes a very useful command completion feature. This feature is not automatically installed so you need to configure it manually. To enable tab completion for bash either use the built-in command complete:
$ complete -C aws_completer aws
//aws.amazon.com/cli/

What Is the AWS Command Line Interface? - AWS Command Line Interface

All IaaS (infrastructure as a service) AWS administration, management, and access functions in the AWS Management Console are available in the AWS API and CLI. New AWS IaaS features and services provide full AWS Management Console functionality through the API and CLI at launch or within 180 days of launch.



Python Boto3 API reference



visual: developer roadmaps

Roadmaps to becoming a modern developer - roadmap.sh

Thursday, January 23, 2020

Windows 10 "God Mode"

A strange back-door "feature" for managing Windows. Just created folder named

"GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}"
https://www.techrepublic.com/article/how-to-use-god-mode-in-windows-10/

figure-c.jpg


PartiQL: SQL++ for Amazon QLDB

partiql.org

"SQL-compatible access to relational, semi-structured, and nested data."


Diagram showing where PartiQL fits with other data sources.


github.com/partiql/partiql-spec


Announcing PartiQL: One query language for all your data | AWS Open Source Blog


Now Available – Amazon Quantum Ledger Database (QLDB) | AWS News Blog

"PartiQL is a new open standard query language that supports SQL-compatible access to relational, semi-structured, and nested data while remaining independent of any particular data source. To learn more, read Announcing PartiQL: One Query Languge for All Your Data
."



Wednesday, January 22, 2020

Google Study: Traits of Most Successful Teams

Google Spent 2 Years Studying 180 Teams. The Most Successful Ones Shared These 5 Traits

1. Dependability.

Team members get things done on time and meet expectations.

2. Structure and clarity.

High-performing teams have clear goals, and have well-defined roles within the group.

3. Meaning.

The work has personal significance to each member.

4. Impact.

The group believes their work is purposeful and positively impacts the greater good.
Yes, that's four, not five. The last one stood out from the rest:

5. Psychological Safety.

Amazon Ion: JSON++ for QLDB

Amazon Ion

"Amazon Ion is a richly-typed, self-describing, hierarchical data serialization format offering interchangeable binary and text representations. The text format (a superset of JSON) is easy to read and author, supporting rapid prototyping. The binary representation is efficient to store, transmit, and skip-scan parse. The rich type system provides unambiguous semantics for long-term preservation of data which can survive multiple generations of software evolution.

Ion was built to address rapid development, decoupling, and efficiency challenges faced every day while engineering large-scale, service-oriented architectures. It has been addressing these challenges within Amazon for nearly a decade, and we believe others will benefit as well.

Available Libraries: Ion JavaIon CIon PythonIon JavaScript
Related Projects: Ion HashIon Schema
Tools: Hive SerDe"

used for:
(QLDB) "Documents – Documents exist within tables, and must be in Amazon Ion form. Ion is a superset of JSON that adds additional data types, type annotations, and comments. QLDB supports documents that contain nested JSON elements, and gives you the ability to write queries that reference and include these elements. Documents need not conform to any particular schema, giving you the flexibility to build applications that can easily adapt to changes."





Tuesday, January 21, 2020

Openpilot: $1000 Self-Driving

$7,000 Tesla Autopilot vs $1,000 Openpilot: Self-Driving Test!

Openpilot: powered by open source software from
comma.ai – Introducing openpilot


5 Enterprise Technology Trends for 2020

5 Technology Trends That Will Change the Enterprise in 2020

1. IoT leads to an explosion of endpoints and a need for centralized management.

2. Companies realize the need to control cloud spending.

3. The role of automation and artificial intelligence (AI) in enterprise solutions will expand.

4. 5G is finally here.

5. Companies approach the next wave of digital transformation.


Thursday, January 16, 2020

big data, big money: Snowflake Inc.

Snowflake Logo.svgwikipedia.org/wiki/Snowflake_Inc


Snowflake Inc. is a cloud-based data-warehousing startup that was founded in 2012. It has raised more than $900 million in venture capital...
valuation to $3.5 billion 
Snowflake runs on Amazon S3 since 2014, and on Microsoft Azure since 2018

snowflake.com


podcast interview
softwareengineeringdaily.com/tag/snowflake/


See Snowflake in 8 Minutes @YouTube


nowflake Architecture - Learn How Snowflake Stores Table data @YouTube


free ebooks

Key Concepts & Architecture — Snowflake Documentation

Architecture overview

AWS Glue

AWS Glue - Amazon Web Services

AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy for customers to prepare and load their data for analytics. You can create and run an ETL job with a few clicks in the AWS Management Console. You simply point AWS Glue to your data stored on AWS, and AWS Glue discovers your data and stores the associated metadata (e.g. table definition and schema) in the AWS Glue Data Catalog. Once cataloged, your data is immediately searchable, queryable, and available for ETL.

Queries against an Amazon S3 Data Lake diagram

Monday, January 13, 2020

Saturday, January 11, 2020

dapr: runtime for building microservices: cloud native, mesh, sidecar

dapr.io

An event-driven, portable runtime for building microservices on cloud and edge.


github.com/dapr (GoLang, MIT license)

"Dapr injects a side-car container/process to each compute unit. 
The side-car interacts with event triggers and communicates with the compute unit via standard HTTP or gRPC protocols. This enables Dapr to support all existing and future programming languages without requiring you to import frameworks or libraries.
Dapr offers built-in state management, reliable messaging (at least once delivery), triggers and bindings through standard HTTP verbs or gRPC interfaces. This allows you to write stateless, stateful and actor-like services following the same programming paradigm. You can freely choose consistency model, threading model and message delivery patterns.

Dapr runs natively on Kubernetes, as a standalone binary on your machine, on an IoT device, or as a container that can be injected into any system, in the cloud or on-premises.

Dapr uses pluggable state stores and message buses such as Redis as well as gRPC to offer a wide range of communication methods, including direct dapr-to-dapr using gRPC and async Pub-Sub with guaranteed delivery and at-least-once semantics."

Dapr Conceptual Model

github.com/dapr/dapr/wiki/FAQ

"Dapr is a distributed application runtime. Unlike a service mesh which is focused on networking concerns, Dapr is focused on providing building blocks that make it easier for developers to build microservices. Dapr is developer-centric versus service meshes being infrastructure-centric."
Learn all about Distributed Application Runtime (Dapr), Part 1 | Azure Friday @YouTube (Channel 9)
Part 2

Dapr, Rudr, OAM | Mark Russinovich presents next gen app development & deployment (Microsoft Ignite)

cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/

microservices app diagram


Sidecar pattern
Sidecar Design Pattern
docs.microsoft.com/en-us/azure/architecture/patterns/sidecar

cncf.io/webinars/monitoring-k8s-workloads-sidecar-pattern/

dzone.com/articles/sidecar-design-pattern-in-your-microservices-ecosy-1



en.wikipedia.org/wiki/Sidecar

A sidecar is a one-wheeled device attached to the side of a motorcycle, scooter, or bicycle,[1] making the whole a three-wheeled vehicle. A motorcycle with a sidecar is sometimes called a combination, an outfit, a rig or a hack

Sidecar on Vespa scooter

Service Mesh

wikipedia.org/wiki/Service_mesh

"In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often using a sidecar proxy.

Having such a dedicated communication layer can provide a number of benefits, such as providing observability into communications, providing secure connections, or automating retries and backoff for failed requests."

nginx.com/blog/what-is-a-service-mesh/





course: Sustainable Building Design | edX | MITx

Sustainable Building Design | edX

"Learn and explore key scientific principles, technologies, and analysis techniques for designing comfortable indoor environments while reducing energy use and associated climate change effects."
Sustainable Building Design

Thursday, January 09, 2020

AWS CloudFormation vs. Terraform

CloudFormation vs. Terraform | cloudonaut

Scope

https://ryaneschinger.com/blog/aws-cloudformation-vs-terraform/

CloudFormation: using JSON or YAML for templates

JSON
{
  "Parameters": {
    "KeyName": {
      "Description": "The EC2 Key Pair for SSH access",
      "Type": "AWS::EC2::KeyPair::KeyName"
    }
  },
  "Resources": {
    "CFExampleInstance": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "SecurityGroups": [ { "Ref": "CFSSHAccess" } ],
        "KeyName": { "Ref": "KeyName" },
        "ImageId": "ami-0f4cfd64",
        "InstanceType": "t1.micro"
      }
    },
    "CFSSHAccess": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "GroupDescription": "SSH access",
        "SecurityGroupIngress": [ {
          "IpProtocol": "tcp",
          "FromPort": "22",
          "ToPort": "22",
          "CidrIp": "0.0.0.0/0"
        } ]
      }
    }
  }
}
YAML
Parameters:
  KeyName:
    Description: The EC2 Key Pair for SSH access
    Type: AWS::EC2::KeyPair::KeyName
Resources:
  CFExampleInstance:
    Type: AWS::EC2::Instance
    Properties:
      SecurityGroups:
      - Ref: CFSSHAccess
      KeyName:
        Ref: KeyName
      ImageId: ami-0f4cfd64
      InstanceType: t1.micro
  CFSSHAccess:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: SSH access
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '22'
        ToPort: '22'
        CidrIp: 0.0.0.0/0
Teraform custom syntax in HCL (HashiCorp Configuration Language)

variable "key_name" {}

resource "aws_instance" "TFExampleInstance" {
  ami = "ami-0f4cfd64"
  instance_type = "t1.micro"
  key_name = "${var.key_name}"
  security_groups = [ "${aws_security_group.TFSSHAccess.name}" ]
}

resource "aws_security_group" "TFSSHAccess" {
  name = "TFSSHAccess"
  description = "SSH access"

  ingress {
    protocol = "tcp"
    from_port = 22
    to_port = 22
    cidr_blocks = ["0.0.0.0/0"]
  }
}
@hackernoon


hashicorp/terraform @GitHub

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned. 

Wednesday, January 08, 2020

C++ smart pointers vs Rust

Smart pointers (Modern C++) | Microsoft Docs

"As shown in the example, a smart pointer is a class template that you declare on the stack, and initialize by using a raw pointer that points to a heap-allocated object. After the smart pointer is initialized, it owns the raw pointer. This means that the smart pointer is responsible for deleting the memory that the raw pointer specifies. The smart pointer destructor contains the call to delete, and because the smart pointer is declared on the stack, its destructor is invoked when the smart pointer goes out of scope, even if an exception is thrown somewhere further up the stack."

void UseRawPointer() { // Using a raw pointer -- not recommended. Song* pSong = new Song(L"Nothing on You", L"Bruno Mars"); // Use pSong... // Don't forget to delete! delete pSong; } void UseSmartPointer() { // Declare a smart pointer on stack and pass it the raw pointer. unique_ptr song2(new Song(L"Nothing on You", L"Bruno Mars")); // Use song2... wstring s = song2->duration_; //... } // song2 is deleted automatically here.

Smart Pointers in C++ - GeeksforGeeks

intro/smart pointers - cppreference.com


How to think about Rust ownership versus C++ unique_ptr · Franklin Chen

Learning smart pointers in C++ and Rust | Junchao's blog

Smart Pointers - The Rust Programming Language


Tuesday, January 07, 2020

Apache NiFi: NiagaraFiles, flow based programming


Apache NiFi - Wikipedia

Apache NiFi is a software project from the Apache Software Foundation designed to automate the flow of data between software systems. It is based on the "NiagaraFiles" software previously developed by the NSA, which is also the source of a part of its present name – NiFi.
Flow-based programming - Wikipedia

flow-based programming (FBP) is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally component-oriented.

FBP is a particular form of dataflow programming based on bounded buffers, information packets with defined lifetimes, named ports, and separate definition of connections.

Saturday, January 04, 2020

AWS-shell vs AWS CLI + JMESPath

https://github.com/awslabs/aws-shell

AWS Command Line Interface and aws-shell Sample for AWS Cloud9 - AWS Cloud9

"The AWS CLI and the aws-shell are unified tools that provide a consistent interface for interacting with all parts of AWS. You can use the AWS CLI or the aws-shell instead of the AWS Management Console to quickly run commands to interact with AWS, and some of these commands can only be run with the AWS CLI or the aws-shell."

AWS Shell demo video @ OReilly Learning (subscription or trial needed)
AWS CLI overview

https://bitbucket.org/awsdevguru/ code

JMESPath.org
a query language for JSON.

JMESPath Tutorial — JMESPath

locations[?state == 'WA'].name | sort(@) | {WashingtonCities: join(', ', @)}

Result

{
  "WashingtonCities": "Bellevue, Olympia, Seattle"
}

AWS Architecture & Solutions

Architecture Resources | AWS

Cloud Solutions | Vetted, Technical Reference Architecture | AWS

AWS Well-Architected - Build secure, efficient, cloud enabled applications

AWS Well-Architected Tool - Amazon Web Services

Product-Page-Diagram_AWS-Well-Architected-Tool_how-it-works