Wednesday, February 27, 2019

AWS IoT

a very informative podcast interview with VP of IoT with AWS

AWS Internet of Things with Dirk Didascalou - Software Engineering Daily


IoT Applications & Solutions | What is the Internet of Things (IoT)? | AWS

AWS IoT Software and Services

What is AWS IoT? 3 Min Overview - YouTube

RISC-V: Open-Source Hardware (OSH)


RISC-V - Wikipedia

"RISC-V (pronounced "risk-five") is an open-source hardware instruction set architecture (ISA) based on established reduced instruction set computer (RISC) principles.

In contrast to most ISAs, the RISC-V ISA is free and open-source and can be used royalty-free for any purpose, permitting anyone to design, manufacture and sell RISC-V chips and software. While not the first open-architecture[1] ISA, it is significant because it is designed to be useful in a wide range of devices. The instruction set also has a substantial body of supporting software, which avoids a usual weakness of new instruction sets.

The project began in 2010 at the University of California, Berkeley, but many contributors are volunteers and industry workers outside the university.[2]

The RISC-V ISA has been designed with small, fast, and low-power real-world implementations in mind,[3][4] but without over-architecting for a particular microarchitecture style"


@riscv
RISC-V @ GitHub


info from:
Amazon just acquired Eero because Wi-Fi is key to the smart home - Stacey on IoT | Internet of Things news and analysis

Tuesday, February 26, 2019

AWS > Amazon

AWS Drives More Than Half of Amazon's Operating Income | Light Reading

"Amazon Web Services comprised just 11% of the company's overall sales in 2018, but delivered more operating income than all other business units combined.

In financial results delivered Thursday, Amazon Web Services Inc. drove $7.43 billion sales for the quarter ending December 31, up 45% from $5.11 billion year-over-year. (See Amazon Reports Q4 Sales Up 20% to $72.4B.)"

React Native Rearchitecture

React Native Rearchitecture with G2i Team - Software Engineering Daily

React Native allows developers to build native applications for iOS and Android using components written in the React JavaScript framework. These ReactJS components render to native application code by going over a JavaScript bridge, a message bus that communicates between JavaScript code and native iOS or Android runtimes.

To address the performance issues of React Native, the core team working on React Native at Facebook is rearchitecting the React Native runtime within a project called Fabric. Fabric consists of changes to the threading model, the data handling system, and the JavaScript bridge.




"RN Fabric allows the UI thread (where UI is drawn) to be in sync with the JS thread (where the UI is programmed)"


this new engine may be more similar to NativeScript, a similar open source tool:

Sunday, February 24, 2019

Posgres as "Document DB" with JSONB

Postgres JSON: Unleash the Power of Storing JSON in Postgres | Codeship | via @codeship

CREATE TABLE cards ( id integer NOT NULL, board_id integer NOT NULL, data jsonb );

INSERT INTO cards VALUES (1, 1, '{"name": "Paint house", "tags": ["Improvements", "Office"], "finished": true}');

SELECT data->>'name' AS name FROM cards

SELECT * FROM cards WHERE data->>'finished' = 'true';

CREATE INDEX idxfinished ON cards ((data->>'finished'));

As of Postgres 9.4, along with the JSONB data type came GIN (Generalized Inverted Index) indexes. With GIN indexes, we can quickly query data using the JSON operators @>, ?, ?&, and ?

CREATE INDEX idxgindata ON cards USING gin (data); 

SELECT count(*) FROM cards WHERE data @> '{"tags": ["Clean", "Kitchen"]}';
How to query JSONB, beginner sheet cheat – Hacker Noon


PostgreSQL: Documentation: 10: 8.14. JSON Types

Fission: Serverless Functions for Kubernetes: FaaS

fission/fission: Fast Serverless Functions for Kubernetes @ GitHub, Apache 2 license

written in GoLang, supporting execution environments for
Go, JVM. .NET, NodeJS, Python, Ruby, PHP, and even Perl :)

trigger is something that maps an event to a function; Fission as of today supports HTTP requesttimed, and message queue triggers.

Performance: 100msec cold start
Fission maintains a pool of "warm" containers that each contain a small dynamic loader. When a function is first called, i.e. "cold-started", a running container is chosen and the function is loaded. This pool is what makes Fission fast: cold-start latencies are typically about 100msec.

Serverless Functions for Kubernetes - Fission

"What is Fission?
Fission is a framework for serverless functions on Kubernetes.

Write short-lived functions in any language, and map them to HTTP requests (or other event triggers).

Deploy functions instantly with one command. There are no containers to build, and no Docker registries to manage."



Saturday, February 23, 2019

Open Source Workflow Engines


Workflow Processing Engine Overview 2018: Airflow vs Azkaban vs Conductor vs Oozie vs Amazon Step Functions - Shawn's Pitstop

AWS Step Functions + Simple Workflow Service (SWF)



"When using Amazon SWF, you implement workers to perform tasks. These workers can run either on cloud infrastructure, such as Amazon Elastic Compute Cloud (Amazon EC2), or on your own premises. You can create tasks that are long-running, or that may fail, time out, or require restarts—or that may complete with varying throughput and latency. Amazon SWF stores tasks and assigns them to workers when they are ready, tracks their progress, and maintains their state, including details on their completion. To coordinate tasks, you write a program that gets the latest state of each task from Amazon SWF and uses it to initiate subsequent tasks. Amazon SWF maintains an application's execution state durably so that the application is resilient to failures in individual components. With Amazon SWF, you can implement, deploy, scale, and modify these application components independently."




               The different entities or "actors" in an Amazon SWF workflow.




"What is SWF not?

  • SWF does not execute any code.
  • SWF does not contain the logic of the workflow.
  • SWF does not allow you to draw a workflow or a state machine

How does it work?

SWF is based on polling. Your code runs on your machines on AWS or on-premises – it doesn’t matter. Your code is polling for tasks from the SWF API (where they wait in queues), receives a task, executes it, and sends the result back to the SWF API.
SWF then issues new tasks to your code, and keeps the history of the workflow (state)."

AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications. Workflows are made up of a series of steps, with the output of one step acting as input into the next. Application development is simpler and more intuitive using Step Functions, because it translates your workflow into a state machine diagram that is easy to understand, easy to explain to others, and easy to change. You can monitor each step of execution as it happens, which means you can identify and fix problems quickly. Step Functions automatically triggers and tracks each step, and retries when there are errors, so your application executes in order and as expected."

sfn_how-it-works


              Visual Workflow.
"You should consider using AWS Step Functions for all your new applications, since it provides a more productive and agile approach to coordinating application components using visual workflows. If you require external signals to intervene in your processes, or you would like to launch child processes that return a result to a parent, then you should consider Amazon Simple Workflow Service (Amazon SWF). With Amazon SWF, instead of writing state machines in declarative JSON, you write a decider program to separate activity steps from decision steps. This provides you complete control over your orchestration logic, but increases the complexity of developing applications. You may write decider programs in the programming language of your choice, or you may use the Flow framework to use programming constructs that structure asynchronous interactions for you."

{ "Comment": "A Hello World example of the Amazon States Language using a Pass state", "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Pass", "Result": "Hello World!", "End": true } } }

ASL: Amazon States Language - AWS Step Functions

{ "Comment": "An example of the Amazon States Language using a choice state.", "StartAt": "FirstState", "States": { "FirstState": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME", "Next": "ChoiceState" }, "ChoiceState": { "Type" : "Choice", "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next": "FirstMatchState" }, { "Variable": "$.foo", "NumericEquals": 2, "Next": "SecondMatchState" } ], "Default": "DefaultState" }, "FirstMatchState": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:OnFirstMatch", "Next": "NextState" }, "SecondMatchState": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:OnSecondMatch", "Next": "NextState" }, "DefaultState": { "Type": "Fail", "Error": "DefaultStateError", "Cause": "No Matches!" }, "NextState": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME", "End": true } } }

State Types:
  • Common State Fields
  • Pass: passes input to output
  • Task: single unit of work; main building block for state machines; properties
    • "Type": "Task",
    • "Resource": "arn:aws:lambda:..."
    • "TimeoutSeconds": 300,  // optional
    • "HearthbeatSeconds": 60 // optional
    • "Next": "NextState1"
    • "Retry": [ { "ErrorEquals": ["CustomError"]. "IntervalSeconds": 1, "MaxAttempts": 5 }]
    • "Catch": [ { ... }
  • Choice: conditional branching logic ("if statement", "switch", create loops)
    • { "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next", "MatchOne" } ... 
    • "And": [...], "Not": [...], "Or": [...]
  • Wait: delay execution
    • "Seconds": 10  ==> wait 10 seconds before next state
    • "Timestamp": "2020-01-01T00:00:00Z" // wait until specified time
    • "SecondsPath": "$.waitseconds" // use input variable
    • "TimestampPath": "$.expirydate" // use input var
  • Succeed: end state
    • "SuccessState": { Type": "Succeed" }
  • Fail: end state
    • "FailState": { "Type": "Fail", "Cause": "Data error", "Error": "inventory low" }
  • Parallel: parallel execution
    • "StateP1": { "Type": "Parallel", "Next": "FinalState",
      "Branches" : [ { "StartAt": "Wait 20s", " States: { "Wait 20s" { ... } } }, 
Input and Output Processing for State/Task; using "$" with field names;
using "path" to filter portion of input for processing and output; example

{ "numbers": [2, 3], "sum": 5 }

{ "Type": "Task", "Resource": "arn:aws:lambda:us-east1:123:function:Add", "Next", "NextStep",
"InputPath": "$.numbers", "ResultPath": "$.sum", "OutputPath": "$.sum" }

State Machine Data - AWS Step Functions



"You can now use a local version of AWS Step Functions to develop and test your workflows."

course: AWS Developer: Lambda Deep Dive @ Pluralsight
Using Step Functions to Control Flow



Azure Logic Apps + Microsoft Flow

workflow cloud platforms

Logic App Service | Microsoft Azure

What are Microsoft Flow, Logic Apps, Functions, and WebJobs? - Azure | Microsoft Docs

"Microsoft Flow and Logic Apps are both designer-first integration services that can create workflows. Both services integrate with various SaaS and enterprise applications.
Microsoft Flow is built on top of Logic Apps. They share the same workflow designer and the same connectors."

"Functions and Logic Apps are Azure services that enable serverless workloads. Azure Functions is a serverless compute service, whereas Azure Logic Apps provides serverless workflows. Both can create complex orchestrations. An orchestration is a collection of functions or steps, called actions in Logic Apps, that are executed to accomplish a complex task. For example, to process a batch of orders, you might execute many instances of a function in parallel, wait for all instances to finish, and then execute a function that computes a result on the aggregate."
Azure Logic Apps Documentation - Tutorials, API Reference | Microsoft Docs

"Azure Logic Apps is a cloud service that helps you automate and orchestrate tasks, business processes, and workflows when you need to integrate apps, data, systems, and services across enterprises or organizations. Logic Apps simplifies how you design and build scalable solutions for app integration, data integration, system integration, enterprise application integration (EAI), and business-to-business (B2B) communication, whether in the cloud, on premises, or both.

To build enterprise integration solutions with Azure Logic Apps, you can choose from a growing gallery with 200+ connectors, which include services such as Azure Service Bus, Functions, and Storage; SQL, Office 365, Dynamics, Salesforce, BizTalk, SAP, Oracle DB, file shares, and more. Connectors provide triggers, actions, or both for creating logic apps that securely access and process data in real time."
Logic Apps Designer - example logic app

"Every logic app workflow starts with a trigger, which fires when a specific event happens, or when new available data meets specific criteria. Many triggers include basic scheduling capabilities so that you can specify how regularly your workloads run. For more custom scheduling scenarios, start your workflows with the Schedule trigger. Learn more about how to build schedule-based workflows.

Each time that the trigger fires, the Logic Apps engine creates a logic app instance that runs the actions in the workflow. These actions can also include data conversions and flow controls, such as conditional statements, switch statements, loops, and branching."

Quickstart - Create and automate your first workflow - Azure Logic Apps | Microsoft Docs

Select trigger: "RSS - When a feed item is published"

Set up trigger with RSS feed, frequency, and interval

Select this action: "Office 365 Outlook - Send an email"

Add contents for email body

Enterprise integration with Azure Logic Apps | Microsoft Docs

Microsoft/vscode-azurelogicapps: Azure Logic Apps extension for VS Code @ GitHub

for low code / no code apps:

Automate processes + tasks | Microsoft Flow

"Turn repetitive tasks into multistep workflows. For example, with a few clicks capture tweets and add them as leads in Dynamics 365, subscribers in Mailchimp, and more...
Start with a template..."
A multistep workflow that adds a record to Dynamics365 and MailChimp when a user with more than 100 followers posts a tweet.

"Microsoft Azure LogicApps Components:
  • Workflow – is used to define the different steps of your business process, using a graphical user interface.
  • Triggers - initiate a workflow, and create a new instance of the workflow. A trigger can be an arrival of a file on FTP site or receiving an email. Types of triggers:
    • Poll triggers: These triggers poll your service at a specified frequency to check for new data. When new data is available, a new instance of your logicapp runs with the data as input.
    • Push triggers: (Webhooks) These triggers listen for data on an endpoint, or for an event to happen, then triggers a new instance of your logicapp.
    • Recurrence trigger: This trigger instantiates an instance of your logicapp on a prescribed schedule.
    • Event-based triggers: i.e. message from service bus queue
  • Actions - An action represents a step in the workflow.
    It can invoke an operation on your API.
  • Connectors - are used to connect to different data sources and services.
    Azure LogicApps provide an in-built set of managed connectors that cover different areas like social media, file FTP and many more.
    • Enterprise Integration Pack (EIP) Connectors - provide functionality like BizTalk.
      These connectors are used for complex enterprise integration scenarios such as XML messaging and Validation. It supports exchange messages through industry-standard protocols, including AS2, X12, and EDIFACT. It requires an "Integration Accounts" to store different artifacts, like schemas, partners, certificates, maps and agreements

Online training classes:

Azure Logic Apps: Getting Started | Pluralsight

Azure Logic Apps: Getting Started : Building Blocks

  • "Connectors" are pre-built wrappers around systems (web service APIs); there are about 200 built-in connects, and you can build your own (that is just an API)
  • "Triggers": to initiate a Logic App; could be "reactive" and "proactive" (see above list); could leverage "trigger templates" and "pre-built templates"; Not all connectors could be triggers
  • "Actions": building blocks of Logic Apps; most of Actions are also Connectors.
    Actions is just "next step" in a logic app. 
  • "Flow Control": sequence of logic in Logic App
    • "if-then", "switch", "for-each", "do-until", "scope"

Microsoft Azure Developer: Creating Enterprise Logic Apps | Pluralsight

Azure Logic Apps: Fundamentals | Pluralsight













Data operation samples - Azure Logic Apps | Microsoft Docs



"actions": {
   "Initialize_variable_-_integer_array": {
      "type": "InitializeVariable",
      "inputs": {
         "variables": [ 
            {
               "name": "myIntegerArray",
               "type": "Array",
               "value": [
                  1,
                  2,
                  3,
                  4
               ]
            }
         ]
      },
      "runAfter": {}
   },
   "Select": {
      "type": "Select",
      "inputs": {
         "from": "@variables('myIntegerArray')",
         "select": {
            "Product_ID": "@item()"
         }
      },
      "runAfter": {
         "Initialize_variable_-_integer_array": [
           "Succeeded"
         ]
      }
   }
},

Schedule and run automated tasks and workflows with Azure Logic Apps | Microsoft Docs
"triggers": {
   "Recurrence": {
      "type": "Recurrence",
      "recurrence": {
         "frequency": "Week",
         "interval": 1,
         "schedule": {
            "hours": [
               10,
               12,
               14
            ],
            "minutes": [
               30
            ],
            "weekDays": [
               "Monday"
            ]
         },
         "startTime": "2017-09-07T14:00:00",
         "timeZone": "Pacific Standard Time"
      }
   }
}
azure-quickstart-templates/101-function-app-create-dynamic at master · Azure/azure-quickstart-templates @ GitHub


Friday, February 22, 2019

JavaScript Workflow Engines, State Machines, Statecharts, React Automata

"JavaScript Promises are just a state machines"


(Finite) State Machine is based on "events" that cause transitions between states. Usually there is a nice graphical presentation of such states and events.

When systems get more complex there are "state-charts" with some useful tools to help.

Workflow Engine vs. State Machine:
"In a workflow engine, transition to the next step occurs when a previous action is completed, whilst a state machine needs an external event that will cause branching to the next activity."

Workflows and state transitions can be defined as meta-data (i.e. JSON or XML files), and tools can dynamically generate GUI and process events.

Workflows are also essential part of  "Business Process Modelling" (BPM). (Business process management) "enterprise" tools.

Videos:

David Khourshid - Infinitely Better UIs with Finite Automata - YouTube

David Khourshid: Simplifying Complex UIs with Finite Automata & Statecharts | JSConf Iceland 2018 - YouTube


Informative docs:
Welcome to the world of Statecharts - Statecharts (statecharts.github.io)

"a statechart is a beefed up state machine. The beefing up solves a lot of the problems that state machines have, especially state explosion that happens as state machines grow"

What is a state machine? - Statecharts
"a state machine is a software component that defines:
  • The machine is defined by a finite list of states.
  • One state is defined as the initial state. When a machine is “started” it automatically enters this state.
  • States can define “events” that trigger a transition.
  • transition causes the machine to exit the state and enter another (or the same) state.
  • A transition may be conditional, in other words might ask “the world” about things. These are called guards and must be side-effect free.
  • A state can define actions upon entering or exiting a state. These are called actions and will typically have side effects."

Comparison of popular Javascript workflow libs downloads in last 6 months:

fsm-iterator vs javascript-state-machine vs machina vs xstate | npm trends



GitHub repos and docs of those libs:

davidkpiano/xstate: State machines and statecharts for the modern web.
Light Machine

Super quick start | XState Docs

XState Version 4 Released 🚀 – David Khourshid – Medium


Robust React User Interfaces with Finite State Machines | CSS-Tricks

Responsive web design tool, CMS, and hosting platform | Webflow



jakesgordon/javascript-state-machine: A javascript finite state machine library

matter state machine


ifandelse/machina.js: js ex machina - finite state machines in JavaScript


jfairbank/fsm-iterator: A finite state machine iterator for JavaScript
by jfairbank (Jeremy Fairbank) / Repositories


fschaefer/Stately.js: Stately.js is a JavaScript based finite-state machine (FSM) engine for Node.js and the browser.


Standard data format for representing state machines for the web
State Chart XML (SCXML): State Machine Notation for Control Abstraction
Statechart XML from W3C



Related helpful tools and sites:

MicheleBertoli/react-automata: A state machine abstraction for React

"A state machine abstraction for React that provides declarative state management and automatic test generation."


spectrum.chat/statecharts



State Machine: State Explosion - Statecharts

"The main problem that’s stopping widespread usage of state machines is the fact that beyond very simple examples, state machines often end up with a large number of states, a lot of them with identical transitions.

Simple state machine with two states, valid and invalid
Not so simple state machine with four states and eight transitions
Complex state machine with eight states and twelve transitions

" Statecharts solve this state explosion problem:"

Simple state machine with six states, and six transitions

In statecharts, states can be organized hierarchically.
Simple state machine with valid/invalid as substate of changed

A guard is a sort of pre condition to a transition, in a way it stops a transition from happening if some condition is not true.
Simple state machine with six states, and six transitions