Saturday, February 23, 2019

AWS Step Functions + Simple Workflow Service (SWF)



"When using Amazon SWF, you implement workers to perform tasks. These workers can run either on cloud infrastructure, such as Amazon Elastic Compute Cloud (Amazon EC2), or on your own premises. You can create tasks that are long-running, or that may fail, time out, or require restarts—or that may complete with varying throughput and latency. Amazon SWF stores tasks and assigns them to workers when they are ready, tracks their progress, and maintains their state, including details on their completion. To coordinate tasks, you write a program that gets the latest state of each task from Amazon SWF and uses it to initiate subsequent tasks. Amazon SWF maintains an application's execution state durably so that the application is resilient to failures in individual components. With Amazon SWF, you can implement, deploy, scale, and modify these application components independently."



The different entities or "actors" in an Amazon SWF workflow.




"What is SWF not?

  • SWF does not execute any code.
  • SWF does not contain the logic of the workflow.
  • SWF does not allow you to draw a workflow or a state machine

How does it work?

SWF is based on polling. Your code runs on your machines on AWS or on-premises – it doesn’t matter. Your code is polling for tasks from the SWF API (where they wait in queues), receives a task, executes it, and sends the result back to the SWF API.
SWF then issues new tasks to your code, and keeps the history of the workflow (state)."

AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications. Workflows are made up of a series of steps, with the output of one step acting as input into the next. Application development is simpler and more intuitive using Step Functions, because it translates your workflow into a state machine diagram that is easy to understand, easy to explain to others, and easy to change. You can monitor each step of execution as it happens, which means you can identify and fix problems quickly. Step Functions automatically triggers and tracks each step, and retries when there are errors, so your application executes in order and as expected."

sfn_how-it-works

Visual Workflow.
"You should consider using AWS Step Functions for all your new applications, since it provides a more productive and agile approach to coordinating application components using visual workflows. If you require external signals to intervene in your processes, or you would like to launch child processes that return a result to a parent, then you should consider Amazon Simple Workflow Service (Amazon SWF). With Amazon SWF, instead of writing state machines in declarative JSON, you write a decider program to separate activity steps from decision steps. This provides you complete control over your orchestration logic, but increases the complexity of developing applications. You may write decider programs in the programming language of your choice, or you may use the Flow framework to use programming constructs that structure asynchronous interactions for you."

{ "Comment": "A Hello World example of the Amazon States Language using a Pass state", "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Pass", "Result": "Hello World!", "End": true } } }

ASL: Amazon States Language - AWS Step Functions

{ "Comment": "An example of the Amazon States Language using a choice state.", "StartAt": "FirstState", "States": { "FirstState": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME", "Next": "ChoiceState" }, "ChoiceState": { "Type" : "Choice", "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next": "FirstMatchState" }, { "Variable": "$.foo", "NumericEquals": 2, "Next": "SecondMatchState" } ], "Default": "DefaultState" }, "FirstMatchState": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:OnFirstMatch", "Next": "NextState" }, "SecondMatchState": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:OnSecondMatch", "Next": "NextState" }, "DefaultState": { "Type": "Fail", "Error": "DefaultStateError", "Cause": "No Matches!" }, "NextState": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME", "End": true } } }

State Types:
  • Common State Fields
  • Pass: passes input to output
  • Task: single unit of work; main building block for state machines; properties
    • "Type": "Task",
    • "Resource": "arn:aws:lambda:..."
    • "TimeoutSeconds": 300,  // optional
    • "HearthbeatSeconds": 60 // optional
    • "Next": "NextState1"
    • "Retry": [ { "ErrorEquals": ["CustomError"]. "IntervalSeconds": 1, "MaxAttempts": 5 }]
    • "Catch": [ { ... }
  • Choice: conditional branching logic ("if statement", "switch", create loops)
    • { "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next", "MatchOne" } ... 
    • "And": [...], "Not": [...], "Or": [...]
  • Wait: delay execution
    • "Seconds": 10  ==> wait 10 seconds before next state
    • "Timestamp": "2020-01-01T00:00:00Z" // wait until specified time
    • "SecondsPath": "$.waitseconds" // use input variable
    • "TimestampPath": "$.expirydate" // use input var
  • Succeed: end state
    • "SuccessState": { Type": "Succeed" }
  • Fail: end state
    • "FailState": { "Type": "Fail", "Cause": "Data error", "Error": "inventory low" }
  • Parallel: parallel execution
    • "StateP1": { "Type": "Parallel", "Next": "FinalState",
      "Branches" : [ { "StartAt": "Wait 20s", " States: { "Wait 20s" { ... } } }, 
Input and Output Processing for State/Task; using "$" with field names;
using "path" to filter portion of input for processing and output; example

{ "numbers": [2, 3], "sum": 5 }

{ "Type": "Task", "Resource": "arn:aws:lambda:us-east1:123:function:Add", "Next", "NextStep",
"InputPath": "$.numbers", "ResultPath": "$.sum", "OutputPath": "$.sum" }

State Machine Data - AWS Step Functions



"You can now use a local version of AWS Step Functions to develop and test your workflows."

 course: AWS Developer: Lambda Deep Dive @ Pluralsight
Using Step Functions to Control Flow



Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)