Temporal.io: Workflows-as-Code

Episode #124: Self-Provisioning Runtimes with Shawn "swyx" Wang - Serverless Chats Podcast

Shawn “Swyx” Wang is currently Head of DX at Temporal.io, based out of Seattle. He is also a frequent writer and speaker best known for the Learn in Public movement and recently published The Coding Career Handbook with more advice for engineers going from Junior to Senior.
Twitter: @swyx
LinkedIn: https://www.linkedin.com/in/shawnswyxwang/
Website: https://www.swyx.io/
Github: https://github.com/sw-yx
YouTube: https://www.youtube.com/swyxTV
The Swyx Mixtape: https://swyx.transistor.fm/
The Self-Provision Runtime: https://www.swyx.io/self-provisioning-runtime

Temporal - the iPhone of System Design

3 core opinions: Orchestration, Event Sourcing, and Workflows-as-Code.

The prototypical workflow state machine is a JSON or YAML file listing a sequence of steps. But this abuses configuration formats for expressing code. it doesn’t take long before you start adding features like conditional branching, loops, and variables, until you have an underspecified Turing complete “domain specific language” hiding out in your JSON/YAML schema.

[
  {
    "first_step": {
      "call": "http.get",
      "args": {
        "url": "https://www.example.com/callA"
      },
      "result": "first_result"
    }
  },
  {
    "where_to_jump": {
      "switch": [
        {
          "condition": "${first_result.body.SomeField < 10}",
          "next": "small"
        },
        {
          "condition": "${first_result.body.SomeField < 100}",
          "next": "medium"
        }
      ],
      "next": "large"
    }
  },
  {
    "small": {
      "call": "http.get",
      "args": {
        "url": "https://www.example.com/SmallFunc"
      },
      "next": "end"
    }
  },
  {
    "medium": {
      "call": "http.get",
      "args": {
        "url": "https://www.example.com/MediumFunc"
      },
      "next": "end"
    }
  },
  {
    "large": {
      "call": "http.get",
      "args": {
        "url": "https://www.example.com/LargeFunc"
      },
      "next": "end"
    }
  }
]

This example happens to be from Google, but you can compare similar config-driven syntaxes from Argo, Amazon, and Airflow. The bottom line is you ultimately find yourself hand-writing the Abstract Syntax Tree of something you can read much better in code anyway:

async function dataPipeline() {
	const { body: SomeField } = await httpGet("https://www.example.com/callA")
	if (SomeField < 10) {
		await httpGet("https://www.example.com/SmallFunc")
	} else if (SomeField < 100) {
		await httpGet("https://www.example.com/MediumFunc")
	} else {
		await httpGet("https://www.example.com/BigFunc")
	}
}