In Theory: Durable Execution or crash-proof code

I bumped into this video yesterday. Very interesting topic and it has taken many years for the founder of temporal.io to get there, all the way back to 2012 while working at Amazon.

Check out the video if you don’t know what this is. This is a very powerful thing. “Never” having your code crash.

When I watched this, I thought to myself, this can be done in plang, just much simpler and more flexible.

When you use temporal.io, you need to use their cloud service or an on-premise service. This locks you into this environment and perpetual monthly payment.

So lets see how we can implement this in Plang.

Note: This is an In theory article as the title says. There a few of them. It means that even though currently this is not possible in plang, there is no technical blocker on implementing it. It just needs to be done. It’s just engineering

Let’s start with a simple code, we get some quotes from the web, store into a variable, then write the variable to file, wait for 1 day and do it again

Start
- get https://zenquotes.io/api/quotes, write to %quotes%
- set %quoteOfTheDay% = %quotes[0].q%
- write %quoteOfTheDay% to quote.txt
- wait for 1 day
- call Start

Many things can go wrong here. zenquotes.io could be down, maybe there are no quotes so %quotes[0].q% will crash, maybe computer dies when we are waiting, and “million” others things.

How would you do durable execution on this code in plang?

Simple. Events.

You can bind events to any executable line in plang. This is incredibly powerful.

So lets implement it. Create Events.goal file.

Events
- before each step, call SaveState
- after each step, call SaveState
- on error on a step, call SaveState

We now have setup our event to call SaveState when each line is executed and if there is an error. Now we need to store the state of the application somewhere.

SaveState
- set %data% = {
                step: %!step%,
                memoryStack: %!memoryStack%,
                runtime: %!runtime%
               }
- save %data% to "state.json"
- if %!error% is not empty then
     - save %!error% to "error.json"
     - throw %!error%

Excellent, we have the state stored in a file. You could of course communicate this to some server somewhere, but we are doing simple stuff, so a file is enough.

If there is an error, we throw that so it crashes the application (we want it to crash), if the computer just dies, we know in which step we were.

Now, comes a pretty cool part. Without modifying our execution code, we are going to start our app where it ended, if there is a state.json file.

We’ll add a new event to the Events file

Events
- before each step, call SaveState
- after each step, call SaveState
- on error on a step, call SaveState
- before goal “Start”, call LoadState

Now the app will call the LoadState goal before running the Start goal, lets do that code

LoadState
- read “state.json”, empty if files is not available, 
        write to %state%
- if %state% is empty then
    - end goal
- set runtime state with %state.runtime%, %state.step%, %state.memoryStack%

That is it. We now have durable execution for our app.

The only thing is missing is the “set runtime state with…” part, everything else you can do today in plang.

There will be some challenges making the runtime start at a specific location with a specific state since I didn’t think of designing the runtime that way, but it is not to difficult to over come. It’s simply engineering. It is already possible run a specific step in plang code from a plang code, so that is “half” the battle.

So there you have it, Durable Execution in plang.

The power of plang is just crazy sometimes.