We rebuilt the workflow builder from scratch, for the coding agent era.
Terse is automation that lives where developers do. Build, debug, and deploy complex agentic workflows from your terminal or IDE. Serverless hosting, scaling, and observability come standard.
TLDR of Terse Architecture
Terse is a code-first Workflow/Automation builder. Every organization in Terse can have multiple projects. Each project, is a npm package that contains multiple Jobs. A Job, is a set of triggers + a closure that is run when a trigger fires. Closures contain multiple action steps and TerseAgents. Of course, all of this is defined in pure Typescript.
Requirements
Hosting is a key feature for us. In the age of AI, we must:
- Run un-trusted 3P code on behalf of our users
- Run agents in the cloud safely
- Pause execution and resume (Human in the Loop)
- Be able to scale easily with demand
- Be serverless (don't want to force users to have set up hosting to get started)
That's when I came across this blog post mentioning how Lovable is fully powered by Modal's Sandboxes as well as the famous Ramp one in January. Looks like Modal Sandboxes was the obvious choice!
I remember building the core of the SDK and thinking in back of my mind this sandbox piece was going to kill me. After all, hosting is usually the hardest part right?
In reality, it took about an hour to get spun up. I was shocked.
Let's walk through my original POC and how I made job startup 100x faster by utilizing the full gamut of Modal's sandbox API.
The Implementation
The original was indeed a bit rough, but deployments took less than a second. It looked something like:
Unfortunately, this creates a pretty complex flow at trigger runtime:
The code for the Modal part was very elegant:
// this part is super fast
const app = await modal.apps.fromName("tandbox", { createIfMissing: true })
const image = modal.images.fromRegistry("node:22-slim")
const sb = await modal.sandboxes.create(app, image)
// Write zip buffer into sandbox filesystem
const writeHandle = await sb.open("/tmp/code.zip", "w")
await writeHandle.write(new Uint8Array(zipBuffer))
await writeHandle.close()
// Unzip the code
const unzipProc = await sb.exec(["sh", "-c", "apt-get update -qq && apt-get install -y -qq unzip > /dev/null 2>&1 && cd /tmp && unzip -o code.zip -d project > /dev/null"], {
stdout: "pipe",
stderr: "pipe"
})
// install dependencies
const p = await sb.exec(["sh", "-c", `cd /tmp/project/Terse-Jobs && ${INSTALL_CMD} 2>&1`], { stdout: "pipe", stderr: "pipe" })
const out = await p.stdout.readText()
const code = await p.wait()
// install our cli
const p = await sb.exec(["sh", "-c", `cd /tmp/project/Terse-Jobs && npm install terse-cli@latest 2>&1`], { stdout: "pipe", stderr: "pipe" })
const out = await p.stdout.readText()
const code = await p.wait()
// run the job!
const p = await sb.exec(["sh", "-c", `cd /tmp/project/Terse-Jobs && ${RUN_CMD}`], { stdout: "pipe", stderr: "pipe" })
const [stdout, stderr] = await Promise.all([p.stdout.readText(), p.stderr.readText()])
const code = await p.wait()This sounded reasonable to my sandbox n00b brain at the time, however there was a problem. Getting the sandbox started up and ready to go took over 15 seconds!
Remember, we run this every-time a job is triggered. That means, we could be installing, unzipping and downloading the exact same dependencies 1000s of times a day. Not only this is unnecessary repetitive work, it can drive up costs due to the increased CPU usage in the sandboxes.
Not to mention, I now have to store users code on object storage. That is a bit strange and could raise eyebrows when people are evaluating our tool.
Thankfully, Modal solves this with images!
Image Magic
Once the user deploys a project, we can spin up a sandbox and immediately snapshot it to get an image. Unless the user deploys a new version, we can guarantee the source code is exactly the same. Trade-off is that the deploy step now incurs the latency, but well worth the jobs starting up faster.
// DEPLOY TIME
// Create a snapshot from FS defined above
const snapshot = sb.snapshotFilesystem()
// store snapshot id in db ...
// TRIGGER TIME
const snapshotId = // fetch snapshotId for project containing triggered job
// Now we can just re-use it whenever we want! This is also super fast.
const app = modal.apps.fromName(APP_NAME, { createIfMissing: true })
const image = modal.images.fromId(snapshotId)
const sb_warm = modal.sandboxes.create(app, image, { timeoutMs: SANDBOX_TIMEOUT_MS })This front loads latency to the deploy step, but we no longer need cloud storage!
And then at Job trigger time, much simpler:
This dropped our job start latency from 15s to just ~150ms, a 100x improvement!
This is much better! However, there is still an issue. If you just change one little line in the project source (a console.log(), swap which slack channel the job posts to etc), your deploy would still take 20s+. Not ideal at all.
This is where layered images come into play.
Instead of having one image for the entire project, we can split between dependencies (ie: Node modules) and the actual business logic source. Again, Modal makes this trivial:
const baseImage = modal.images.fromRegistry("node:22-slim")
const depsSB = modal.sandboxes.create(app, baseImage, { timeoutMs: SANDBOX_TIMEOUT_MS })
// Install dependencies.... on sandbox install cli etc...
// Create snapshot for dependencies
const depsImage = depsSB.snapshotFilesystem()
// Create new sandbox based on the dependencies sandbox
const sb = modal.sandboxes.create(app, depsImage, { timeoutMs: SANDBOX_TIMEOUT_MS })
// Download + unzip source code
// Now this is our final snapshot!
const image = sb.snapshotFilesystem()Now, if the user only changes the source, we simply rebuild the source image on top of the existing dependencies one. Now, small changes to the jobs only take around 3 seconds to deploy to production.
End Result
Job start times are under 150ms, deploying a new project takes around 20 seconds, and source updates take 3 seconds. In addition, we can keep costs low on the Modal side by preventing the constant downloading + unzipping of the source code projects.
No express servers needed, no security issues on the docker side to worry about.
Best part, Modal is already built to quickly scale up when needed. No load balancing needed in this architecture either.
We really enjoyed building on Modal, and we think you will as well.
TK
Appendix: Why Sandboxes to begin with?
The old way to do this would be to host an Express server for every project. Not only is that difficult, but would be cost anywhere from $10-30 per project with AWS (not mention we would need to use the AWS API 😭). Also, what is the story for running agents that mutate file systems? We will want to support running claude code/codex in these jobs as well.
Another option would be to create docker images and spawn them in worker queues. Unfortunately, there is no elegant way to do filesystem persistence, durability or Human In the Loop. Not to mention, docker wasn't designed to for running untrusted code thus could create security issues.
Hence, I went with sandboxes!