Deno is joining Cloudflare

The Deno team is joining Cloudflare, and we're merging workerd and celld! For more on what's happening to the Deno runtime and our various efforts, see my post on the Deno blog.

Long ago, I stood in a Berlin warehouse and introduced Node.js with a 500-line JavaScript IRC server and live audience participation. Node.js was an exploration of what a purely asynchronous programming model could enable. Async I/O was already known to be important for fast servers, but it was difficult to wield. Removing sync networking made it easier to build systems that previously required much more machinery. What made the IRC demo exciting was how little code it took.

I started Deno to find more powerful abstractions like this — looking for places where we could simplify and remove capabilities while giving engineers ever more power. With Deno, we improved the experience of writing JavaScript, but we didn’t fundamentally change what developers had to assemble around the runtime. The bigger problems are around networked applications: distributing computation, coordinating state, data storage in general, and autoscaling to meet demand.

Something about that old IRC demo has always bothered me — it was a single server, single thread even. It could handle many user connections, but it would start getting slow if you had too many. Scaling IRC eventually means multiple machines with the channels shared across them. A modern chat app also needs to store message history, user accounts, and other data.

Could the programming model make partitioning across machines part of the design from the start?

The solution clicked into focus when I read about Cloudflare's Durable Objects. It provided a very powerful abstraction: the distributed singleton with a SQLite database. Each Durable Object is like a small, individually addressable server with its own relational database. Its JavaScript execution is single-threaded (which is easy to reason about), it handles WebSockets, and its local SQLite database can be accessed synchronously. By building a chat application using one DO per channel, you have sharded your data and WebSocket connections, and thus made your application scalable.

Much more than chat can be built with DOs. On top of this abstraction other services can be built: Queues, KV, Durable Execution (CF's Workflow API), even Git storage. The Durable Object abstraction is simple yet incredibly powerful.

But running this model outside of Cloudflare has remained difficult. Cloudflare’s network is great — but there are a million reasons one might want to run it themselves. Workerd is already open source, but its Durable Objects support is limited to a single instance. What I wanted was the distributed version: IRC channels spread across machines, with the platform handling placement, routing, and durable storage. The power of this abstraction is how it scales. That's why I started celld.

Building and operating Deno Deploy taught me how complicated cloud infrastructure could become: multiple public clouds, multiple databases, and deeply entangled services. With celld, I wanted the opposite: one binary, written in Rust, with object storage as its only external service dependency.

What excites me is the leverage this model gives developers: compute, relational databases, queues, and real-time communication primitives designed for applications distributed across machines. To run it, you manage many instances of celld and a single object storage bucket. Your application can contain many services without each requiring a separate infrastructure project. This is much closer to the simplicity I've been chasing since that first Node.js demo.

We’re joining Cloudflare to make the Workers programming model a mainstream way to build servers. By bringing celld and workerd together, we want it to be radically easy to build and operate distributed applications on your own infrastructure.

This may come as a surprise to some, but when the Deno team released celld in August, we were delighted.

Why a surprise? Well, as it happens, there is a popular theory that suggests we shouldn't be happy.

You see, it is commonly claimed around the Internet that we made Cloudflare Workers work differently from other cloud platforms in order to create "lock-in." The claim goes, software built on Workers is hard to move to other hosting platforms, therefore if you write your app on Workers you are "trapped" on Cloudflare forever. This is especially true of apps that use Durable Objects: they are not just written differently but architected differently. All this is, supposedly, a clever trap: once you are too deep to be able to switch, the rug will somehow be pulled out from under you.

Under this theory, celld is a threat: as an open-source implementation of Workers and Durable Objects, it allows applications built on Workers to seamlessly migrate to other providers. No more trap!

This theory is wrong

Workers is different because it is better

We built Workers differently because we believe it is a better way to build web applications.

• The Workers design makes it dead simple and dirt cheap to manage an application that runs in hundreds of locations around the world, a feat which no other popular hosting platform has achieved. The pricing isn't a trick: we've genuinely created a novel architecture that is more efficient than competitors, and we've passed on the savings to you.
• Workers' live environment design (aka "bindings") simultaneously makes configuring access to external resources easier and more secure. Usually, those two things don't come together!
• Durable Objects make it easy to build real-time collaboration and distributed systems that are difficult or impossible in a classical three-tier web architecture.

In short, we believe we've simply created a better programming model for cloud software. But being better requires being different: we couldn't achieve these benefits while remaining perfectly compatible with existing platforms.

If we could offer these benefits and be perfectly compatible with traditional cloud platforms, we absolutely would! It would be a great benefit to our business if every kind of compute workload could simply be lifted and shifted onto Workers. But the reality is, innovation sometimes requires doing things differently. We feel that, in the long run, it's better to be different and better, than to be the same as everyone else.

"Lock-in" actually hurts us — that's why we went open source

If there were truly no escape hatch from Workers, then some of our largest customers would never have signed on with us in the first place.

When we originally pitched Workers for Platforms to customers like Shopify in 2022, the feedback was clear: They couldn't possibly build on us unless the Workers Runtime was open source.

And so, we open-sourced it. That's right: not a lot of people seem to know this, but the Cloudflare Workers Runtime, workerd, is open source. This is not a parallel implementation, it's the same code we run in production.

As a matter of fact, we have former customers that migrated away from us by using workerd. And that's OK! Those customers would never have signed with us in the first place if we hadn't been open source, and for every one that migrated away, there are many others that have stayed.

It turns out, being open source and giving people an escape hatch is good business.

But we could do better – and with Deno's help, we will

We released workerd genuinely intending for people to use it in production. A few have, but so far, not that many. Most people don't seem to have any idea that this is even possible.

Why not?

Frankly, we haven't done a great job of building out the ecosystem of adjacent services and tooling to make it really work. We'd released the runtime, and then we sort of just hoped the community would step in and start adapting it to every possible environment. It hasn't happened that way. Perhaps we were naive.

In the interest of transparency, I do have to admit that workerd has always had one big gap in its production-readiness: Durable Objects. Workerd implements them, but only in a single-instance way, good enough for local testing, but unable to scale. We have always wanted to fix this. There are TODO comments in the code going back to the original release. The problem is, our production implementation of Durable Object routing is simply not the implementation any self-hoster would want. It's a beast, designed to handle hundreds of locations, with numerous dependencies on external services operated by a team of site reliability engineers. We knew we needed something different for the self-hosting use case, but we never quite found the time to build it. (I actually made an attempt last spring, but, embarrassingly, it didn't work.)

So now, hopefully, it's clear why we were happy about celld: Here was an implementation of Workers and Durable Objects designed to be fully compatible with our own, while being focused on being self-hostable and scalable. That's what we'd wanted to build for a while, and Deno Land was building it for us!

And who better to do it than Ryan Dahl and co? Ryan is the creator of Node.js. He invented the idea of using JavaScript on the server — without his work, Workers may never have existed. And then, he, Bert Belder, and the rest of the Deno Land crew did it again, creating Deno, another JavaScript (and TypeScript) runtime that further advanced the state of the art. The Deno team understands how to create a good developer experience around a self-hosted runtime — something we never quite cracked with workerd.

Imagine, then, our excitement when the Deno team told us they'd like to join forces on this project.

The Plan

Ryan and Bert will be leading a new effort to make workerd self-hosting a first-class supported way to build and run apps using the Workers programming model. This will involve merging code and ideas from celld back into workerd. I'm incredibly excited for this work — I will personally be using it to host an instance of Cloudflare OS in my home.

We'll have more announcements about this work in the coming months. But if you can't wait, you can start self-hosting celld or workerd today.