Powerful, programmable CI/CD engine that runs your pipelines in containers — pre-push on your local machine and/or post-push in CI
A few weeks ago, we launched Dagger Cloud v3, a completely new user interface for Dagger Cloud. One of the main differences between v3 and its v2 predecessor is that the new UI is written in WebAssembly (WASM) using Go. At first glance, this might seem an odd choice - Go typically isn't the first language you think of when deciding to program a Web UI - but we had good reasons. In this blog post, I'll explain why we chose WebAssembly, some of our implementation challenges (and how we worked around them), and the results.
Two Codebases = More Work, Fewer Features
Dagger works by building up a DAG of operations and evaluating them, often in parallel. By nature, this is a difficult thing to display. To help users make sense of it, we offer two real-time visualization interfaces: the Dagger terminal UI (TUI), included in the Dagger CLI, and Dagger Cloud, an online Web dashboard. The Dagger TUI is implemented in Go, and Dagger Cloud (pre-v3) was written in React.
Obviously, we want both user interfaces to be as close to each other as possible. But the actual act of interpreting Dagger's event stream in real-time and producing a UI is pretty involved. Some of the more complex event streams we've seen have hundreds of thousands of OpenTelemetry spans, and managing the data structures around them gets very complicated, very quickly. The Web UI often couldn't keep up with the huge volume of data it had to process and it would become laggy and slow; to fix this performance bottleneck, we were forced into a different implementation model for the React application.
So, we ended up with two interfaces trying to accomplish the same thing, one of them in one language and ecosystem (TypeScript/React), the other in a totally different language and ecosystem (Go), and we couldn't easily share business logic between them. As a small team, we need to ship fast. Having to re-implement every feature twice was just a massive tax on our velocity.
We started thinking about a new approach to Dagger Cloud, with two main goals:
Unify the codebases, to eliminate duplication and make it more efficient to ship new features
Deliver on the promise of a crisp, snappy Web UI, matching the speed and performance of the terminal UI
Choosing Go + WebAssembly
Our starting goal was to be able to reuse one codebase for both Dagger Cloud and the TUI. We decided fairly early to make it a Go codebase. Technically, we could have gone the other way and used TypeScript for the TUI. But we're primarily a team of Go engineers, so selecting Go made it easier for others in the team to contribute, to add a feature or drop in for a few hours to help debug an issue. In addition to standardizing on a single language, it gave us flexibility and broke down silos in our team.
Once we decided to run Go code directly in the browser, WebAssembly was the logical next step. But there were still a couple of challenges:
The Go + WebAssembly combination is still not as mature as React and other JavaScript frameworks. There are no ready-made component libraries to pull from, the developer tooling isn't as rich, and so on. We knew that we would need to build most of our UI components from scratch.
There is a hard 2 GB memory limit for WebAssembly applications in most browsers. We expected this to be a problem when viewing large traces, and we knew we would have to do a lot of optimization to minimize memory usage and keep the UI stable. This wasn't entirely bad though; the silver lining here was that any memory usage improvements made to the WebAssembly UI would also benefit TUI users, since it was now a shared codebase.
De-Risking the Project
Once we'd made the decision, the next question was, "how do we build this?" We decided to build the new WebAssembly-based UI in the Go-app framework. Go-app is a high-level framework specifically for Progressive Web Apps (PWAs) in WebAssembly. It offers key Go benefits, like fast compilation and native static typing, and it also follows a component-based UI model, like React, which made the transition easier.
Since the Go + WebAssembly combination isn't mainstream, there was some healthy skepticism within the Dagger team about its feasibility. For example, there was no real ecosystem for Go-app UI components and we knew we’d have to write our own, but we weren’t sure how easy or difficult this would be. We also had concerns over integrations with other services (Tailwind, Auth0, Intercom, PostHog), and about rendering many hundreds of live-updating components at the same time.
To answer these questions and de-risk the project, I spent almost a month prototyping, with the goal of re-implementing as much of the existing UI as possible in Go-app. As it turned out, there weren't many blockers: WebAssembly is already a well-documented open standard and most other questions were answered in Go-app’s own documentation. The biggest challenge, as expected, was the memory usage limit, which required careful design and optimization.
From Prototype to Production
Once we had a working proof of concept, the team's comfort level increased significantly and we kicked off project "awesome wasm" to deliver a production implementation. Here are a few notes from the journey:
Memory usage was easily the most existential threat to the project’s success. I spent a lot of time figuring out how to render 200k+ lines of log output without crashing. This led to optimizations deep in our virtual terminal rendering library, which dramatically reduced TUI memory usage at the same time (as mentioned already, sharing codebases means that important optimizations in one interface become "free" in the other!)
Go WASM is slow at parsing large amounts of JSON, which led to dramatic architecture changes and the creation of a “smart backend” for incremental data loading over WebSockets, using Go's rarely-used encoding/gob format.
Initially, the WASM file was around 32 MB. By applying Brotli compression, we were able to bring it down to around 4.6 MB. We tried to perform Brotli compression on-the-fly in our CDN but the file was too large, so eventually we just included the compression step into our build process.
Apart from the memory challenges, most of our other initial worries turned out unfounded. The UI components weren’t very hard to write, integrations with other services were straightforward, and I found good techniques for handling component updates in real-time.
There were a number of useful NPM packages I found, so I wondered if I could use them with Go. WebAssembly has a straightforward interface to both Go and JavaScript, so I built a Dagger module that uses Browserify to load an NPM package. This module allows us to generate a JavaScript file that can be included in a Go application. This means that we can work primarily in Go and then, if needed, we have a way to load helpers that are implemented in native JavaScript.
Disclaimer: I'm not a React professional so with that in mind...it seemed to me that React had a very rigid way of implementing components, while Go-app was much more flexible. In Go-app, you can have any component update whenever you like, which gives you many more degrees of freedom for optimization. For example, I needed to optimize a component rendering 150,000+ lines of output. Just having the ability to try different approaches and then pick the one that worked best, made the entire exercise much easier!
Even though Go-app doesn't have React-like developer tools built into the browser, I was able to use Go's own tools (pprof) plus the default profiler built into the browser for profiling and debugging. This was very useful to inspect functions calls, track CPU and memory usage, and evaluate the effectiveness of different approaches for optimizing memory usage.
I discovered a side benefit of using Go-app: since Dagger Cloud is built as a PWA, it can be installed as a desktop or a mobile application. This makes it possible to launch Dagger Cloud like a native application and get a full-screen experience without needing to open a browser first, or just have a dedicated icon in your desktop taskbar/dock.
We soft-launched Dagger Cloud v3 to our Dagger Commanders a few weeks ago to collect feedback and made it available to everyone shortly thereafter.
Benefits
Our switch from React to WASM has resulted in a more consistent user experience across all Dagger interfaces, and better overall performance and lower memory usage, especially when rendering large and complex traces.
From an engineering perspective too, the benefits to our team are significant. Optimizations very often involve just as much, if not more, work than actually implementing features. So it's great to not have to spend time optimizing the Web UI, and then more time optimizing the TUI, and instead actually focus on delivering new features.
Should You Do This?
Dagger Cloud v3 has the Dagger community buzzing and one of the more common questions we've been fielding recently is: who should consider doing this and who shouldn't?
We want to be clear that we're not generally recommending making front-ends in Go. We had some very good reasons to do it: a team of strong Go engineers; a complex UI that TypeScript/React didn't scale well for; a requirement for standardization and reuse between two codebases; and a company-wide mandate to increase our velocity. That's a fairly specific set of circumstances. If you're in similar circumstances, this is certainly an option worth evaluating; if not, there are other tools and standards that you should consider first.
Dagger Cloud v3 is still in beta and we're excited for you to try it out. If you'd like to know more about our implementation or simply have feedback to share on the new UI, join our Discord and let us know what you think!
Read the original article
Comments
Early in the post:
> As a small team, we need to ship fast.
So they chose a solution that required them to:
> Spent almost a month prototyping
> there was no real ecosystem for Go-app UI components and we knew we’d have to write our own
> Go WASM is slow at parsing large amounts of JSON, which led to dramatic architecture
Not sure what their definition of shipping fast is, since this just sounds like resume oriented programming.
By vito 2025-02-1116:07 What you aren't including is the chronic cost of maintaining the status quo, which is the entire reason for making the change. Sometimes making no decision costs you more than making a risky one - and if the decision is risky, you'll want to investigate it first.
During that investigation you discover things (like having to implement your own components) and try to account for how much they'll subtract from the time and energy you save. If the calculus works out, you keep going. If it doesn't, you stop. We kept going, and now we can ship complex features and optimizations to both UIs within a day or two.
> I spent a lot of time figuring out how to render 200k+ lines of log output without crashing. This led to optimizations deep in our virtual terminal rendering library...
Interesting. I have two wildly different takes on this.
1) Dagger is such an interesting company that they let developers do whatever they want, as long as it works and works well. A good mix of pragmatism and fun.
2) Holy crap, it's a real company with real customers. Why would they attempt something like this in one of their key products, where are the adults?!
The adults would make a quarterly plan, schedule 16 hours of meeting each week, 4x the head count and put everyone into a silo. And then nothing gets done except next quarter.
Is it? What if instead it was another “We did a full rewrite and it was a mistake. Here is why” type of postmortem?
I could have been, but it wasn't so far. And even if it was, there is a lot of things to learn in such mistakes. Audacity of doing the ambitious thing and the newly gained experience of why not doing it is how we get experienced people in the industry. Otherwise it's just dogma and nobody breaks past the established practices, which are always suboptimal.
Sure, if you're like uber at the peak of over-hiring and have hundreds of engineers more than you actually need. Then go ahead and try to reinvent things and hopefully you'll be able to improve the status quo.
But if you claim to be a small team that needs to ship fast, I believe you should just pick the industry standard boring technology and grind away.
Exactly. People only look at success.
The question is to fully see (and boringly like the parent post was talking about) what are the rewards and the risks to decide if it is worth it or not. What if it made the company tanked financially (like you would have to go back to the state before but have to rewrite the rewrite because you had to change the database design in your first rewrite or something) and not be able to ship for another 6 months? Being a small team you don't have all the cash you want unless you already made it big. Also you have to explain to your investors why you won't ship. And really how come you are an expert in tech when you had to redo it from scratch so next time maybe they can reinvest.
This type of decision needs clear planning, reward and risk, and a real business goal to happen. But let's be real many times it's not the case and X is the new cool tech. I am all about being audacious but in a calculated way.
By Muromec 2025-02-1114:26 To me what they did looks exactly that -- being audacious in a calculated way that benefited them.
By andrewstuart 2025-02-117:34 What could this company possibly doing that can't be handled by react or Vue or Vanilla or whatever the myriad other ones are?
By pjmlp 2025-02-118:56 Definitely, if there as a gold lesson I had with WebForms and JSF, was that frameworks that abstract the way that browser APIs work, without debugging tool for those additional cake layers are not what I want to spend my time on, unless I am told that is the way by higher ups.
It sounds like people having fun, which means they would care more and ship faster if it works out. If it doesn't work out, they will do something boring.
Why hate on people actually enjoying the thing at times when everyone with the boring stack is one round of layoff away from the door?
By pwdisswordfishz 2025-02-116:532 reply Why call every little disagreement and pointing out of inconsistency "hate"?
By Muromec 2025-02-118:40 Disagreement on values, especially coupled with outright dismissal falls into the hate category for me.
By politelemon 2025-02-117:221 reply The comment in GP is not a 'disagreement', it is just bad faith.
"Not sure what their definition of shipping fast is, since this just sounds like resume oriented programming."
By pwdisswordfishz 2025-02-118:39 It's not allowed to disbelieve someone's claims, especially given evidence to the contrary?
By groby_b 2025-02-1119:46 I mean, with selective quoting you can make any point you want. Let's do the full quote:
> So, we ended up with two interfaces trying to accomplish the same thing, one of them in one language and ecosystem (TypeScript/React), the other in a totally different language and ecosystem (Go), and we couldn't easily share business logic between them.
> As a small team, we need to ship fast. Having to re-implement every feature twice was just a massive tax on our velocity.
Oh. Wait. They're cutting away an entire code base and a second language & ecosystem to support. Maybe it's not quite as resume-oriented after all.
We can certainly argue if that was the right way to do this, but it's not like it's an entirely unreasonable resume padding exercise.
By hnlurker22 2025-02-117:303 reply It simply means that, for their purposes, React wasn't good enough. Personally, I applaud any effort that doesn't use React.
By hnlurker22 2025-02-117:501 reply >We had some very good reasons to do it: a team of strong Go engineers; a complex UI that TypeScript/React didn't scale well for
By yieldcrv 2025-02-1112:01 Because its a bullshit assessment, they ran into the same limitations in their obscure choice, they didn’t move fast and break things, and both choices have optimizations to deal with these problems
and we would all fail system design interviews for considering the same thing
By jbreckmckye 2025-02-1114:161 reply > Personally, I applaud any effort that doesn't use React.
Why?
By hnlurker22 2025-02-1116:40 There needs to be competition
>We had some very good reasons to do it: a team of strong Go engineers; a complex UI that TypeScript/React didn't scale well for;
After going though demo a bit, I can see this is what they wanted to say here: we have a team of strong Go engineers who are not that strong when it comes to web frontend, the smells of it are everywhere. The signup page don't fit in the screen and have empty scrollable space around it. Every click inside the dashboard is a spinner on the whole screen with waiting for page reload. Even icons are showing their alt-texts while loading. These are all smells of just low-experience web application, but it's relieving to say React didn't scale
By hnlurker22 2025-02-117:541 reply Web developers without mobile experience have been using React Native to create smelly mobile apps for a long time now, and it's still growing
By gloosx 2025-02-117:56 Mobile experience is two distinct Java/ObjC worlds, so creating smelly apps on web technologies is justified by that.
> a complex UI that TypeScript/React didn't scale well for
You can go a long way with render optimizing on the web. Even some basic virtualization should allow for showing millions of log entries. Or, you could still use web technologies (OffscreenCanvas), a bit weird to go directly with WebAssembly.
Exactly, it's hard to imagine a UI so complex it would "not scale" in the browser.
Not looking like they are making Concorde cockpit or something.
I mean, the browser itself was made to display complex UI elements, and is optimized for displaying interactive dashboards (at least it tried to in the last 20 years).
Plus, computers are fast enough, if something is somehow still slow, you can still make it seem fast if you add proper transitions and fix the timing of animations and loading order. For example, maybe one log file is 10GB, of course the browser will crash if you try to load all of it in the browser. Simply preload the latest 1000 lines, then use websockets to load new data as the user scrolls. Now, if you scroll first and then load the data, the app will seem slow, the trick is to start loading the data when the scroll already reached 90% of current buffer. Anyway, tricks like this will make any app seem instant, even if things still actually take time to load.
By gloosx 2025-02-1215:07 I sometimes remember what mad tricks game developers did in the late 90s and early 00s to optimize their games and all that "web doesn't scale" crap sounds just funny to me ;)
I can hear these Gophers in the meeting complaining how TypeScript is „completely unreadable”.
By notpushkin 2025-02-114:104 reply I was slightly worried it would be one of those “render to canvas” frameworks (which are an accessibility nightmare and are ever so slightly broken), but that’s not the case here! Looks like WASM – DOM interop is finally fast enough for this to work.
32 MB binary is a big no-no though.
Not a fan of this bundle size, but it's not a web store landing page — it's a professional application, and it's users are likely to use it extensively through their day, and all hundreds of navigation events and new tabs will (if handled correctly) hit the same cached bundle.
By notpushkin 2025-02-117:121 reply It’s a good point, but keep in mind that’s 32 MB of code: you can cache it, but you still have to parse and initialize it for every tab. (It might not be as bad as it sounds, WASM is lighter in this aspect than JS; but it’s still a good proxy for startup performance IMO.)
It should work well for applications with longer session size, like Figma: you open a project one time and work on it for hours. I’m not sure if that’s the case with Dagger Cloud: you might be looking at a single trace, or compare dozens of builds throughout the day. The post only claims “better overall performance and lower memory usage”, but without any numbers it’s hard to tell anything.
By gedw99 2025-02-168:32 I use golang WASM and use web workers and shared workers with a service worker.
This means that a new tab does not reload the whole thing each time.
Once you get it going it’s easy to build large complex systems.
By whitehexagon 2025-02-116:132 reply I also struggled with the binary output size of Golang WASM, and that it was growing with each version. Someone recommended TinyGo, and although I haven't needed it for a couple years now, at the time the WASM binary output size was much better.
Now I am settled on Zig/WASM, which gets me pretty close to C/WASM combo i.e. tiny. The JS bridge has some overhead, but I've rewritten most of that and it works good enough for canvas 2D/3D or even DOM. It would be nice to ditch the JS bits, and I suspect the compositors all-the-way down are probably hitting my 2D performance.
By notpushkin 2025-02-117:172 reply I’ve been looking at Zig for some time, too. Perhaps I should give it a go!
Any particular tricks on making bridged DOM fast?
By qvq 2025-02-1112:28 By whitehexagon 2025-02-1114:57 I use a custom API on the JS side, and making the most of the shared memory for strings. For widget creation within DOM I have small API wrappers on the JS side to avoid too much bridge crossing with attribute sets/gets.
For 2D/3D moving the canvas animation loop to the JS side helped.
So far the html (inc custom JS API) comes in at 20KB, and my wasm around 80KB (SVG based UI with a small embedded font).
Not sure how the latest Zig compares, but I'd be quite happy stuck on this older version for this project because it seems very stable.
By J_Shelby_J 2025-02-116:171 reply i swear, some people hate rust as much as the comment sections claim rust people are fanatics
By aledalgrande 2025-02-116:19 Figma is much more than that and lots of people use it daily
By theanonymousone 2025-02-117:00 Coincidentally, I just checked the LinkedIn page of a colleague, and Vivaldi said it was loading ~30MB of data for that.
By nicoburns 2025-02-114:53 Yeah, but with most JavaScript frameworks (and even Rust compiled to WASM), 4MB (at most!) would be the size of the uncompressed bundle.
By WhyNotHugo 2025-02-1110:03 I had the same worry: that they'd be rendering everything into a canvas. It's very neat to see that the WASM<>DOM interop is there now, and they render actual native elements. Even inspecting the DOM it's quite readable (far easier to read that I've seen React produce).
OTOH, Rust solutions have been pretty disappointing so far; usually relying on macros (which are a pain to write, debug or maintain in any way) and lots of DSLs (which negate the value of any development tools).
The binary size for this approach is a big issue, but assuming that it can be decreased, I'd think that go can end up being a very viable candidate in this space.
