Go for real-time applications

Some say Go isn't suitable for real-time applications because Go's garbage collector gets in the way of delivering real-time responses. Is this true?

A seasoned engineer answers this question with the seasoned engineer's standard answer:

It depends.

And then the seasoned engineer will ask back:

What kind of “real-time” are you talking about?

What is “real-time”?

The term “real-time application” means different things to different people.

• Some expect zero latency.
• Some need a form of processing load synchronously.
• Some want a steady stream of information with no pause.
• And some require that a certain operation must occur within a specified deadline.

The last case needs a closer look. How long is this deadline? Are we talking about milliseconds? Seconds? Or even hours or days? (These questions make clear that saying “real-time” does not necessarily mean “superfast”.)

Then we have to distinguish between three different levels of “real-time”:

• Soft real-time operations may miss a deadline, but the result remains useful.
• Firm real-time operations may occasionally miss a deadline, but not too many, or the quality of service degrades.
• Hard real-time operations must meet the deadline or the service fails completely.

Obviously, we don't need to talk about soft real-time systems with deadlines of one day. But what about timings where garbage collection activities might get in the way?

Go's garbage collector

Go's garbage collector cleans up unused memory in batches. Every cleanup cycle includes a brief stop-the-world pause when the GC transitions between two phases, and increased CPU load while the GC works concurrently in the mark or the sweep phase. In summary, each cycle delays the rest of the application. The garbage collector has a few knobs to turn for fine-tuning its behavior, but a certain delay is unavoidable.

For operations with a hard deadline in the milliseconds range, this delay could already result in missing a deadline. Go certainly isn't the right choice for this kind of high-speed, short-deadline applications. Prepare to enter the space of tedious manual memory management with GC-less languages. For softer requirements, however, Go may actually be a good choice.

Moreover, the border between suitable and not suitable is not fixed. It can be moved by reducing the number of allocations in the hot path of the application, even down to zero allocations.

Zero-allocation Go

If you don't want the garbage collector to get in the way, ensure it has nothing to collect. Go code can be designed for low- or zero-allocation with a few techniques, including the following:

• Re-use slices, arrays, and buffers. For example, declare a slice outside the hot path and pre-allocate the maximum capacity needed by the app. Inside the hot path, re-use the slice instead of throwing it away and creating a new one.
• Use pass-by-value where possible. If you pass an object to a function by value, the function call only needs to use memory on the stack. If you pass a pointer to a function, the object the pointer refers to might “escape to the heap” and create memory that needs to be garbage-collected. The same happens if a function creates a new object and returns a pointer to it. So do not use pointers “for performance reason” but only if the application logic absolutely requires the use of a pointer.
• Avoid function literals as they can escape to the heap in certain situations.
• Use sync.Pool. Resources, especially external ones, are expensive to instantiate, not only in terms of memory consumption. Pooling them makes them re-usable, even across goroutines.
• Use the unique package for eliminating duplicate data. The unique package stores only one copy of a value in memory. If the application wants to store the same value again, unique creates a reference to the unique copy instead. This technique not only reduces allocations and memory usage but also enables the GC to clean up unique data in a single cycle.
• Be aware of hidden pointers. For example, slices and interfaces contain an internal pointer. And time.Time contains a pointer to location information. All these may cause heap allocations.

How to determine allocation

You shouldn't apply the above techniques blindly. A few tests and measurements help reveal memory usage and allocations.

• Measure memory allocations with go test -bench .
• See which objects escape to the heap: go build -gcflags '-m' (or go build -gcflags=-m=3 for more contextual information)
• Create a CPU profile and look for GC activity in the profile, such as runtime.gcBgMarkWorker, runtime.mallocgc, or runtime.gcAssistAlloc.
• If CPU costs turn out to be significant, create a memory profile and inspect the profile's alloc_space view to identify allocation hot spots.

The above tools and techniques are a good start into writing real-time applications in Go. The Guide to the Go Garbage Collector is my recommended read for digging deeper into this topic.

When do you write your first real-time Go app? (Don't say, “it depends.”)