Cloudflare freed up 100TB of RAM behind its 1.1.1.1 DNS without adding a single server - TechSpot
But DNS records do not need to expand after they enter the cache. Engineers also rethought how DNS responses are laid out in memory. With the RAM savings, Cloudflare plans to allocate the memory to larger DNS caches.
IN A NUTSHELL: Cloudflare has cut the memory used by the DNS cache behind its 1.1.1.1 resolver by more than half, freeing about 100TB of RAM across its network. The company did not replace server hardware or remove memory modules. Instead, engineers changed how cached DNS data is stored in software.
The optimization work was centered on Big Pineapple, Cloudflare's DNS caching platform behind the popular 1.1.1.1 resolver, DNS Firewall, and several other DNS services they offer. In a technical blog post, systems engineer Sebastiaan Neuteboom detailed five changes to the Rust codebase that shrank a typical cache entry from 953 bytes to 420 bytes.
The changes also made the cache faster. Cloudflare reported that insert throughput rose from 625,000 to 893,000 entries per second, while lookup latency fell from 828 nanoseconds to 670 nanoseconds. During the rollout, from mid-May to early July, p99 memory use per instance dropped from 9.3GB to 5.3GB.
Cloudflare says Big Pineapple holds more than 250 billion DNS entries at any given time, and at that scale even small amounts of wasted memory add up fast. The company estimates that a single unnecessary byte in each cache entry costs more than 250GB across its fleet.
Big Pineapple, Cloudflare's DNS caching platform, was redesigned in Rust to free about 100TB of RAM.
One of the biggest changes involved Rust's Vec and String types. Those containers are designed to grow, so they store capacity data in addition to the data itself. But DNS records do not need to expand after they enter the cache. Cloudflare replaced those structures with fixed-size boxed slices, eliminating metadata that was no longer useful. The company said that change alone saved more than 15TB of RAM.
Engineers also rethought how DNS responses are laid out in memory. Big Pineapple used to store three separate record lists per response. The new design puts the records in one buffer and identifies them with two-byte offsets.
The cache no longer retains owner names when they simply repeat the queried domain. Cloudflare rebuilds those names when it serves the response. It also now stores record data as length-prefixed raw wire-format bytes. That change addresses a mismatch in the earlier design. A four-byte IPv4 A record occupied the same 144 bytes as the largest DNS record type Cloudflare caches, the relatively uncommon NAPTR record. Storing raw data in a variable-length format reduces that waste.
The 100TB of RAM Cloudflare recovered across its fleet is roughly equal to the memory in 130 of its Gen 13 servers, each of which carries 768GB of DDR5-6400. The company had considered a 1,152GB configuration for that server generation but passed on it after weighing the cost of higher-capacity RAM.
With the RAM savings, Cloudflare plans to allocate the memory to larger DNS caches. Keeping more records locally should improve cache hit rates and reduce the traffic it sends to authoritative DNS servers.
This project follows another major memory-efficiency effort completed last year, when Cloudflare rewrote its FL2 request-handling layer in Rust. That's probably a good way to spend resources and squeeze more out of its existing infrastructure as server memory gets pricier.