TL;DR
Cloudflare freed up roughly 100 TB of memory by cutting the per-entry footprint of its DNS cache by more than half, while actually making the cache faster.
Cloudflare's DNS service, Big Pineapple, caches over 250 billion entries at any time. One extra byte per entry costs the whole fleet 250 GB of memory.
Five optimizations cut the per-entry footprint from 953 bytes to 420 bytes, saving roughly as much RAM as 130 servers hold. The cache also got faster: insert throughput rose 43 percent and lookup latency fell 19 percent.
Don't leave room for what you'll never use
Cached entries are never modified after they're stored, but the old code used Vec to hold data, which carries a capacity field and reserves spare space. Switching to Box removed both.
Each entry stored eight such fields, saving 64 bytes per entry. Packing several booleans into a single bitflag also trimmed away padding bytes.
Who needs a copy of the owner name?
Every DNS record carries an owner name, but in most cases the owner is just the queried domain. The engineers stopped storing it, reconstructing it from the cache key at read time and avoiding a heap allocation.
Only records behind a CNAME keep the full owner name, and those are rare.
Big enums are a memory sinkhole
The record type was an enum, and an enum is as big as its largest variant. NAPTR records take 144 bytes, while an A record needs only 4. A and AAAA make up 80 percent of traffic, so most records wasted over 120 bytes.
Boxing the large variants moved them to the heap, leaving small types inline; each A record immediately saved 120 bytes.
Just use the wire format
Enum boxing still costs. The final move was to store record data as raw bytes: a 2-byte length prefix plus the serialized record, packed into a single buffer per entry.
This cuts allocations, improves CPU cache locality, and lets most record types be copied straight into the response without reserializing.
After the rollout, per-entry memory dropped 56 percent and the fleet saved 100 TB. The freed memory is now slated to expand cache capacity and improve hit rates.
In one line: saving memory isn't about being cheap — it's about making room for more cache.
Curated from high-quality sources, with concise summaries and key takeaways.
