Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

comments (10)

• This is the right way to deliver software.

  Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs.

  In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

  lpapez 1d ago

• This is why system programming still matters.

  Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.

  irdc 1d ago

• With my own MaraDNS, I aggressively optimized the memory usage of blacklist entries by having a single really big malloc() to allocate the memory for the entries, then traversing that memory block for potentially blacklisted entries.

  When I was using one malloc() per entry, a large blacklist took up 237 megabytes of memory. The same blacklist, once optimized to be loaded with a single malloc() call, only took up 9.5 megabytes of memory.

  https://samboy.github.io/blog/entries/MaraDNS.html#BlogEntry...

  strenholme 1d ago

• These seem like some fairly standard approaches for reducing memory usage. I can't help to think that the approach of joining several distinct list into a single one in some way undercuts Rust's safety guarantees.

  If you previous had three distinct Vec objects, then Rust would guarantee that you can't index out of bounds. If you now put all those objects into a single Vec and rely on offsets, then you now open the door to indexing out of range of these sub-slices without any panics.

  It's a minor point, and it doesn't really invalidate the optimization, but I'm surprised the article didn't mention it.

  vinkelhake 1d ago

• This reminds me how you can save a bunch of bytes just by making sure your structs are aligned. In go for example:

    type Wasteful struct {
      a int16
      b int
      c byte
    }
  
    type Aligned struct {
      b int
      a int16
      c byte
    }
  
  

  Will have sizes of 24bytes and 16bytes (on a 64bit system). Same data 8bytes more. If you are storing millions of those objects, then it adds up.

  grep_it 1d ago

• One of my proudest professional moments was when me and three others managed to reduce memory load of the game Wavetale from 20+GiB to under 3GiB so we could port it to Nintendo Switch.

  The 100 TiB number almost gives me vertigo. Though in this context it was "just" 50%

  Agentlien 1d ago

• Not sure what they use to hold the cache key and entry. If a hashmap is used, then a radix tree (adaptive radix tree) would be better in saving memory space. Most of content of the qname field of the CacheKey is hostname, like www.site.com. The reverse version com.site.www fits nicely in navigation path of a radix tree. The common prefixes like "com." are shared and compressed in the parent nodes of the tree.

  Even a BTree with compressed prefix keys can save space in the qname.

  ww520 1d ago

• Funny thing about cloudflare. I have a dns warming script that uses their top 1k or 10k addresses. Then when my master starts up it warms the entire cache. Everything else uses memcache so the cluster is nice and toasty. As far as I can tell no one else releases domain statistics like them.

  BikiniPrince 1d ago

• General theme: A programming language's native in-memory object format is typically optimized for random access, uniformity, and mutability (fields at fixed offsets, etc). Serialization formats for network or disk tend to be designed explicitly to be more compact. But you can design your own in-memory representation too, with the properties you need.

  edflsafoiewq 1d ago

• It's weird that it took so long for these trivial optimizations but it might just be that they were working on optimizing other stuff.

  0xAstro 1d ago